My OperaDigital Ciphers
Thursday, July 29, 2010 2:22:23 AM
This is an open discussion on digital ciphers. Speculations abound about the cipher used in K4. One of the ideas that keeps coming back is that K4 uses a digital cipher based on the Morse segment DIGETAL INTERPRETATION. Numerical representations can come in many different formats. Here's a list of some of the ways we can write English using numbers.
That's just SOME of what you can do. All of these technique seem to encrypt, but they don't actually do that (or are very weak form of encryption in most cases). Well, Baconian is stegonagraphy. But in each case, the actual format is simply a conversion process from English to numbers. Think of it this way. Is Morse code encryption? Not really. Sure, it makes the actual English letters invisible. But it's just using a different set of symbols for the exact same alphabet. Once you recognize the symbols, you can read it as if it was plain English because that's what it actually is.
So all those methods up there are basically using numbers as the symbols for each letter instead of ABC's. Are they substitution ciphers? YES! They are very much like monoalphabetic ciphers. But some are a little more complex like Homophonic, straddling checkerboard and a few others.
The nice thing about converting text to numbers is that they become much more difficult for humans to interpret or see patterns. With letters, we can manipulate that in our minds and we can see duplicate letters without too much touble. Then take a look at numbers. It's much harder. Not only because we're using digits, but because these digits will have a different amount of symbols in its own alphabet. For example, the English alphabet has 26 unique and distinct symbols (A-Z). But decimal digits have only 10 unique and distinct symbols (0-9). So we can't do a one to one translation. We need to come up with some kind of scheme to not only be able to represent all the letters, but also be able to tell where one letter starts and where it ends. The easiest way is to say we'll use a fixed amount of digits per character. Using 01 to 26, we can represent all the symbols. But not seeing 00 or anything about 26 would give it away. No way are we going to see this in Kryptos.
The point of all this is that K4 uses letters. So if it truly uses a digital cipher, then we need to somehow convert these letters to digits. It's not so simple. How do we do that? We talked about digital representations, but we can also use binary representations which is a form of digital encoding. We could convert each letter to Morse code and use the dits and dashes as 1's and 0's. We could merge all the dits and dashes together and then create groups of 6 symbols. Why 6? Because that would allow us to represent up to 32 different characters. Trouble is that K4 has 299 Morse symbols when converted. If we divide 299 by 6, we get almost 50 characters. That's HALF of what Sanborn said we should be getting (approx one English letter for each ciphertext letter).
One clue we do have is that Scheidt has said to even out the 1's and 0's. So we're definitely dealing with some kind of binary representation. Perhaps we can rule out decimal digits. Well, at the very least we can make a fairly good assumption that at some point a binary representation will be used even if you can convert them to digits later on.
Digits tend to have a common feature that they can represent powers of two. For example, DYAHR, if interpreted as 5 binary digits where the raised letters are 1's and the lowered letters are 0's, we would get the number 01101 which is 13 in decimal. 5 binary digits can represent 32 different numbers (or characters). Each extra digit will double that. Also, if the raised letters are 0's instead of 1's, then we have the number 18. As such, binary numbers have opposites (or complements).
Baudot and Murray code is the most obvious binary encoding that has a fixed amount of digits. Variable amounts of digits is easily done with Morse code. The thing with binary is that you can also compress it rather easily. There is a simple technique called Huffman coding. This is great because it almost completely removes any hint of English distribution. In fact, that's how it can achieve compression. Some letters are used more often and will use fewer bits in the coding. Letters that are rarely used will lots of digits, but since they happen so rarely, it will not affect the size of the whole text. It will actually help by allowing fewer bits for the most used characters as just mentioned.
Huffman coding seems like a great tool. If we have 299 binary digits and we can only get 50 characters with a direct conversion, maybe they are compressed and that the uncompressed version would yield 100 characters. Unfortunately, that would be a 50% compression rate. That's a LOT! However, it means that it's using 3 digits on average per character. That's not unrealistic. Problem is that Morse code is already optimized to some degree to use as few symbols as possible for the most used letters. Huffman coding can only do better if the distribution is MORE skewed than regular English. Quite possible with short text, but I'm not holding my breath. Huffman coding needs something else. A mapping between the letters and the compressed binary pattern.
So there are many ways to convert to digits and many issues to deal with. Beyond that, we can apply an actual cipher to the data. We can take the digits or symbols and do a transposition or further substitution. This would really make it difficult to break even though technically it isn't superencoded.
Then there's still yet another possibility. The digits could be part of the key. There are tons of ciphers that use digits as the key. I won't go through them here. But I should mention things like Lagged Fibonacci Generator, XOR cipher and one time key/pads. Gronsfeld and variants are also cool.
I am starting to really believe that K4 isn't an existing cipher. I believe it is custom made though I believe that one of the steps is pre-existing, but that this isn't really part of the original cipher for K4. What the heck am I talking about?
I am going to reveal what I think K4 is, but not the actual details of the second part of the cipher. I believe that K4 is an end to end cipher rather than a point to point cipher. What's the difference? Well, a point to point cipher is where the transmission itself automatically encrypts everything. But at each leg of the transmission (if there is more than one leg), it must decode and re-encode your message. So there is a weakness at the connection points. An end to end cipher doesn't work that way. It is encrypted before being sent and only the recipient at the end of the transmission can decode it. BTW, end to end encryption is how it's usually done.
If this is all normal, then what's so special about pointing out the run of the mill end to end cipher. Well, two things. First, we have the word END at the beginning of K3. This is not so unusual when you consider end to end ciphers because the two ends are not only at the destination, but also at the source. So BOTH the beginning and end are end points. The second point is discussed later about a message in a message.
We don't see the other end point right now. Here is what I think. I think K1 and K2 use the same cipher. I also believe that K3 and K4 use the same cipher. In each case, they would use different keys. Well, K4 doesn't have an English distribution so using a double rotation won't to a lick of good. Or will it?
I think what's going on is that K4 is a message sent by WW. A third party from the two people talking in K2. I still believe that K3 comes before K2, but it's also possible that K3 is part of the message sent by WW. In either case, it doesn't affect what happened to K4.
What's I think is happening is that two people are talking in K2. Maybe one of them is being tortured and forced to answer questions. WW may have sent him a message, but he could not decode it. So he tells the interviewer what that message is in its original encrypted form. At this point, the ENTIRE discussion is encrypted and this is what we find on the sculpture. So the discussion itself is encrypted using Vigenere (Quagmire III). And the King tut discovery along with WW's message is encrypted using double rotation.
Get it?
The double rotation is not part of WW's original message. The original encrypted message sent by WW is given in the decoded text when the person says "HERE WAS HIS LAST MESSAGE" followed by coordinates. I believe that K4 should follow this and is also part of the message.
Think of it this way. Suppose you discover a secret code that you cannot decipher. Say it reads "My phone number is 903 472 947F ASHUTBCKJEOLATSUETNOUSTHUS". The phone number is bogus even after checking all 10 possible digits. You supect it might a key or indicator of some sort for the message. You then decide to send it to a friend to see if he can decode it. So you send something like "Hi Frank, I have a secret code that I can't decode. Can you have a look? The message is as follows. My phone number is 903 472 947F ASHUTBCKJEOLATSUETNOUSTHUS"
Then your whole message is encoded before sending it to Frank!
This means that the already encrypted text ASHUTBCKJEOLATSUETNOUSTHUS will be encrypted AGAIN when you send it to Frank. This is what I believe happened with K4. I believe it contains the last part of WW's message. Obviously, with K4, the conversation was encrypted using two methods (Quagmire III and Double Rotation).
The issue now is that a double rotation works best with equal number of letters per row. With 97 letters, that's hard to do. I'm working on something where that's not the right amount of letters. After the rotation, there is a digital interpretation involving the masking technique. I have found some really peculiar properties. But I haven't really had time to look much into it. And as discussed here, the possibilities for ciphers are numerous. On top of that, I believe that WW's message used a unique encryption technique never seen or used by anyone. I think Scheidt made up a technique.
So the point I'm trying to say is that we really need to figure out what the other sections are saying. Maybe K3 really is part of WW's message along with the coordinates. If so, it could very be code for the instructions on how to decode K4. This would mean that the person in K2 giving us this information may have sent a duress cipher. Usually duress ciphers are to throw someone off. In this case, it may be to simply throw his captors off. But to let allies know that there is another sub channel with the required information to decode K4.
I believe this is a very realistic scenario. It would go to the whole theme behind CIA and intelligence gathering. It's basically a detective story with agents in the field trying to covertly send you information.
BTW, when I applied the rotations on K4, a few words popped out on the first rotation with one of them at the upper left hand corner, and the word END was revealed after the second rotation at the lower right hand corner. With K3's ciphertext, this gave the word END both at the beginning and end of the text.
I did get one step further. Like I said, we need to figure out what exactly is being said by the plaintext of Kryptos. Is the agent under duress? Is he trying to tell us something? Does he know we are listening in? I'm now fairly confident that UNKNOWN LOCATION means unknown cipher. The interviewer then asks "WHO KNOWS THE EXACT LOCATION?" IOW, who knows the exact cipher? Only WW! And his message follows. We thought it was the coordinates. But I'm saying it included K4 as well. K3 may be part of it too. It could hold the instructions on how to decode K4.
The weird part is that both the interviewer and the responder use the same code. Perhaps the encryption was done by the same person who is leaking this information. Perhaps both people know their conversation is not secure and must use a secondary channel by using codewords. I'm not sure what scenario is the right one. But I think I'm at least in the correct ballpark. I think that whoever encrypted the message found on Kryptos is trying to tell us something about WW's message.
Then again, maybe the people talking in K2 are our adversaries and they know that we can listen in. So the plaintext is simply a duress cipher meant to throw us off. The real message lies in what they were able to send to each other in a subchannel. It's up to us and figure it out.
edit: I like that last idea. WW is on our side. The two people in K2 found out about WW's message and are trying to decipher it. But they know their communication channel is insecure, so they are using code. It's like Spy vs. Spy. They've found a CIA message and we've found them talking about finding our message.
I don't think K2 would have the cipher if the two talking are our enemies. We might conclude that K3 has those clues. But K3 has an X before CAN YOU SEE ANYTIHNG Q?. That X links it to K2. Unless what is going on is that someone is overhearing the conversation and is sending us what he's overhearing. That would make sense. In such a scenario, anything goes. The instructions could be anywhere.
I'm still going with the fact that K2 is code and that the two people are enemies. One of them says "THEY SHOULD" when asked about if Langley knows. Does this not mean that the two are NOT part of the CIA? OTOH, it could mean that he's a CIA agent and not in Langley right now. Still, it seems very suggestive that one person is monitoring traffic to Langley and the other is asking him about it. IOW, enemies. If they were allies, would they not know how to decode WW's message? They are stating specifically that they do not know the LOCATION. Even if you don't agree that LOCATION stands for CIPHER, they are still talking about something they don't know. This does not make sense in the context of K2.
- Morse code (but requires spacings)
- ASCII
- Baudot (and Murray) code.
- Index in alphabet (1-26 or 0-25)
- Straddling Checkerboard
- Nihilist cipher
- Baconian cipher
- BIFID TRIFID (the intermediate stages)
- GRANDPRE[PDF]
- Homophonic
- POLLUX (Requires Morse code)
That's just SOME of what you can do. All of these technique seem to encrypt, but they don't actually do that (or are very weak form of encryption in most cases). Well, Baconian is stegonagraphy. But in each case, the actual format is simply a conversion process from English to numbers. Think of it this way. Is Morse code encryption? Not really. Sure, it makes the actual English letters invisible. But it's just using a different set of symbols for the exact same alphabet. Once you recognize the symbols, you can read it as if it was plain English because that's what it actually is.
So all those methods up there are basically using numbers as the symbols for each letter instead of ABC's. Are they substitution ciphers? YES! They are very much like monoalphabetic ciphers. But some are a little more complex like Homophonic, straddling checkerboard and a few others.
The nice thing about converting text to numbers is that they become much more difficult for humans to interpret or see patterns. With letters, we can manipulate that in our minds and we can see duplicate letters without too much touble. Then take a look at numbers. It's much harder. Not only because we're using digits, but because these digits will have a different amount of symbols in its own alphabet. For example, the English alphabet has 26 unique and distinct symbols (A-Z). But decimal digits have only 10 unique and distinct symbols (0-9). So we can't do a one to one translation. We need to come up with some kind of scheme to not only be able to represent all the letters, but also be able to tell where one letter starts and where it ends. The easiest way is to say we'll use a fixed amount of digits per character. Using 01 to 26, we can represent all the symbols. But not seeing 00 or anything about 26 would give it away. No way are we going to see this in Kryptos.
The point of all this is that K4 uses letters. So if it truly uses a digital cipher, then we need to somehow convert these letters to digits. It's not so simple. How do we do that? We talked about digital representations, but we can also use binary representations which is a form of digital encoding. We could convert each letter to Morse code and use the dits and dashes as 1's and 0's. We could merge all the dits and dashes together and then create groups of 6 symbols. Why 6? Because that would allow us to represent up to 32 different characters. Trouble is that K4 has 299 Morse symbols when converted. If we divide 299 by 6, we get almost 50 characters. That's HALF of what Sanborn said we should be getting (approx one English letter for each ciphertext letter).
One clue we do have is that Scheidt has said to even out the 1's and 0's. So we're definitely dealing with some kind of binary representation. Perhaps we can rule out decimal digits. Well, at the very least we can make a fairly good assumption that at some point a binary representation will be used even if you can convert them to digits later on.
Digits tend to have a common feature that they can represent powers of two. For example, DYAHR, if interpreted as 5 binary digits where the raised letters are 1's and the lowered letters are 0's, we would get the number 01101 which is 13 in decimal. 5 binary digits can represent 32 different numbers (or characters). Each extra digit will double that. Also, if the raised letters are 0's instead of 1's, then we have the number 18. As such, binary numbers have opposites (or complements).
Baudot and Murray code is the most obvious binary encoding that has a fixed amount of digits. Variable amounts of digits is easily done with Morse code. The thing with binary is that you can also compress it rather easily. There is a simple technique called Huffman coding. This is great because it almost completely removes any hint of English distribution. In fact, that's how it can achieve compression. Some letters are used more often and will use fewer bits in the coding. Letters that are rarely used will lots of digits, but since they happen so rarely, it will not affect the size of the whole text. It will actually help by allowing fewer bits for the most used characters as just mentioned.
Huffman coding seems like a great tool. If we have 299 binary digits and we can only get 50 characters with a direct conversion, maybe they are compressed and that the uncompressed version would yield 100 characters. Unfortunately, that would be a 50% compression rate. That's a LOT! However, it means that it's using 3 digits on average per character. That's not unrealistic. Problem is that Morse code is already optimized to some degree to use as few symbols as possible for the most used letters. Huffman coding can only do better if the distribution is MORE skewed than regular English. Quite possible with short text, but I'm not holding my breath. Huffman coding needs something else. A mapping between the letters and the compressed binary pattern.
So there are many ways to convert to digits and many issues to deal with. Beyond that, we can apply an actual cipher to the data. We can take the digits or symbols and do a transposition or further substitution. This would really make it difficult to break even though technically it isn't superencoded.
Then there's still yet another possibility. The digits could be part of the key. There are tons of ciphers that use digits as the key. I won't go through them here. But I should mention things like Lagged Fibonacci Generator, XOR cipher and one time key/pads. Gronsfeld and variants are also cool.
I am starting to really believe that K4 isn't an existing cipher. I believe it is custom made though I believe that one of the steps is pre-existing, but that this isn't really part of the original cipher for K4. What the heck am I talking about?
I am going to reveal what I think K4 is, but not the actual details of the second part of the cipher. I believe that K4 is an end to end cipher rather than a point to point cipher. What's the difference? Well, a point to point cipher is where the transmission itself automatically encrypts everything. But at each leg of the transmission (if there is more than one leg), it must decode and re-encode your message. So there is a weakness at the connection points. An end to end cipher doesn't work that way. It is encrypted before being sent and only the recipient at the end of the transmission can decode it. BTW, end to end encryption is how it's usually done.
If this is all normal, then what's so special about pointing out the run of the mill end to end cipher. Well, two things. First, we have the word END at the beginning of K3. This is not so unusual when you consider end to end ciphers because the two ends are not only at the destination, but also at the source. So BOTH the beginning and end are end points. The second point is discussed later about a message in a message.
We don't see the other end point right now. Here is what I think. I think K1 and K2 use the same cipher. I also believe that K3 and K4 use the same cipher. In each case, they would use different keys. Well, K4 doesn't have an English distribution so using a double rotation won't to a lick of good. Or will it?
I think what's going on is that K4 is a message sent by WW. A third party from the two people talking in K2. I still believe that K3 comes before K2, but it's also possible that K3 is part of the message sent by WW. In either case, it doesn't affect what happened to K4.
What's I think is happening is that two people are talking in K2. Maybe one of them is being tortured and forced to answer questions. WW may have sent him a message, but he could not decode it. So he tells the interviewer what that message is in its original encrypted form. At this point, the ENTIRE discussion is encrypted and this is what we find on the sculpture. So the discussion itself is encrypted using Vigenere (Quagmire III). And the King tut discovery along with WW's message is encrypted using double rotation.
Get it?
The double rotation is not part of WW's original message. The original encrypted message sent by WW is given in the decoded text when the person says "HERE WAS HIS LAST MESSAGE" followed by coordinates. I believe that K4 should follow this and is also part of the message.
Think of it this way. Suppose you discover a secret code that you cannot decipher. Say it reads "My phone number is 903 472 947F ASHUTBCKJEOLATSUETNOUSTHUS". The phone number is bogus even after checking all 10 possible digits. You supect it might a key or indicator of some sort for the message. You then decide to send it to a friend to see if he can decode it. So you send something like "Hi Frank, I have a secret code that I can't decode. Can you have a look? The message is as follows. My phone number is 903 472 947F ASHUTBCKJEOLATSUETNOUSTHUS"
Then your whole message is encoded before sending it to Frank!
This means that the already encrypted text ASHUTBCKJEOLATSUETNOUSTHUS will be encrypted AGAIN when you send it to Frank. This is what I believe happened with K4. I believe it contains the last part of WW's message. Obviously, with K4, the conversation was encrypted using two methods (Quagmire III and Double Rotation).
The issue now is that a double rotation works best with equal number of letters per row. With 97 letters, that's hard to do. I'm working on something where that's not the right amount of letters. After the rotation, there is a digital interpretation involving the masking technique. I have found some really peculiar properties. But I haven't really had time to look much into it. And as discussed here, the possibilities for ciphers are numerous. On top of that, I believe that WW's message used a unique encryption technique never seen or used by anyone. I think Scheidt made up a technique.
So the point I'm trying to say is that we really need to figure out what the other sections are saying. Maybe K3 really is part of WW's message along with the coordinates. If so, it could very be code for the instructions on how to decode K4. This would mean that the person in K2 giving us this information may have sent a duress cipher. Usually duress ciphers are to throw someone off. In this case, it may be to simply throw his captors off. But to let allies know that there is another sub channel with the required information to decode K4.
I believe this is a very realistic scenario. It would go to the whole theme behind CIA and intelligence gathering. It's basically a detective story with agents in the field trying to covertly send you information.
BTW, when I applied the rotations on K4, a few words popped out on the first rotation with one of them at the upper left hand corner, and the word END was revealed after the second rotation at the lower right hand corner. With K3's ciphertext, this gave the word END both at the beginning and end of the text.
I did get one step further. Like I said, we need to figure out what exactly is being said by the plaintext of Kryptos. Is the agent under duress? Is he trying to tell us something? Does he know we are listening in? I'm now fairly confident that UNKNOWN LOCATION means unknown cipher. The interviewer then asks "WHO KNOWS THE EXACT LOCATION?" IOW, who knows the exact cipher? Only WW! And his message follows. We thought it was the coordinates. But I'm saying it included K4 as well. K3 may be part of it too. It could hold the instructions on how to decode K4.
The weird part is that both the interviewer and the responder use the same code. Perhaps the encryption was done by the same person who is leaking this information. Perhaps both people know their conversation is not secure and must use a secondary channel by using codewords. I'm not sure what scenario is the right one. But I think I'm at least in the correct ballpark. I think that whoever encrypted the message found on Kryptos is trying to tell us something about WW's message.
Then again, maybe the people talking in K2 are our adversaries and they know that we can listen in. So the plaintext is simply a duress cipher meant to throw us off. The real message lies in what they were able to send to each other in a subchannel. It's up to us and figure it out.
edit: I like that last idea. WW is on our side. The two people in K2 found out about WW's message and are trying to decipher it. But they know their communication channel is insecure, so they are using code. It's like Spy vs. Spy. They've found a CIA message and we've found them talking about finding our message.
I don't think K2 would have the cipher if the two talking are our enemies. We might conclude that K3 has those clues. But K3 has an X before CAN YOU SEE ANYTIHNG Q?. That X links it to K2. Unless what is going on is that someone is overhearing the conversation and is sending us what he's overhearing. That would make sense. In such a scenario, anything goes. The instructions could be anywhere.
I'm still going with the fact that K2 is code and that the two people are enemies. One of them says "THEY SHOULD" when asked about if Langley knows. Does this not mean that the two are NOT part of the CIA? OTOH, it could mean that he's a CIA agent and not in Langley right now. Still, it seems very suggestive that one person is monitoring traffic to Langley and the other is asking him about it. IOW, enemies. If they were allies, would they not know how to decode WW's message? They are stating specifically that they do not know the LOCATION. Even if you don't agree that LOCATION stands for CIPHER, they are still talking about something they don't know. This does not make sense in the context of K2.
Unregistered user # Monday, August 9, 2010 8:33:47 PM