CONTENT RATING: PG-13 (themes: torture, death)

See the Self-care page if you need support.

Enigma was one of the most advanced mechanical ciphers of its time. In this first episode, we look back at the history of cryptology to see the ashes from which this cryptographic titan rose. Below you will find data, audio credits, further reading, and a transcript of the podcast.

Did you like this podcast miniseries? Would you have changed anything? Want something different? More of the same? This Festival of Social Science event was supported by the ESRC and they’d love to know your views on it, so if you have a moment, please fill in this quick survey to let them know your thoughts.

Audio credits

Scott Holmes – Business Gateway
Kai Engel – Floret [Mild]
Kai Engel – Interception [Satin]
Kai Engel – Snowfall (Intro) [Cold]
Kai Engel – River [Mild]
Kai Engel – Snowmen [Cold]
Lee Rosevere – Healing

Credits, sources, and more

Bletchley Park (2019). Podcast. Available at <https://www.bletchleypark.org.uk>

Corera, G. (2014). “Poland’s overlooked Engima codebreakers”, BBC News (Warsaw), 5^th July 2014. Last accessed 10^th July 2019. Available at: < https://www.bbc.co.uk/news/magazine-28167071>

Dziewanowski, M.K. (1994). “Polish Intelligence During World War II: The Case of Barbarossa”, East European Quarterly, 28(3), 381-391.

Helm, S. (2015). If This Is A Woman. Inside Ravensbrück: Hitler’s Concentration Camp for Women. London: Little, Brown Book Group.

Kozaczuk, W. (1984). ENIGMA: How the German Machine Cipher Was Broken, and How It Was Read by the Allies in World War Two. London: Arms and Armour Press.

Kuhl, A. (2007). “Rejewski’s Catalog”, Cryptologia, 31(4), 326-331.

Lelwic, J. (2005). “Marian Rejewski – the man from Bydgoszcz who helped the allies win the war”. In: J.S. Ciechanowski, J. Garbowski, E. Maresch, H. Piechocka-Lipka, H. Sowińska, and J. Sylwestrzak (Eds.), Rejewski: Living with the Enigma Secret (pp. 45-66). Bydgoszcz: Bydgoszcz City Council.

Milner-Barry, S. (1993). “Hut 6: Early days”. In: F. H. Hinsley and A. Stripp (Eds.), Codebreakers: The Inside Story of Bletchley Park (pp. 89-99). Oxford: Oxford University Press.

Mollin, R.A. (2005). Codes: The Guide to Secrecy from Ancient to Modern Times. Boca Raton: Taylor & Francis Group.

Oleksiak, W. (2014). “The Hacker Who Saved Thirty Million Lives”, Culture.pl. Last accessed 10^th July 2019. Available at: <https://culture.pl/en/article/the-hacker-who-saved-thirty-million-lives>

Rejewski, M.  (1981). “How Polish Mathematicians Deciphered the Enigma”, Annals of the History of Computing, 3(3), 213-234.

Schmidt, U. (2005). “‘The Scars of Ravensbrück’: Medical Experiments and British War Crimes Policy, 1945-150”, German History, 23(1), 20-49).

Sebag-Montefiore, H. (2000). Enigma: The Battle for the Code.

Singh, S. (1999) The Code Book. New York: Doubleday.

Sowińska, H. (2005). “Life’s Enigma”. In: J.S. Ciechanowski, J. Garbowski, E. Maresch, H. Piechocka-Lipka, H. Sowińska, and J. Sylwestrzak (Eds.), Rejewski: Living with the Enigma Secret (pp. 21-44). Bydgoszcz: Bydgoszcz City Council.

Stripp, A. (1993). “The Engima Machine: Its mechanism and use”. In: F. H. Hinsley and A. Stripp (Eds.), Codebreakers: The Inside Story of Bletchley Park (pp. 83-88). Oxford: Oxford University Press. 

Turing, D. (2015). Prof: Alan Turing Decoded. Gloucestershire: The History Press.

Turing, D. (2018a). “Finally remembered: the Polish codebreaker key to Bletchley Park cracking Enigma”, iNews: The Essential Daily Briefing. Last accessed 11^th July 2019. Available at: <https://inews.co.uk/news/uk/bletchley-park-alan-turing-dermot-henryk-zygalski-chichester-polish-codebreaker/>

Turing, D. (2018b). X, Y & Z: The Real Story of How Enigma Was Broken. Gloucestershire: The History Press.

Winston, G. (2016). “Polish codebreakers cracked Enigma in 1932, before Alan Turing”, War History Online, 30^th May 2016. Last accessed 11^th July 2019. Available at: <https://www.warhistoryonline.com/guest-bloggers/wheatcroft-collection-m4-105mm-tank-engine.html>

Woytak, R.A. (1982). “A conversation with Marian Rejewski”, Cryptologia, 6(1), 50-60.

Transcript

These Enigma episodes have been supported by the Economic and Social Research Council, or ESRC, as part of their annual Festival of Social Science. This Festival celebrates the amazing research and advancements of our best and brightest scientists, and this year, almost five hundred events are happening all over the country from Saturday the 02^nd to Saturday the 09^th of November, 2019. You can check out the official hashtag #ESRCFestival on Twitter, and you might even find that some of the events are in the news.

Case S01E13 – Enigma, part 1 of 3.

This miniseries is special. The story of Enigma, at least as it’s told in this country – the UK – is often incomplete. On the one hand, that’s not such a surprise. Enigma involved numerous decades, dozens of countries, and thousands – hundreds of thousands – of people. But on the other hand, the story as it has been told here has still suffered from some skewing. In the UK, the cracking of the Enigma code is a legend, a source of ineffable national pride, and when told in movies and history books, one character always comes to the fore.

Alan Mathison Turing.

This is the British genius whose incandescent brilliance did not merely crack the Enigma code in piecemeal or intermittent bursts, but comprehensively crushed it on an industrial scale. Such is his stature that his face and name are on British banknotes, buildings, institutes, and monuments. Turing has become a byword for extraordinary intellectual ability. Even the devastating miscarriage of justice that repaid him so poorly for his incredible loyalty is well documented.

So what is missing in all these stories? Well, most of them largely start not at the beginning, often not even in the middle, but instead, almost at the very end, and plenty of them never look further afield than the leafy, quiet grounds of Bletchley Park. To look only at the story of Turing is to look only at the final mile of a marathon that started all those decades earlier across thousands of tempestuous miles of war-torn Europe.

Enigma is the story of a fast-changing secretive encryption system – arguably the most advanced mechanical cipher ever invented by that point. It is the story of bitter jealousy leading to the selling of secrets. It is the story of impoverished scientists striving to achieve the impossible, all while the dragon of war breathed over the borders, threatening to consume everything in its path. Enigma is stolen manuals, extraordinary breakthroughs, building replicas on miniscule budgets, smuggling, spying, and the stark, breathtaking nerve of a whole range of people, many of whom died or were executed as war consumed entire swathes of Europe. And without all of this coming first, Turing’s task would almost certainly have been simply impossible. Or, at best, it would have been so near impossible that he would have had no chance of completing it before the war came to an end in some other way instead.

In reality, far from doing it all himself, Turing, and his team were handed a gift-basket of breakthroughs and advances and insights, and yet, it is those first gruelling miles of the marathon, and the people who effectively ran them through grave danger, across the gulfs of seas and lands, that are so often forgotten.

In this miniseries then, we go back to the beginning of Enigma – even before Enigma – so that we can see the ashes from which this cryptographic titan rose. Then, in the second episode, as war foments, we criss-cross Europe throughout the 1920s all the way to 1937, following a handful of extraordinary people as they converge and part ways, each trying to navigate and survive the dark turmoil of war. And in the final episode, we pick back up in 1938. Only then do we finally turn to Bletchley Park and Alan Turing, and then we cast a last eye over the fate of those other, less well-known figures who also played such pivotal roles in cracking Enigma.

Welcome

Welcome to en clair, an archive of forensic linguistics, literary detection, and language mysteries. You can find case notes about this episode, including credits, acknowledgements, and links to further reading at the blog. The web address is given at the end of this episode. And, if you get a moment, leave us a rating wherever you get your podcasts from.

Secret writing

Our story begins deep in the shadows of pre-history, as civilisations were just starting, chaotically, to form and, more importantly, to clash. As hierarchies emerged, they brought with them concepts like royalty, kingdoms, trade, and war. But power, commerce, and military superiority all rely for their success on keeping secrets. After all, sending a regiment of soldiers to loot a neighbouring kingdom’s wealthy village requires covert reconnaissance, careful planning, and disciplined execution. Similarly, striking a trade deal with that same neighbouring kingdom is more difficult if they know in detail everything about your kingdom’s assets and liabilities. And keeping power consolidated in one figure – one monarch, one emperor, one tyrant – requires having access to the best and latest intelligence. It is vital to know who is loyal to whom, what their weaknesses are, whether they are amassing secret forces and plotting to seize power, but one does not then want to let that self-same precious intelligence leak to others. The trouble is, no matter how trustworthy a messenger or courier or soldier, everyone is susceptible to tiredness, errors of judgement, alcohol, bribery, torture. However, the person who does not know the secret, cannot reveal it. Thus, all those thousands of years ago, as the need to communicate precious intelligence in safety grew, this secrecy became the mother of codes, and ever since, leaders and royals have been disguising their communications as a way to protect, defend, and govern their empires, both large and small (Singh, 1999:ix). And from there, the use of codes has bled into every part of our lives – encoded letters, secret classifieds ads, messages hidden in music, art, and literature, plot devices in fiction and film, even just the way that parents talk to each other in front of children when they would rather not be understood.

But no sooner had someone invented codes, than someone else who couldn’t stand being in the dark invented codebreaking, and almost immediately this turned into a pitched battle. The earliest attempts at codes, mere letter shuffling or simple substitutions, were quickly cracked by these primitive cryptanalysts. The codemakers responded with more complex codes, and the codebreakers started to develop decryption principles, attack patterns, and methodological rubrics. Back and forth the battle raged, one side endeavouring to bamboozle with ever-more complex, opaque-seeming nonsense, and the other side striving not merely to solve the mystery, but to do so as efficiently as possible.

In the words of Simon Singh, author of The Code Book:

Codebreakers are linguistic alchemists, a mystical tribe attempting to conjure sensible words out of meaningless symbols. The history of codes and ciphers is the story of the centuries-old battle between codemakers and codebreakers, an intellectual arms race that has had a dramatic impact on the course of history…the development of codes can be viewed as an evolutionary struggle. A code is constantly under attack from codebreakers. When the codebreakers have developed a new weapon that reveals a code’s weakness, then the code is no longer useful. In turn, this new code thrives only until the codebreakers identify its weakness, and so on. (Singh, 1999: ix)

Code words

So far I’ve used a lot of terminology and actually I’ve been playing a little loose with it, but if you’re going to enjoy this sort of case properly, it’s useful to have a handle on the major terms and their meanings. Let’s start with the bigger concepts first.

The study of this field in general is called cryptology – crypt– meaning hidden, and –ology meaning the study or science of something. But you can study and specialise in cryptography – that is, the methods and means of encrypting and hiding. Thus, you would be a codemaker. Or you can study cryptanalysis – the methods and means of decrypting and revealing. As such, you would be a codebreaker. If it helps to remember them, cryptography has a G in it, for generating codes and ciphers. Cryptanalysis has an A in it, for attacking codes and ciphers.

Creating Enigma will have taken plenty of cryptographers – that is codemakers, and cracking it took stupendous amounts of cryptanalysts, or codebreakers. Some even argue that cryptanalysis – codebreaking – is harder than cryptography – or codemaking, and there may be some truth to this, but creating a completely unassailable code in the first place is also not easy, as this story will show.

Another pair of terms that often get used interchangeably, but which have specific meanings, are code and cipher. A code is actually something that you replace a full word or phrase with, so we have codenames – that is, cryptonyms – for secret agents, like 007 in the James Bond franchise. We have codenames for projects, like Operation Treadstone in the Jason Bourne series of novels and movies. We have codenames for military installations and operations and assets and projects and manoeuvres, and of course, we have code-words or code-phrases as a means of establishing authenticity or passing on information.

Imagine you and I were spies who had been instructed to rendezvous and we were establishing each other’s identity. I might be instructed to arrive carrying a copy of a particular magazine, let’s say, Rolling Stones, rolled up under my arm, and I might expect you to be wearing a New York cap and be talking on a phone. Upon seeing such a person who fits these visual requirements, I might attempt to light a cigarette three times and then give up in apparent disgust when my lighter doesn’t work. You might approach to offer your own lighter, and I might then casually say to you, The Darkness was rock music’s greatest tragedy. In turn, I will wait for you to respond with, Hawkins should have stuck to pedicures.

Or it might be something more dynamic. I might be instructed to say five natural-sounding words, and for each word I say, you might have to produce words starting with the same letters of the alphabet. If I say What a day for this, you might reply We are definitely forgetting Tuesday. Of course, that’s far more cognitively taxing, it’s less likely to look, or even sound natural, and 99% of espionage is about being as boring and unremarkable as humanly possible. But out in the field, when your life could be at risk if you put your trust in the wrong people, there is a crucial cost/benefit analysis to be made. Espionage by its very nature involves layer upon layer of deception. Everyone is pretending to be someone or something they are not. Spies are trying to look like ordinary people. Imposter spies who are trying to infiltrate your organisations and networks are also attempting to look like friendly spies… who are trying to look like ordinary people.

One wants to ensure true positives and negatives – that is, the right people can validate themselves because the system is replicable, and the wrong ones cannot validate themselves because the system is rigorous. You don’t want false positives – imposters getting in because the system is too replicable, nor false negatives – friendly people being shut out because it’s too rigorous.

Ordinarily, higher levels of security tend to bring with them higher levels of cognitive effort or, worse, greater degrees of obviousness. And, more to the point, any codephrase, or the key to producing an acceptable codephrase can be intercepted, or tortured out of a captured agent, or even inferred if the agency’s protocols are not what they ought to be. This makes codephrase validation alone, by modern standards, a pretty dicey means of fully and confidently determining the veracity of someone’s identity. Just as we cannot get all the way into GCHQ or the Pentagon via a single door and a single swipe card, establishing who someone is, especially on critical missions, generally tends to take far more steps than just producing a single short sentence.

Modern espionage and cognate occupations rely much more heavily on technological means. If you visually check out with your New York cap, and then behaviourally check out by lending me your cigarette lighter, and even linguistically check out by producing the correct passphrase, I might then go onto another stage of validation that involves me handing you an innocent looking device – something that appears to be an e-reader or phone or pen or folder or lighter – that swiftly takes biometric readings by, for instance, scanning your fingertips or eyeballs or the patterns of veins in your hand or whatever. As this relays that information back to my home agency for cross-checking, I might also have a remote handler communicating with me via an earpiece, who is using audio and visual streams, perhaps from a tiny camera I have on me, perhaps from CCTV, perhaps from a third party taking the footage, to additionally verify your identity. Or there might be tiny iris implants or invisible skin tattoos or microchips under the skin or one of a myriad of ways in which the uniqueness and validity of someone’s identity can be established. However good the tech, though, it can and does fail, the internet goes down or isn’t available, and we can end up losing all our devices for endless different reasons.

In such a case, if I suddenly find myself in a dicey situation, perhaps amongst hostiles and far from home, having the absolute last resort of a call-response validation protocol to fall back on could be the difference between suddenly turning enemies into friends, and ending up just another anonymous star on the wall back home at headquarters.

Quite however you end up in a scenario of providing such a codephrase to someone else, if they successfully produce the second part of this linguistic handshake, you have both covertly established your identities, and if they look at you like you’ve lost your mind or they produce the wrong phrase, you can make good your escape, or pretend that what they said was fine whilst now secretly plotting to assassinate them. Or whatever it is spies actually do in these circumstances. Maybe they text home or something. I don’t know. I imagine they’re more socially literate than I am so if you listen to en clair, you work in espionage, and you feel like sharing the proper protocol for a failed walk-up validation, drop me an email.

Moving on, another similar use of codephrase is in conveying critical, pre-arranged information. As we so often see in films, an agent might send a communication to their handler saying, The blackfish has flown north, and this might actually mean that an enemy country has started testing weapons, or, as we’ll see later, if the message comes from an embassy and arrives with a clandestine services operative, it might mean that someone has some valuable information to pass on.

These examples sound dramatic, but actually we used codes all the time. For instance, I might say to my partner, there’s some post-prandial cocoa-based sustenance, and by phrasing it in a way the children won’t understand, let him know there’s chocolate cake in the fridge for after dinner. Or, in broken French, I sometimes say, maman voudrais un moment de jour – at which the children get distracted whilst I go have a mindfulness moment of necking Dime bars in the blissfully dark and peaceful pantry. Or if we are disagreeing about how to respond to something one of them has done, but want to maintain the illusion of a well-oiled parental management unit, we use the code phrase, Jim’s Dad, which means “we need to pause, calm down, and discuss this but out of the way of little ears”.

Essentially, all languages are actually codes. If you speak the language, then it’s transparent to you – you mentally possess the codebook necessary to understand it, but if not, then the code is opaque and you need someone to translate it. In the case of codes created for cryptological reasons, there will often be something like a codebook, whether physical or memorised, and you can even combine both artificial codes where you switch certain words or phrases for other words or phrases, and also, as an added layer, produce the whole thing in an unusual language. This happened in the case of the World War II code talkers, but we’ll save them for another episode.

Compared with codes, ciphers work at a finer level of granularity. Instead of replacing whole words or phrases, the individual letters or sounds within the word are substituted for something else. In very simpler ciphers, you might just move every letter on one place, so cat would become dbu. Or you might switch every letter of the plain alphabet for a symbol. A could be the laughing emoji, B could be the number five, C could be the question mark, D could be something you’ve made up, and so on, all the way through until you’ve created a whole cipher alphabet. Then you use your cipher alphabet to turn your plaintext – that is, your currently unencoded and easily read message – into a ciphertext. In doing this, you’ve encrypted or enciphered your message. To decrypt or decipher the ciphertext, the recipient needs the key, or in other words, they need to know what each of the symbols means so that they can turns the ciphertext back into a plaintext.

So how do you go about cracking codes and ciphers? Well, spies and other operatives might concentrate on trying to intercept or steal the key (if it’s a cipher), or the codebook if it’s a code. By contrast, codebreakers try to retrospectively reconstruct the key or codebook based on clues in and around the ciphertext or code. And just for those of you who haven’t listened to the Welcome Waffle episode, this podcast’s very name also uses some cryptological nomenclature. En clair is French for in clear. Colloquially it’s used to mean something like, in simple terms or speaking plainly or to put it clearly, so it often comes at the start of brief explanations. In cryptographic parlance, however, it actually means to send a message plaintext – that is, to send it unencoded, or unenciphered. By contrast, if you were to send the message encoded or enciphered, you would say that you were sending it en chiffre. That is, you would say you were sending it in numbers, or ciphers.

Anyway, that’s the terminology down, but don’t worry too much about trying to remember it all now. I’ll keep regularly glossing it as we proceed until it hopefully becomes part of your growing espionage lexicon.

Power, prestige, privacy

What the ordinary person like you or I knows of applied cryptography and cryptanalysis – that is, codes that have been created, used in the field, and then cracked by adversaries – is largely historical. Outdated even, versus modern methods and standards. And there is much even from history that is still kept under a very dark cloak of secrecy. That’s unsurprising on reflection. Extremely strong ciphers and codes are invaluable for conveying ultra-sensitive information and intelligence. At the same time, successful methods of cryptanalysis are invaluable for intercepting and decoding such messages from others. The diplomatic and military value of both is incalculable, and the work of a gifted cryptanalyst or cryptologist is beyond price. Such people and their achievements and breakthroughs are therefore generally kept well out of the public eye. In some cases their work or some of the information around it may eventually be partially or fully declassified, but this will often be long after their death, when the revelation of such sensitive information can no longer be leveraged against the country or individuals in question, and then, at long last, that codemaker or breaker  may finally receive some recognition for all they have done. But even then their work may be so advanced, or sensitive, or even controversial, that the records are still kept firmly under wraps. In other words, if you are thinking of doing cryptology for the glory, you couldn’t pick much worse, since there will be the exact opposite of fame in your lifetime. And for the amateur cryptologist who merely enjoys the subject and enjoys reading about it, pretty much everything the ordinary person can reasonably learn about this field is only a partial picture anyway, and it’s already substantially out of date.

Steganography

Let’s look in depth at some real life examples of secret communications, and all the many ways we’ve found of hiding messages for others to find.

So far I’ve said a little bit one way of sending secret messages – making it inaccessible, even if it is intercepted, and this can be done through codes, where we substitute whole phrases for a single word, like broken arrow to mean “oh god someone has stolen one of our nuclear submarines”, or through ciphers, where we substitute letters or sounds in the words for other symbols or letters. Or, of course, for extra security, you could combine the two by enciphering the words broken arrow using some sort of cipher alphabet. There’s more, but I’m going to come back to this in a bit because it will lead us, finally to Enigma. Before we get that far, however, there is another, easier way to hide communications.

You literally hide them. This strategy is known as steganography. This comes from the Greek steganos, meaning covered, and gráphein, meaning to write, or in other words, it’s writing that you’ve somehow hidden (Singh, 1999: 5). Rather wonderfully, the term seems to date back to 1499. Johannes Trithemius was German Benedictine Abbot and polymath. During the German Renaissance he was a lexicographer, cryptographer, and occultist – essentially exactly the sort of person you want to invite to a dinner party if you possibly can. But maybe one at someone else’s house. Anyway, Trithemius created a polyalphabetic cipher which he published in his book, Polygraphia. We come back to polyalphabetic ciphers later. Polygraphia is is sometimes cited as the first published work on cryptography, but more to the point for steganography, he also published a three-volume book called, somewhat unsurprisingly, Steganographia. Of course you have to remember that he was essentially coining the word, or at least, this new meaning for it, so the title alone wasn’t a giveaway. This Steganographia looked like a three-part monograph on magic – that is, the use of spirits to communicative to others far away. However, in 1606, a decryption key was printed for the first two volumes, revealing their real contents. For centuries afterwards, the third volume yielded no secrets and finally people came to believe that it was purely about the occult, with no deeper, hidden messages. Then, in 1996, Dr Thomas Ernst, a Professor of German at La Roche College in Pittsburgh managed to crack the ciphers in the third volume, but though he wrote up a 200-page paper explaining the insights, perhaps because it was written in German and published in a Dutch journal, Daphnis, it sank into obscurity, leaving others to believe that it remained undeciphered. The third volume was then cracked a second time in 2010 by mathematician Dr Jim Reeds of AT&T Labs, and it was only when he went to publish the paper that he discovered that Ernst had beaten him to it fourteen years earlier. Anyway, the point is, Steganographia was both an example of, and a treatise on cryptography and steganography.

Stego that funky signal

There are as many ways to employ steganography as there are colours in the rainbow, or as you have the imagination to dream up.

As long ago as 440BCE, in his book, The Histories, Herodotus described how Demaratos warned the Lacedemonians of an impending attack by writing his message on a wooden tablet, and then covering the writing with wax. Upon receiving the tablet, the Lacedemonians scraped off the wax, revealing the message.

In another incident described in the same book, Herodotus explains how Histiaios shaved the head of his most trusted slave and tattooed a message on his scalp, and then, after his hair had grown back, sent him to Miletos, with the aim of presenting himself before Aristagoras. The message was an encouragement to Aristagoras to revolt against the Persians. The benefits of this are several. The messenger doesn’t need to know the message. Indeed, depending on the circumstances and the use of drugs and so forth, they may not even know they have a tattoo on them. And, if they’re captured and searched, there is nothing to find, unless the captor randomly decides to shave their head. But the drawbacks are considerable. The message will be weeks, if not months old by the time it arrives, substantially reducing its value. If the enemy learns that this tactic is in use, it’s easy to check for, and difficult to hide. The message could also be in code, or enciphered, or both, but by then you’ve caught the messenger, and they’re at your mercy. Messengers also can go astray for other reasons – sickness, fear, a moral epiphany (wait, are we the bad guys?!) etc., and sending a flurry of messages in a fast moving situation is going to start to get logistically awkward. Overall, then, sending secret messages across the office tattooed onto the heads of those under our command is generally a non-starter. What else can you do then?

Other methods include skip-codes, where the hidden message is retrieved by reading the first letter of each word, or every fifth word, or every other sentence, and so on. This is also sometimes called a null-cipher, because the plaintext message is mixed in with a whole heap of non-cipher, or null text, designed to throw the cryptanalyst off. The downside is that it can be tricky to create natural sounding text to go round the actual intended message, and it can take a lot of null-cipher to suitably drown out the actual cipher. The bonus, however, is that a null-cipher is especially effective. If you’d like to try your hand at one, go to the transcript of episode five, there are no less than three Easter eggs hidden away in there. One is a steganographic message in the text. One is a steganographic message hidden at two different levels in some audio. And the third is for you to discover… 😉

Another example is the method used by the Doll Woman. Real name, Velvalee Dickinson, the Doll Woman was a New York City dealer of rare, collectible, and antique dolls. And she was also, apparently, a Japanese spy. During the Second World War, she would write up orders for, and letters about her dolls. These described quantities, designs, types, and scenarios, but they were in reality descriptions of warships, shipyards, and coastal defences, which she was sending on to Buenos Ares in Argentina. When the recipient of the letters moved, five of her letters were returned and intercepted by the FBI, who began to piece together the messages hidden in the coded language.

But one needn’t invent random letters to send messages. Historically, book printers used to run out of fonts for publishing books, so it wasn’t unusual to find a mixture of font faces and styles in a page. One could therefore write the ciphertext in italics, or Times New Roman letters, and use Arial for all the null-cipher. On modern webpages it’s even easier to execute, invisible to the naked eye, and faster to detect with a computer. One can add in transparent content, or underline certain letters and set the underline colour to match the page, or use letters that look exactly the same as the ordinary alphabet but have different ASCII codes, or implement a fractional font size or kerning or other change, or embed something within the HTML of a webpage, and so on and so forth through thousands of different possibilities.

But writing is not the only medium for hidden messages. Another method involves music. In classical music, people have hidden names, initials, and short messages through the sequences of musical notes. Other methods in more modern music have involved recording messages backwards, known as backmasking. And other music has Morse code hidden in it, but for more on all of these things, you should listen to episode five. One that I don’t mention in that episode is hiding images in digital music files. When he audio is rendered as a spectrogram, this graphical output reveals the hidden picture. Go read about the image in the Nine Inch Nails song, My Violent Heart.

But what if you don’t fancy tattooing, retailing, or composing? Well, how do you feel about knitting and similar handicrafts? For the crafty and creative, there is a world of possibilities. For instance, you might tie smaller and larger knots representing Morse code into a thread, and then sew the thread into clothing. Or you might use the colours and patterns and stitch counts on a scarf or sweater or any item at all to encode information.

Feeling like these are all rather hands-on and that you could maybe do better with something a bit less Jane Austen, and a bit more Ada Lovelace? Modern steganography, unsurprisingly, involves computers, and the possibilities here are about as limitless as the internet and computing in general. An extremely simple version involves hiding a plaintext message as another file-type such as images. Then, when the file extension, such as .jpg is changed to .txt, the image of, say, a lolcat turns into a notepad file that opens as normal, revealing its internal message. Of course, this message too could be a code that is also enciphered and then further hidden via a skip-code method inside a null-cipher… Alternatively, instead of encoding a text file in a picture, one could hide one picture inside another – useful for conveying intelligence that is visual in nature. Or one could disguise audio as a PDF. Or a PDF as a spreadsheet.

And that’s just the start.

Steganography online can be as innovative as it is global. One form of blog steganography involves turning the message into a cipher, then breaking it up into fragments and posting those little bits as comments across thousands of abandoned blogs. Part of the key involves knowing which blogs to check and how to reconstruct the message, but the benefit is that one can access it from anywhere in the world with an open internet.

But let’s up the stakes considerably. What if you’re a captive, about to be broadcast on TV or in pictures, and at risk of execution if the people holding you realise you’re conveying a message of some sort? And what if this is your only opportunity to signal for help? All the methods I’ve just mentioned are useless here, and even words are risky. One option that you might have available to you is movement, or gesture.

One famous example of live, silent steganography was Jeremiah Denton. In 1965 during the Vietnam War, Denton’s plane was shot down and he was held as a prisoner of war in North Vietnam for eight years. In the first year of his captivity, he was forced to take part in a live interview in which he was asked to describe how well he and his fellow prisoners were being treated. As he spoke, praising the conditions and saying that he was being taken good care of, Denton seemed to be having trouble with the bright studio lights, but in fact he was blinking the word TORTURE in Morse code, thereby confirming for the first time that prisoners of war in North Vietnam were indeed being tortured.

The same war, only two years later, in 1968, the intelligence ship, USS Pueblo was captured by North Korea. In their communications with the US, the North Koreans claimed that the ship had defected to North Korea and staged photographs of the crew members in various poses, looking comfortable and as if they were there of their own volition. Except that in many of the pictures, the crew were subtly giving the finger, sometimes resting their hand against their face the way you did as a kid in the classroom (don’t pretend you never did this) and at other times sitting with their hands clasped in their laps, but with the middle finger extended on one hand.

Initially unfamiliar with the gesture, the North Koreans believed the crew when they claimed that it was a Hawaiian good luck sign. When they found out the truth, the crew members were subjected to even more extreme beatings and torture than they had been experiencing till that point. Eleven months later, when the US acknowledged that USS Pueblo was indeed a spy ship, apologised for the incident, and promised not to spy any further on North Korea, the crew was released. But the capture of USS Pueblo was a devastating blow for other reasons. The North Koreans were able to reverse engineer some of the cryptographic equipment on the ship and this gave them access to some US communications. But the Americans and their wars of the 1960s and 1970s did not have a monopoly on incredibly courageous captives risking their lives to hide messages in plain sight.

Another example, another war, another set of prisoners. But this time, it’s World War II, the prison is in Germany, and the prisoners are women.

Letters from Ravensbrück

Ravensbrück was the largest Nazi concentration camp for female prisoners during WWII (Helm, 2015). It opened in 1939, and from 1942 until it closed down in 1945, experiments were carried out on some of the prisoners. I won’t go into details because honestly, it is utterly harrowing. It’s enough to say that some died horrific deaths, others were executed after the experiments on them had run their course, and those who survived were permanently damaged and traumatised. These women came to be known by both prisoners and guards as kaninchen – “rabbits” (Helm, 2015: 246), and their plight very quickly garnered them a huge surge of sympathy and compassion from the other prisoners.

A group of four Polish “rabbits” – Wanda Wojtasik, Krysia Czyż, and sisters Janina and Krystyna Iwańska – decided that something needed to be done to reach the outside world. While they were aware that the existence of concentration camps was widely known, they felt sure that the sadistic experiments being conducted on women at Ravensbrück would not be common knowledge, as the SS seemed extraordinarily keen to keep the rabbits quiet (Helm, 2015: 272). Letters from prisoners to their families were heavily censored by the SS, so the women had to figure out a way to deliver the message to the outside world without being caught.

All four women had been scouts and knew that it was possible to hide messages using lemon juice, milk, or onion juice as invisible ink – but as food was strictly rationed and mostly consisted of bread or watery soup, they had access to none of these things (Helm, 2015: 271). Krystyna, suggested that they try using the one substance they all had access to on a daily basis: urine. They could dip a stick in urine and use it to write, and the urine would become invisible to SS censors as soon as the paper dried. The recipients then simply had to apply gentle heat the letter to view the secret message (such as by running a hot iron over the surface of the paper), as this would char any iron-rich substances and turn the ink brown (Singh, 1999:6). But how would the rabbits alert the recipients of the letters that they contained secret messages written in invisible ink?

As a child, Krysia loved to read adventure stories with her younger brother Wiesław, particularly those by the Polish writer Kornel Makuszyński. One book, entitled The Demon of the Seventh Form, told the story of a protagonist who sent secret messages hidden in texts. Krysia drew on this story to come up with a plan. First of all, she suggested that they should write their secret messages between the lines and in the margins of their official letters. Secondly, she suggested that in their initial attempt – which would be sent to Krysia’s brother and parents – the first letter of each line should spell out LIST MOCZEM in Polish. This translates to LETTER WRITTEN IN URINE. Given that Wiesław had also enjoyed Kornel Makuszyński’s stories, she hoped he would spot and recognise what she was trying to do. To help him along, she also made reference to a story by Makuszyński in the main body of her letter. The first secret message read, “We have decided to tell you the whole truth”, followed by a few short sentences about the medical experiments (Helm, 2015: 272).

Krysia’s previous letters had all been written in a very structured, formal style. When a new letter arrived to the Czyż residence in Lublin early in 1943, Wiesław immediately spotted the reference to the book and noted that it seemed strangely out of place (Helm, 2015: 273). Along with his parents, Tomasz and Maria Czyż, Wiesław managed to figure out that he needed to string together the first letter of every line. Unfortunately, part of the lower half of the letter was rubbed out, which meant that Krysia’s message seemed to read something like WET THE LETTER. They duly sprinkled some water over the paper, but this had the unfortunate effect of making the message only briefly visible before it quickly disappeared. It was enough, however, for them to realise what Krysia was trying to do. To prevent future messages being lost, they took the letter to a trusted friend who was a chemist, and he advised them to run a warm iron over future letters instead. Maria Czyż, Krysia’s mother, was a Major in the Polish Home Army, and she was able to pass the secret messages on to stations in Warsaw and Sweden (ibid.).

Meanwhile, the four women at the camp began to develop their technique further, sending parts of their secret messages to each of their four families, who would then meet and put the puzzle pieces together. The families were asked to acknowledge receipt of the secret message through various subtle means in their return letters, such as sending a coloured piece of thread or scratching the number of the letter onto a tin within a food parcel (Helm, 2015: 275). Over time, the little group of four began to expand, including Zofia Sokulska, a lawyer from Lublin, and Wojciecha Buraczyńska, a student from Warsaw.

The letters contained vital information about specific Nazi guards who conducted or oversaw the experiments, and listed prisoners’ names and numbers, the operations that they had undergone, and surgery dates. Women who had died as a result of being experimented upon, or through being executed or gassed, were listed with crosses next to their names, so that families could be informed of their deaths (Helm, 2015: 276-7). Much of this information was later broadcast on an underground radio station based in London called Dawn Radio (SWIT) (Helm, 2015: 426). On 3^rd May 1944, for example, SWIT broadcasted a news item about the rabbits entitled “Vivisection in Ravensbrück”.

In 2008, the author of the only chronicle about Ravensbrück, Sarah Helm, found that Krysia was still alive and living with her daughter Maria Wilgat in Lublin. Krysia was suffering from memory loss and was unable to communicate. While Krysia’s brother Wiesław believed that Maria likely had access to Krysia’s letters, Maria did not want to talk about them at the time. Instead, she showed Sarah Helm an essay which had been written by Krysia about the secret messages. Krysia had apparently refused to talk about her traumatic experiences at the camp, except in reference to the letters. She had written:

After we received sign from my family that the first secure letter had been deciphered this dangerous game absorbed us completely. We began to work on improving and expanding our correspondence. The first improvement we made was to stop writing between the lines. Instead we used the inside of the envelopes of the camp letters. This way we gained some extra space, because we could write more densely on clean paper. It was also safer. In the first period of our correspondence we put a successive number on each envelope so that our families in Poland could know if they were receiving all of the letters we had written. (Krysia Czyż, cf. Helm, 2015: 274)

In 2010, Maria Wilgat contacted Sarah Helm to notify her that Krysia was in critical condition. Krysia had in fact kept and carefully preserved all twenty-seven letters, and Maria presented them to Helm. Most of them confirmed what Krysia had written in her essay, but there were some letters Krysia had not spoken about. In one of these, Krysia wrote:

Mama dear, from yesterday I am depressed and I cannot stand it, so I have to write to you my thoughts and imagine we are close and that I can feel you near to me. I feel how nice it is and I start to cry. Sometimes it is so bad I have to talk to you in my head or write, or I have to start thinking about something else because otherwise I collapse. ( Krysia Czyż, cf. Helm, 2015: 277)

If there is any critical catalyst for codemaking, and codebreaking, it is war, because everyone from president to prisoner has everything to lose. The ingenuity, and courage, of captives in getting information out from the very heart of a brutal concentration camp is, in itself, extraordinary, and for those who can bear it, I recommend that you read the book on Ravensbrück by Sarah Helm, entitled If This is a Woman. I won’t pretend that it isn’t utterly heart-breaking reading, but if anyone ever had courage in the face of absolute horror, those women did.

What, then, of the appalling regime that put those women there? How did the agents and operatives and soldiers that were part of this organised monstrosity communicate their plans, successes, and losses, without themselves giving away vital intelligence to their enemies? To understand this, we need to look at the different types of cryptography, and then dissect Enigma, one of the most advanced mechanical ciphers ever invented

Cryptography

A quick recap. To help refresh your memory:  cryptology is the study of encoding and decoding, cryptographers make codes, and cryptanalysts break them. A plaintext written in a plain alphabet is what you put in, and a ciphertext written in a cipher alphabet is what you get out. Finally,  there are two fundamental way of secreting communications – steganography, where one hides the message itself in some way,, and cryptography, which involves hiding the meaning.

So let’s now look at that other form of sending secret communications – cryptography. And that’s where we’ll stay for the rest of this miniseries. As we’ve already seen, cryptography can be carried out through codes, where whole phrases are substituted for words or other phrases, or it can be done through ciphers at the level of the letter or the sound. A message is effectively obscured, or scrambled, or both, according to a particular system that is typically agreed upon between the sender of the message and its recipient(s). As the recipient knows the system or protocol, they can effectively reverse it to uncover the meaning. Without access to that original protocol, anyone who intercepts the message is, theoretically at least, unable to read it. So let’s look a bit more at the two cryptographic processes of transposition and substitution

Transposition

Transposition, as the name suggests, involves rearranging the letters in the original message. It’s basically a glorified, and probably unreadable, anagram. The shorter the message, the easier it is to decrypt. For instance, it’s certainly true that there is more than one way to scramble a cat, but aside from CAT itself, it turns out that the limit is actually only five: ACT, ATC, CTA, TAC, and TCA. Of course, this is a level of transposition that even a child could work out in a few short minutes, but as the number of letters increases, so too do the potential combinations. Dramatically.

On its surface, transposition – at least of a long enough message – might seem more secure than steganography. If someone intercepts the message they’re not guaranteed to decode it, after all. But it too has its drawbacks, and the classic problem that has plagued cryptographers since the dawn of the field is getting the key to the recipient securely. The key, as you may remember, is the means of deciphering the message, and in this case, it would consist of some explanation or indication of how the message has been scrambled, so that the recipient can reverse the process.

If ever there were something worth protecting on the one hand, and intercepting on the other, it would be the key to a good encryption method. For safety, of course, you’d encrypt it, but… maybe you can see where this is going… then they’d need the key for that encryption. In short, ironically, for the key to be useful to the intended recipient, they have to be able to read it. Which they can’t if you’ve encrypted it. Because they need… the key (Singh, 1999: 8). Catch-22.

Substitution

Anyway, onto obscuring, or in technical terms, substitution. I’ve already mentioned the really simple versions of this, where you might bump each letter of the plaintext one along in the alphabet, so that, for example, en clair becomes fo dmbjs. As Singh (1999:9) notes:

“In transposition each letter retains its identity but changes its position, whereas in substitution each letter changes its identity but retains its position.”

I should add, there’s no reason why you can’t go a little Carly Rae Jepsen on your message and do it all – substitute all the letters, then transpose them all, then use steganography to hide the whole ciphertext in, I don’t know, a picture of Elvis. But for now we’ll stick with one layer of encryption at a time.

There are countless historical examples of substitution ciphers being used for military purposes. Julius Caesar, for instance, used substitution ciphers very frequently. In particular, he was fond of using what is now known as a Caesar shift cipher, in which he replaced every letter in the original message with the letter three places down in the plain alphabet (Singh, 1999: 10). Shift ciphers can involve shifting as far as one likes up or down the ordinary alphabet, or indeed, mapping the plain alphabet against a scrambled one. Whatever the case, one develops a system. For instance, we might create a cipher alphabet by turning the ordinary alphabet backwards and then starting it at S. When it gets to Z, we carry on through A until we finish our twenty-six letters at T. You can see this in the blog post:

abcdefghijklmnopqrstuvwxyz
srqponmlkjihgfedcbazyxwvut

For the next stage in our algorithm, we might say that whatever the resulting ciphertext, all the odd letters must be written first, followed by all the even ones. Essentially we are going to use a Caesar shift substitution first, followed by transposition. That whole process is known as the algorithm, and in our example, the plaintext CATS would first be enciphered as qsza, and then this would be transposed as qzsa, with the first and third letters written out first, and then the second and fourth written out afterwards. In a short message like that, transposition doesn’t have much of an effect, but in a longer one the difference would be more profound.

If one has both the algorithm and cipher alphabet, it is possible to firstly reverse the transposition process and put the letters back into their correct places, and then decipher the message to reveal its contents. However, if one only knows the algorithm – that the letters have been substituted and then transposed, even if they know that I’ve used a Caesar shift of seven, unless they have access to the cipher alphabet itself, they will find it extremely difficult, if not impossible, to crack the code. This is because the cipher alphabet could have been scrambled in an almost infinite number of ways. For simplicity, I picked reversing it, and it would be an obvious thing to try if you were attempting to crack the code. Similarly it would be sensible to try the alphabet straightforward, and see what happens if you use it with a shift of one, and then two, and then three, and so on, till you’d tried all twenty-five possibilities. But imagine I’d taken a minute more and scrambled the cipher alphabet instead. One method of reproducibly, weakly scrambling is to start the cipher alphabet with a pre-agreed keyword or keyphrase. We could use en clair, for example, and then continue the rest of the alphabet in order, making sure no letters were repeated, so this cipher alphabet might be ENCLAIRSTUVWXYZBDFGHJKMOPQ. Of course, instead of the rest running in order, that too could be scrambled but then I would have to pass on the method so that my recipient could also make an identical cipher alphabet. For the purposes of this example, let’s imagine I don’t bother scrambling the rest though. The recipient could then reconstruct the cipher alphabet I’ve used now just by knowing the keyphrase, en clair. But how do you get a keyword or keyphrase to the recipient without just telling them it, and possibly having that information intercepted? Well, instead of the keyword or keyphrase, you might give them three digits – the page, sentence, and word number of some pre-agreed book. Perhaps that book might be whatever is number seven on that day’s bestseller’s list. Or you might send them a timestamp which relates to a word in a pre-agreed song, or a longitude to cross reference with a pre-agreed latitude which then identifies a specific street name. There are any number of ingenious ways of conveying to the recipient the intended key without simply telling them, though again, in all cases, you must communicate to them the system, if not the key itself. And whatever the chosen sentence, phrase, or word, this then becomes the key at the start of the cipher alphabet. Or at the end. Or ten letters in. Whatever the system that you’ve both agreed on.

This all tends in one direction – randomising the cipher alphabet in some way that can be flawlessly reproduced by the recipient. And just how many ways are there to randomise a mere twenty-six letters? Brace yourself. You can create four hundred septillion randomised cipher alphabets. Yes, not four hundred billion, or trillion, or quadrillion, or quintillion, or sextillion. Four hundred septillion. That a four with twenty-six zeros after it: 400,000,000,000,000,000,000,000,000, or in common parlance, it’s a lot. Imagine you decide to add the numbers zero to nine to your cipher alphabet, and maybe you want both upper case and lower case letters, and perhaps you even want some punctuation too. Then you would be randomising more than sixty characters, with exponentially many more possibilities of randomisation. So, as I said, let’s say you intercept my enciphered message. And perhaps you work out my algorithm too. You discover that I’m using a Caesar shift, and you even know that I’m just employing a basic twenty-six letter cipher alphabet that’s been scrambled in some way. Despite having both enciphered message and algorithm, you would still not be able to decrypt my code. To do that, you would need the key – that is, the cipher alphabet that I’d used. Without it, it wouldn’t even help you to know that my code is shifting seven letters along, because if the cipher alphabet is scrambled, what is it even shifting to?

In different words, Caesar shifts with a randomised cipher alphabet are pretty easy to create and extraordinarily difficult for the average person to crack, and for centuries, we thought that this type of substitution cipher – the monoalphabetic substitution cipher (Singh, 1999: 13) – was essentially unbreakable… until we abruptly discovered that it wasn’t.

Cryptanalysis

If creating a good cipher is tricky, then cracking one is somewhere between the agony of genius and the tranquility of madness. One must be part linguist, part mathematician, part psychologist, and these days, it really doesn’t hurt to be part software coder too.

Fortunately for cryptanalysts – that is, the codebreakers, their codemaking nemeses, the cryptographers are bound by some basic principles, and these include consistency and replicability. They must apply their own algorithm the way that they’ve said they will, and generally speaking, the recipient must be able to replicate that process in reverse to extract the hidden plaintext. But this very principle of having an encryption algorithm is a double-edged sword. Properly applied algorithms produce consistent, repeatable patterns, and patterns are exactly the weak points that cryptanalysts look for when trying to crack codes.

Psychologically, for instance, we might look for patterns in behaviour – when trying to create a random-looking cipher alphabet for the first time, plenty of people literally type out the QWERTY keyboard layout because, to them, this seems random. Or we might note that the intercepted messages always start with Hi, and end with Love you, X. This immediately gives us nine letters of their cipher alphabet. But this would require case-by-case knowledge and there are bigger patterns that we can tap into.

Frequency analysis

Cryptanalysis really only began once scholarship in several key disciplines was sufficiently sophisticated – namely, mathematics, statistics, linguistics, and, perhaps somewhat surprisingly, theology. Arab theologians who were interested in establishing which statements in the Koran were attributable to the Prophet Muhammad carefully studied etymology, sentence structure, and letter frequency. In essence, they were attempting an early form of authorship attribution by trying to determine whether certain texts were “consistent with the linguistic patterns of the Prophet” (Singh, 1999:16).

It’s not clear precisely who first realised that letter frequency could be exploited to break ciphers. The earliest written description of this form of cryptanalysis is by a scientist in the 9^th century, al-Kindī. Al-Kindī appears to have published some 290 books on topics ranging from medicine, astronomy, and linguistics, to mathematics and even music. One book entitled A Manuscript on Deciphering Cryptographic Messages was only discovered in 1987. In it, al-Kindī carefully described his method, as follows:

1. Determine the relative frequency of each letter of the alphabet through examining a large sample of texts in the relevant language.

2. Examine your ciphertext and calculate the frequency of each letter within the ciphertext.

3. If you assume – based on your analysis of a large sample of texts – that the most common letter in English is E, and you see that the most common letter in the ciphertext is X, you can assume that X is likely to be substitute for E.

This breakthrough was huge. Now, instead of having to manually try eleventy zillion possible cipher alphabets one by one, cryptanalysts could essentially jump to frequent analysis to at least get them started. Even if some letters mapped across incorrectly, enough might still be correct to identify some partially deciphered words, and as the errors in these words were corrected, the rest of the cipher alphabet could gradually be filled in. In other words, patterns in the orthography of language itself allow the cryptanalyst to induce parts, or possibly all of the cipher alphabet, and crack the code.

Of course, this method isn’t perfect. Imagine that plaintext was in English but included references to numerous Polish surnames, or even quotes from some other language. This could throw the frequencies off radically. Or if the message is very short – just a few words long, there simply won’t be enough data to generate a meaningful frequency list. And if it’s both short and contains non-standard words for that language, then you can forget it. Frequency analysis will be close to useless.

This said, there are still possible tiny cracks in the cipher into which one might crowbar one’s intellect and attempt to lever a gap big enough to get in. For instance, one might focus on just three of the most frequent letters, and then look at the letter ordering. Most English words contain vowels, for instance. Some consonant clusters simply never occur. Some consonant clusters that do occur, like ng, will almost always prefer the end of a word, and will never appear at the start. Through a long set of rules and an exhaustive process of elimination, one can strike in certain possibilities, exclude others, and formulate possible and probable solutions.

As the monoalphabetic substitution cipher came under attack from frequency analysis, the cryptographers fought back. Oh, they said. So you’re looking at the words and guessing where things go? Well what if you can’t tell where one word ends and another starts? Removing spaces and replacing them with other characters is no different than replacing the letter A with the letter B, after all. Indeed, the space can even be one of the substitutions. One might always write T as a space, and always write a space as an E, and in so doing, at a glance, the ciphertext would appear to be full of individual words, but we would be inadvertently seeing them starting and ending in all the wrong places. But you can also analyse the frequency of spaces no differently than you can analyse the frequency of letters, so this is no less secure if the cryptanalyst knows to check for it.

Again, the cryptographers fought back. Well, they reasoned, if you are using counting as your way in, then let’s screw up your figures. And they began to introduce nulls – symbols or characters that represented nothing, but were simply there to thwart efforts at frequency analysis (Singh, 1999: 29).

With every advancing step, however, the algorithms for applying ciphers grew ever more complex, and the likelihood that a poor, tired human enciphering messages might make a mistake and garble some part, or even the whole message, grew ever greater. Inevitably, as war and commerce became industrialised and science and engineering advanced, we began to turn to technology to help us. Like all our efforts in codemaking, historically this technology started out in charmingly simple form. One example is the Caesar shift coin or medallion – a small disc with a rotating centre. The outer edge of the coin had one alphabet or set of symbols engraved around it, and the rotating centre had another alphabet or set of symbols engraved round it. When a message was received, the centre disc would be turned to some pre-agreed setting, thus lining up the two alphabets against each other with a shift of, say, five, and one could quickly refer back and forth between medallion and ciphertext to decode the message. In a remarkably short space of time, however, the technology leapt forward, and in a matter of decades we progressed from little medallions to large machines.

Enigma: mechanising cryptography

Between WWI and WWII, the Germans worked feverishly on their cryptographic techniques, and created several extraordinarily sophisticated mechanical ciphering devices. Entire books have been written about all of these, and all are worthy of their own podcasts, but for the purposes of this miniseries, we’ll focus on just one, and the best known of all: Enigma (Dziewanowski, 1994: 381).

The origins of Enigma are somewhere between quaint and surprising. German inventor Arthur Scherbius co-founded engineering firm Scherbius & Ritter with his friend Richard Ritter. Scherbius had studied electrical engineering and he was particularly interested in how the cryptographic techniques used in WWI could be improved using advances in technology (Singh, 1999: 127). Enigma was Scherbius’ invention, and his vision for the future of uncrackable, secure communications, and he applied for a patent for it in 1918 (Mollin, 2005: 91; Singh, 1999: 137).

To look at, the Enigma machine was, frankly, unremarkable, even a little ugly. Far from looking like a machine that contained heretofore unknown cryptographic power, it instead resembled something like a cash register crossed with a typewriter. It also weighed around 12kgs (Mollin, 2005: 91; Stripp, 1993: 83) and for good measure, it was also incredibly expensive to produce. Given its eye watering price and its generally uninspiring appearance, it’s perhaps no surprise that Scherbius was initially unsuccessful in attempting to sell Enigma either to business or to the German military (Singh, 1999: 138). But events in the wider world triggered an unexpected chain of events that would eventually work in Scherbius’ favour.

In 1923, at the same time that an unknown individual by the name of Adolf Hitler was in prison for a failed coup in Munich, the publication of Winston Churchill’s The World Crisis alerted the Germans to the fact that British cryptographers had successfully cracked German ciphers during WWI – a breakthrough which had afforded them significant military advantage. The German military were forced to confront the evident failure of the security of their communications, and it was agreed that Scherbius’ mechanical cipher, Enigma, represented the best chance they had to avoid making the same mistakes in future (Singh, 1999: 142). A mere two years later, by 1925, just as the first volume of Hitler’s Mein Kampf was being published, Enigmas were being mass-produced by Scherbius, both to businesses and to the military, but those he sold to the military had different internal wiring to the commercial versions he had previously put on the market. This is a crucial detail that comes back up later, so try to keep hold of it if you can.

But how does Enigma actually work? There are several key elements to the machine: the lampboard and keyboard, the scramblers, the reflector, and the plugboard. I’ll explain each as briefly as I possibly can, and as I go through remember that cracking Enigma in the end required the cryptanalysts and scientists to mentally reverse every single one of these steps accurately. It’s only in knowing the machine’s complexity that you begin to appreciate the mental agility that was required to defeat it.

Lampboard and keyboard

The keyboard was, unsurprisingly, the way in which Enigma operators inputted plaintext letters. The keys were arranged according to German typewriters used at the time (Oleksiak, 2014). And the lampboard was effectively a display board made up of lamps (Singh, 1999: 127). When the operator pressed a key on the keyboard, an electric pulse would travel in through the scramblers, bounce off the reflector, and then travel back out, illuminating the ciphertext letter on the lampboard (ibid.).

Scramblers

So what are these scramblers? Each Enigma housed of a number of disc-shaped turning rotors – that is, scramblers, and a fixed-in-place reflector. Imagine, if you will, a layer cake turned on its side, with each layer being a scrambler, and the base being the reflector. This encryption cake is the heart of the machine and transforms the plaintext into ciphertext (Kuhl, 2007: 326). Around the circumference of each scrambler are the twenty-six letters of the alphabet, but, each scrambler is wired up in a different way, so all the scramblers within one machine are different to each other. However, each military Scrambler 1 was the same as all the other Scrambler 1s, each Scrambler 2 was the same as all the other military Scrambler 2s, and so on. This stands to reason. If the scramblers in one machine were wired up differently to those in another, then a message encoded on one Enigma could not be decoded on any other, rendering the whole effort entirely useless. It was, in short, crucial that each machine could provide an enormous range of different settings to keep the Enigma encryption as secure as possible through continual changes, but, at the same time, it had to be possible to calibrate all the different machines to the exact same settings as each other so that each could decrypt any messages that had been encrypted by any other.

To ensure uniformity of settings across the entire theatre of war, all Enigma operators would receive Enigma codebooks containing a month’s worth of daily keys and settings. And at the stroke of midnight, the previous day’s settings would be discarded and the new day’s settings would be implemented. This continual and exhaustive method of instantiating new settings every twenty-four hours was crucial. Sometimes Enigma messages could be individually, laboriously cracked, perhaps because a spy had learned enough to reconstruct part of the message, or because the operative had been especially careless with length, or because some unfortunate individual had been encouraged to give information, or simply because some cryptanalyst had been struck with a blinding moment of extraordinary insight and had made headway. Through one decipherment, others messages could sometimes be partially or even fully cracked too. But changing the settings at midnight every day ensured that any weakness that cryptanalysts were able to exploit was temporary at best, lasting only until midnight, and the new settings took effect (Rejewski, 1981: 216).

Back to the actual operation of the scramblers though. They functioned just like the seconds, minutes, and hours on a clock. The first scrambler, like the hand counting the seconds, turned one position every time a plaintext letter was typed in, so that, crucially, even if you typed the same letter three times in a row, it would be encrypted as three different letters because with each press of the keyboard, the first scrambler moves on one setting. This is crucially distinct to the Caesar shift cipher which is a monoalphabetic cipher. In a monoalphabetic cipher, the same plaintext letter will always be encoded to the same ciphertext letter, thus, mono. By contrast, the rotating scramblers make Enigma a polyalphabetic cipher, meaning that the same plaintext letter will be encoded in multiple different ciphertext letters.

If you wanted to risk blowing your whole military operation clean out of the water in a moment of incredible stupidity, you could just keep on pressing the same letter over and over and get a long string of random-seeming letters back. Just like the second hand completing one rotation of the clock face, once the first scrambler has made a full rotation around all twenty-six letters, the next scrambler will move on one setting, in the same way that the minute hand will now click forward to mark one minute. Then, once this middle scrambler has made a full rotation, the end scrambler will rotate one setting, just as when the minute hand travels all the way round the clock, the hour hand finally clicks forward to mark one hour (Kuhl, 2007: 326-7).

For added security, it was also possible to put the scramblers in starting at different letters. Remember that the wiring in each of the scramblers is different, but if every operative always starts the day with all three scramblers reset to exactly the same positions as the day before, and the day before that, and so on, you immediately introduce that most dangerous of weaknesses – a pattern. Instead, the settings for that day might dictate that you put the first scrambler in starting at G, and then the second starting at Q, and the third one starting at A, or whatever. With just three scramblers, each starting at any one of twenty-six positions, you now already have 17,576 different possible starter settings of the scrambler system, but this is just the beginning, as we’ll see in another moment (Kuhl, 2007: 326).

By using multiple scramblers, the problem of repeating patterns, and also patterns in language itself – like highly frequent Es and rare Zs in English – is reduced to an almost imperceptible level. Remember that frequency analysis rests on being able to count the occurrence of the same letters appearing over and over, but since this is a polyalphabetic cipher, the same letter will be encrypted as different letters over and over, and this makes basic frequency analyses useless. But let’s imagine that you’ve gone back to that moment of idiocy, and you keep continually bashing A on the keyboard for some reason, perhaps a hundred times, and then you send this message, and then it’s intercepted by the enemy. For twenty-five bashes at least, you’ll have produced different letters, but after the twenty-sixth, if your Enigma has only one scrambler in it, then it will return to its original position and the pattern will begin again. In a 100-character message, your pattern will recur fully three times and will be cut off towards the end of the fourth iteration. Even an absolutely entry level cryptanalyst should spot the repetition, and a good one might even be able to discern what you’d done, and use this to infer the settings for the whole of that one scrambler. Generally, however, military operatives sending critical messages are not so stupid, they don’t only use one scrambler, and they actually type meaningful things. But even if the person using the Enigma were typing a real message – let’s say, an intelligence report on a new weapon being devised by the enemy – if they were using just the one scrambler, should that message happen to be quite long, then this could be enough for a gifted cryptanalyst to make some sort of headway (Singh, 1999: 130). This is because, as I’ve said, after every twenty-sixth letter, the scrambler returns to its original position, and a long enough message might contain just enough common letters and other clues to start to suggest some ways in.

So, the more scramblers, the more secure your encryption, because even if you somehow slide off your saucer and keep hitting the same letter over and over, it will take that much longer for the pattern of encryption to repeat. With two scramblers, you’d have to type 676 letters before you’ve sent the first scrambler round twenty-six times, and the second scrambler round just once, thereby completing the pattern and starting again. With three scramblers, you’d have to type 17,576 letters – that’s around a third of the words in this episode – before you’d sent the first scrambler round 676 times, the second one round twenty-six times, and the third one round once. Again, this is just like a clock, where the second hand has to complete 43,200 rotations, and the minute hand 3,600 rotations, for the hour hand to travel all the way round a twelve hour clock face just once. If you were to really ramp it up and put in four scramblers, you’d have to type almost half a million letters – that’s about three times more words than this entire miniseries contains, or about the length of a good-sized novel like Harry Potter and the Prisoner of Azkaban – to get from the start to the end of one complete encryption cycle.

Essentially, with each new scrambler, the pattern in the encryption grows exponentially larger. And as an added precaution, since German cryptographers knew that enemy cryptanalysts were looking for any patterns they could find in longer messages, they tended to keep their messages short too (Kuhl, 2007: 327). This might sound needless, but remember, the enemy was intercepting everything it possibly could, and whilst a pattern might not appear in one short message, over dozens or hundreds of messages sent during the course of a day, there might then be enough text to start unpicking the encryption.

But there’s a practical limitation to the scramblers. Remember that the Enigma machine was already heavy and it needed to be dragged onto ships, up and down mountains, into remote war encampments, and so on. More scramblers might be more effective, but there comes a point at which the sheer size and clumsiness of the machine renders it more of a hindrance than a help. As a result, three was the standard, at least at first. But even for this limitation, there was an elegant solution.

As I’ve said, each scrambler in a set was wired up differently, but all Scrambler 1s were wired up the same, all Scrambler 2s were the same, and so on. Scherbius made the scramblers removable, so that it was possible to put them into the machine in different orders, and each machine also came with five scramblers in total (Singh, 1999: 134). This meant that you could pick any three of the five scramblers, and also put them into the machine in any one of six distinct arrangements (Stripp, 1993: 84). If you remember, there are already 17,576 different ways to align the three scramblers anyway, so we might start one on Q and the next on A and the third on C, but this is then multiplied by the six different arrangements of the scramblers that one can choose from, creating an incredible 105,456 different possible starting settings for the Enigma (Kuhl, 2007: 326). Starting to get how complex this is? And we’re not done yet, because if all the German cryptographers encrypted all messages in the space of one day with the scramblers in the exact same initial orientation, this would introduce a “letter-for-letter substitution” (Rejewski, 1981: 216) that would be much easier for cryptanalysts to break. The Germans decided that the best way to overcome this would be to allow the German cryptographers to randomly select different scrambler orientations for each message. This they referred to as the “message key” (Rejewski, 1981: 216) or the “indicator” (Sebag-Montefiore, 2000: 40).

But we’re back to that age old problem: how would the message key be communicated to its recipient? The German cryptographers opted to begin every new message with the three-letter message key, encrypted twice. This means that every message sent by the Germans began with six letters (Rejewski, 1981: 216). To achieve this, the cryptographer would do the following:

1. Turn the Enigma scramblers into the initial position specified in the codebook for that day, e.g., CBA.

2. Choose a message key for that message, e.g., ENC, which they would type into the Enigma keyboard twice.

3. Two enciphered versions of ENC, the message key, would then light up on the lamp board. Let’s say this becomes ZLT FGB. This would be the first six letters of the message.

4. The operative would then change their scrambler orientations to match this key – that is the first scrambler would be rotated to E, the second to N, and the third to C, and then they would continue with the rest of their message.

But how does the recipient determine the message key. They essentially reverse the procedure thus:

1. Turn the Enigma scramblers into the initial position specified in the codebook for that day, in this case it was CBA.

2. Type in the encrypted six letters at the beginning of the message, which in our example were ZLT FGB.

3. Look at the letters lighting up on the lampboard, and check that it spells out the same three letters twice, so in our case, that would be ENCENC. If they didn’t get two matching sets then it would be clear that some sort of error had occurred.

4. Adjust the orientation of their scramblers to ENC, and then go on to decipher the rest of the message.

Reflector

The next key component on the Enigma machine is the reflector. Like the scramblers, the reflector is a rubber disc, but, unlike the scramblers, it is fixed in place and does not rotate (Singh, 1999: 133). When the operative types in a letter, that letter sends an electrical signal through the three scramblers to the reflector. As the name suggests, the reflector then bounces that signal back through the same three scramblers, but this time it goes via a different route. This has to be the case for two reasons. If the reflector sent the pulse back through the same route, you’d get back out the very letter you put in, but more importantly, the reflector is crucial for decrypting other Enigma messages. To encrypt a plaintext message, one types each letter, almost as if typing the message out, and jots down each letter that lights up on the lamp board, thereby creating the encrypted output, or ciphertext. And then, to decrypt an Enigma ciphertext, one ensures that the machine is calibrated to the same settings as the machine that produced the encrypted message, and then one simply types in the ciphertext, jotting down each time the letters that light up on the lamp board. Each ciphertext letter will be converted back to its plaintext letter, thus revealing the underlying message. In other words, when using an Enigma machine, “encipherment and decipherment are mirror processes” (Singh, 1999: 133).

But there’s one more component to enhance the level of complexity yet further.

Plugboard/steckerbrett

On the front of the Enigma machine is a plugboard known as a “steckerbrett” (Oleksiak, 2014). Imagine one of those old-fashioned telephone switchboards, except this one only had twenty-six sockets, one for each letter on the keyboard (Kuhl, 2007: 327). The plugboard is inserted between the keyboard and the first scrambler, and allows the cryptographer to insert cables to swap some of the letters around before they enter the rotary system of scramblers. For instance, the cryptographer could insert a cable which connects two letters of the alphabet, such as a and b. Now when the cryptographer wants to encrypt the plaintext letter b, the electrical signal running through the machine will ensure that b follows the same pathway through the three scramblers that was previously the path for the letter a (Singh, 1999: 134). In 1932, Enigma had six plugs, which meant that cryptographers could swap six pairs of letters. The remaining fourteen letters of the alphabet would not be swapped (Kuhl, 2007: 327; Singh, 1999: 135). The combination of both scramblers and the plugboard is what makes Enigma so incredibly difficult to crack because of the sheer number of possible permutations it generates (Kuhl, 2007: 327). In addition, the swapped letters had no underlying mathematical explanation that could be deduced – they were simply chosen at random by the operator (Oleksiak, 2014).

Cracking Enigma by hand

With all of the above components in mind, the key to any message decrypted using Enigma has several vital elements. To decrypt the secret message with your own Enigma machine, you would need to know which of the 17,576 possible scrambler orientations had been chosen, which of the six possible orders of scramblers was implemented, and which of the 100,391,791,500 possible choices of plugboard switches the operator had randomly decided on. These three elements taken together create over ten quadrillion possible settings, so that simply guessing at the right settings is essentially futile. And this therefore made the codebooks with their daily keys almost as precious as the Enigma machines themselves.

How, then, did the battle to crack Enigma begin? As war rages across Europe, and then the world, in the next part we begin to meet the civilians, scientists, and spies who all found themselves caught up in one of the most deadly races against time in modern history.

End of part 1 of 3.

Outro

This episode of en clair was researched and fact-checked by Rebecca Jagodziński. And it was scripted, narrated, and produced by me, Dr Claire Hardaker. And it was supported by the Economic and Social Research Council as part of their Festival of Social Science. However this work wouldn’t exist in its current form without the prior effort of many others. You can find acknowledgements and references for those people at the blog. Also there you can find data, links, articles, pictures, older cases, and more besides.

The address for the blog is wp.lancs.ac.uk/enclair. And you can follow the podcast on Twitter at _enclair. If you like, you can follow Rebecca on Twitter at RjJagodzinski, and you can follow me on Twitter at DrClaireH.