“Cryptographers at Bletchley Park use a Colossus computer to decrypt German military communications during the Second World War”

Marking a century of state surveillance at GCHQ
by Andrew Robinson / 20 October 2020

“Most professional scientists aim to be the first to publish their findings, because it is through dissemination that the work realises its value.” So wrote mathematician James Ellis in 1987. By contrast, he went on, “the fullest value of cryptography is realised by minimising the information available to potential adversaries.” Ellis, like Alan Turing, and so many of the driving forces in the development of computers and the Internet, worked in government signals intelligence, or SIGINT.

Today, this covers COMINT (harvested from communications such as phone calls) and ELINT (from electronic emissions, such as radar and other electromagnetic radiation). Ellis and Turing are just two of the many code-breakers and code-builders in Behind the Enigma, the first authorized history of one of the world’s pre-eminent secret intelligence agencies, GCHQ, the United Kingdom’s Government Communications Headquarters. Famous for its Second World War decryption of the German Enigma cipher at Bletchley Park, there is so much more to this secrecy-shrouded outfit, reveals Canadian historian John Ferris.

Fielding formidable research, Ferris tells a global tale of mathematics, engineering, data sciences and linguistics in the service of politics, diplomacy, war and security. Spanning a century, it ranges from telegraphic intercepts to malware that can bring down infrastructure. After a brief introduction to pre-1914 intelligence based on letters, cables and wireless messages, his story begins with First World War cryptography and the foundation of GCHQ in 1919 as the Government Code & Cypher School.

It ends with the agency’s current, not-so-secret incarnation as a protector of the cyber commons. In September 2001, the director of GCHQ crossed the Atlantic on the only aircraft allowed into the United States immediately after the al-Qaeda attacks, to work with his US opposite number. What emerges is that SIGINT has ranged from highly effective to almost useless.

In July 1962, a few months before the Cuban missile crisis, GCHQ picked up enciphered Soviet messages suggesting that two Soviet passenger and cargo ships were “possibly en route Cuba” and that their voyages might be “other than routine”. But there was no hint of the ships’ purpose and content. Then, in mid-October, a US U-2 spy plane detected the first proof of Soviet missiles in Cuba, triggering the crisis.

Two weeks later, soon after US president John F. Kennedy’s announcement of a naval blockade of Cuba, GCHQ detected a flurry of urgent enciphered messages sent from Moscow to Soviet ships. Thus, SIGINT helped to alert and inform governments, but the US political decision depended on ground observations by the military.

By contrast, at the end of the Falklands War against Argentina in 1982, the commander of the British task force declared that, without GCHQ’s advance penetration of the Argentine plan of attack, mainly through COMINT in Spanish, the invasion would have failed at sea. But once the soldiers landed on the Falkland Islands, SIGINT failed them in battle, because of the improvised nature of the chain of command.

Central to these events was UKUSA, or ‘Five Eyes’ — which receives frequent mention in the book. This is the still-operative multilateral agreement for cooperation in SIGINT between Australia, Canada, New Zealand, the United Kingdom and the United States. It was inaugurated between GCHQ and the US National Security Agency in 1946, at the beginning of the cold war, but its existence was concealed from the public until 2005.

Intriguing are the backgrounds and mindsets of past and present GCHQ staff — today 6,000 in number, compared with 10,000 at its wartime peak — and their working conditions, breakthroughs and varied relationships with peers in other countries. Of their US counterparts, retired GCHQ director David Omand joked to the BBC in 2013: “We have the brains. They have the money. It’s a collaboration that’s worked very well.” Certainly, GCHQ mathematicians were often secretly ahead of the academic game.

For example, in 1970 Ellis came up with the possibility of “secure non-secret digital encryption”, but could visualize no way to implement it. In 1973, a younger colleague, Clifford Cocks, later chief mathematician at GCHQ, realized Ellis’s concept by inventing the public-key system now known as the RSA encryption algorithm. Its name derives from Ron Rivest, Adi Shamir and Leonard Adleman, who invented it independently in 1977 in the United States.

In 1974, another GCHQ mathematician, Malcolm Williamson, devised the technique for public exchange of a common secret key between two parties that later became the basis for all secure transactions on the Internet. This one is also named after US cryptographers — Whitfield Diffie and Martin Hellman — who discovered it independently in 1976. Only in 1997 were these two crucial GCHQ discoveries declassified.

Even in 2020, writes Ferris, “Siginters feel disquiet when they see the name GCHQ in press headlines”. Often recruits were linguists, sometimes with unusual skills. During the Second World War, many were gifted academics from Oxford and Cambridge universities (although GCHQ turned down Oxford’s J. R. R. Tolkien, despite his mastery of languages).

One notable was the young Cambridge classicist John Chadwick. He took a crash course in Japanese in 1944 to help decrypt messages sent by Japanese naval representatives working in wartime Berlin and Stockholm. Post-war, Chadwick, with architect and philologist Michael Ventris, deciphered Europe’s earliest readable script, Minoan Linear B, an archaic form of Greek.

Bletchley’s staff was famously more than 75% women. Compared with “virulent sexism” in the computing industry, Ferris notes of GCHQ in the 1930s that, “once inside, the standards were those of flair, not gender”. But the stories of notable women from those days are still only now coming to light, as witness the dearth of female portraits in the book’s plate sections.

Staff included linguist Emily Anderson, a former professor of German who became a world-leading cryptanalyst in the 1930s, and mathematician Joan Clarke, who used Bayesian statistics to speed decryption at Bletchley, where she collaborated with (and was briefly fiancée of) Turing. These days, the organization — like most in cybersecurity — realizes that it has a sizeable gap to close on gender balance in its workforce; Ferris doesn’t dwell on that. Inevitably, official secrecy limits this analysis — as do the author’s academic interests (more military than scientific).

In inviting Ferris, GCHQ ruled out discussion of diplomatic-communications intelligence from after 1945 and the technicalities of current methods. Other intelligence agencies, such as the US National Security Agency, had power of veto over details of joint projects. Also off limits were records of the period after the end of the cold war in 1991. For these decades, Ferris had to interview current staff, mostly under ‘deep background’.

“Bombe unit room in Eastcote, similar to one at Bletchley Park. Bombe machines were used extensively to crack coded German military communiqués.”

Thus, the 1990–2020 era is covered less critically. In a discussion of the 2013 leaks about UKUSA surveillance by National Security Agency contractor Edward Snowden, which were followed by a UK government inquiry into GCHQ, Ferris rejects the charge that the agency collects intelligence on everybody, regardless of their risk to UK security. His unsatisfying take is that their sins are more of omission than commission. He writes: “GCHQ did not openly address the operational and legal elements of bulk collection because it did not know how to do so, rather than having anything to hide.”

Today, a secure cage in GCHQ’s basement archives contains the vetting records of each member of staff, collected from interviews with friends and families before hiring. These were unavailable to Ferris. Each record is destroyed when the member dies. “Nothing better typifies GCHQ than this focus on privacy for people who strip secrecy,” he writes. Perhaps this is why even the deceased in this pioneering history seldom come alive as individuals. For all Ferris’s scholarly sleuthing, not even Turing — a key contributor to decrypting Enigma, and a globally compelling human enigma — really emerges from the shadows.”

Why NSA Told Kissinger to Drop Dead When He Tried to Cut Intel Links with Britain
by Nico Hines  /  Oct. 23, 2020

“Henry Kissinger once tried to come between the National Security Agency (NSA) and Britain’s GCHQ, their signals intelligence (SIGINT) brothers from the other side of the pond, and the response from the U.S. intelligence agency was short and swift. “The NSA simply said, ‘Drop dead,’” says the author of a new authorized history of GCHQ, who explains that the two intelligence agencies have a closer relationship with each other than they do with their own governments. The world’s two leading signals intelligence agencies are so tightly bound together that they share virtually all of the material they gather with no questions asked.

Over the years, GCHQ (Government Communications Headquarters) has frequently protected the NSA from rivals within the U.S. including the CIA and Naval intelligence units—and even their respective presidents and prime ministers come second in the hierarchy of loyalty, according to Behind the Enigma: The Authorized History of GCHQ, Britain’s Secret Cyber-Intelligence Agency by John Ferris.


“I say in the book—and both GCHQ and NSA allowed me to say it—that at some point or another, every director of GCHQ and NSA colludes with each other in order to do something which their own national authority might try to impede,” Ferris told The Daily Beast. One such clash arose in 1973 when Kissinger, who was President Nixon’s national security adviser at the time, ordered the NSA to stop sharing signals intelligence with Britain in order to pressure London to support Nixon’s Israel policy.

The NSA refused to comply, challenging Kissinger’s authority despite his key role at the White House. Ironically, under the shared intelligence agreement between the agencies, Kissinger’s move would have left the U.S. flying blind in the Middle East because collecting signals intelligence in the region was entirely the domain of the British who funnelled the intel back to Fort Meade.

One of the most bizarre aspects of this unparalleled intelligence sharing partnership is that it is not enshrined in any treaty; it’s a subnational, totally non-binding agreement, which makes the NSA’s willingness to stand up to Kissinger even more extraordinary. “If Jeremy Corbyn had been elected with a majority, I think he would have broken it and he could have done so. And if Donald Trump wanted to break it, he could do so. Any British prime minister or American president is free to choose. The problem is they’re so closely intertwined that it would cause massive immediate problems, or huge amounts of expenditure to overcome. That wouldn’t have bothered Corbyn,” said Ferris.

The relationship was also entirely secret for 25 years after World War II. It wasn’t until 2010 that the documents behind the agreement were put into the public domain. This comprehensive book uses unprecedented access to GCHQ files to chart the full history of the agreement, which is called UKUSA (pronounced yoo-kusa, a bit like the Japanese mafia, by those in the know). “The only organization I can think of which in any way comes close is NORAD, the North American air defense system where the Canadian and American air defense systems are integrated. But that’s much more narrow and specific than UKUSA, but that’s the only other thing that comes close. So, yes, this is really unique,” said Ferris, who is a professor of history at the University of Calgary in Canada.

At the end of the Cold War, during which British expertise on intercepting Russian communications had been instrumental, there was a fear that GCHQ’s influence would wane, but the agency, which is based in Cheltenham, southwest England, bucked expectations and repositioned itself as a trailblazer in modern signals intelligence. With the resources freed from exhaustively covering the Soviet Union, GCHQ was able to start doing what it’s really good at, which is exploring new territories—in this case, the early days of the internet, mapping it out for themselves and the Americans and then coming up with new methods of interception and cryptography to suit the new environment.

British paymasters recognized the outsized diplomatic clout they maintained in Fort Meade and in Washington, where GCHQ intel product remains highly respected, so long as the intelligence agency was allowed to thrive, and so investment in the agency remained relatively high despite the end of the Cold War. Ferris was not allowed to detail current intel methods in the book for obvious reasons, but the documents published by Edward Snowden, who was employed by a contractor to work at an NSA facility, give an unmistakable insight into the current balance of the relationship between GCHQ and the NSA.

“If I’m judging simply by material which has been leaked, mostly by Snowden in the past five or six years, my sense is that the British are relatively much more significant than they were at any point since the 1960s,” said Ferris. “If you go through the Snowden material, you’ll find that a huge number of the technical innovations clearly are British, and, in fact, the Americans pay GCHQ to develop them.” The book argues that regular NSA efforts to subsidize GCHQ, which has a much smaller budget and staff, is a sign not of GCHQ’s weakness but of its strength. The British SIGINTers are seen as valuable scouts and innovators who routinely deliver a good return on investment.

As the former director of GCHQ, David Omand, once joked, “We have the brains. They have the money.” This is not to say, the NSA is not filled with brilliant people in itself, and their capacity for intelligence gathering is unparalleled. “The Americans have this raw power, which once focused is overwhelming,” said Ferris. “I would personally say that NSA is one of the most technologically disruptive organizations in history. So, the two of them together are very formidable.”

The relationship between the American and British signals intelligence communities blossomed during World War II, when U.S. pioneers were invited over to Bletchley Park, the legendary home of the codebreakers who cracked Germany’s secret wartime communications. “Genuinely, they were astounded by the quality of every branch of British SIGINT and, in fact, came to understand that what the British were doing was very ahead of us in every single way,” said Ferris.

Anglo-American relations were complicated during the course of the war, with Washington initially reluctant to become embroiled in another predominantly European conflict. After the war, American SIGINTers, with the help of GCHQ input, succeeded in convincing President Harry Truman that a large-scale peacetime SIGINT operation was necessary to ensure there was never a “Nuclear Pearl Harbor.” UKUSA was established in 1946, linking American and British SIGINT efforts ever since. The agreement also took in Britain’s recent Dominions; Canada, New Zealand, and Australia. Together they formed a global network which is now known as Five Eyes.

The relative merits of NSA and GCHQ have fluctuated over the decades. In the ’50s and even early ’60s, GCHQ was still seen as the more impressive intelligence producer. American internal memos bemoaned the supremacy of GCHQ’s final product, which was often deemed better written and more fully analysed. In the latter decades of the 20th century, big American investments in supercomputing and expensive advances, including satellite technology, ensured NSA was in the ascendancy.

The agreement was founded on individual personal relationships between SIGINTers, and sometimes those were rocky. There were complaints that GCHQ was hogging the most prestigious roles; British assessments of American product were sometimes deemed “too rude to share;” and in the mid-’80s NSA Director William Odom complained that GCHQ did not carry out its share of the work given how much authority it demanded.

“The British clearly can’t accept happily their own loss of pre-eminence in this business,” Odom wrote in his remarkably frank diary. “Socially I no longer find the British amusing, merely a pain in the ass.” But throughout it all, NSA and GCHQ, two largely civilian organizations, maintained their togetherness. All of the Five Eyes countries would send senior liaison officers and up-and-coming “integrees” to work at the other agencies, sharing intel techniques and honing each other’s skills.

A no-poach policy ensures that the agencies are willing to let their best and their brightest take part in the exchanges. In Behind the Enigma, Ferris writes: “In one legendary moment, an American integree at Cheltenham and a British one at Fort Meade conducted negotiations between GCHQ and NSA on behalf of their adopted services; in another, every member of a Sigint conference between Australia, Britain, Canada and the United States held a British passport.”

GCHQ is also part of American decision-making. There are lots of interagency meetings and important issues where GCHQ representatives are part of the decision-making process right on U.S. soil. On Sept. 12, 2001, the head of GCHQ was on the only aircraft allowed into the United States immediately after 9/11. General Michael Hayden, former director of the NSA, has since said it was decided in the aftermath that GCHQ would assume command of all American SIGINT if Fort Meade was compromised.

“NSA could trust GCHQ to have its back in a way that it cannot trust any other American agency to have its back. And GCHQ and NSA provide each other with state secrets, which only a handful of other people would see. It is one of the most unusual arrangements I’ve ever seen,” Ferris said. General Hayden, who was director of both the NSA and CIA, was an exception, but there has often been a rivalry between the two agencies which dates back to the 1950s when NSA was created: CIA operatives around the world had previously been responsible for foreign SIGINT collection.

“There was a huge amount of blood on the floor,” said Ferris, and relations were often tough over the decades to come. “There are moments when CIA—for good reasons or bad—is not doing what NSA would like. And GCHQ helps NSA avoid some of those problems. GCHQ has perfectly civil relations with CIA. So, it’s actually easier for GCHQ to get CIA to help NSA than it is for NSA to get CIA to help NSA.”

Many SIGINTers believe UKUSA will eventually fall apart now that the unifying threat of the Cold War has faded away and there is no guarantee that new generations of political leaders will share common foreign policy goals. The strength of the agreement was tested in the Middle East in the ’70s when British and American governments disagreed over Israel, and similarly two decades before when Washington did not support British policy during the 1956 Suez crisis.

On that occasion GCHQ actually hampered British government policy by refusing to cooperate with French intelligence. If the agreement does eventually collapse it will cost the U.S. billions of dollars—Ferris believes the NSA budget would have to increase by around a third—to replace the input from Britain. But even more than that, one of the greatest intelligence-gathering partnerships the world has ever seen would be permanently damaged.”



Leave a Reply