NOTICE: Citizendium is still being set up on its newer server, treat as a beta for now; please see here for more.
Citizendium - a community developing a quality comprehensive compendium of knowledge, online and free. Click here to join and contribute—free
CZ thanks our previous donors. Donate here. Treasurer's Financial Report -- Thanks to our content contributors. --

Signals intelligence before the Second World War

From Citizendium, the Citizens' Compendium
Jump to: navigation, search
This article is developed but not approved.
Main Article
Talk
Related Articles  [?]
Bibliography  [?]
External Links  [?]
Citable Version  [?]
 
This editable Main Article is under development and not meant to be cited; by editing it you can help to improve it towards a future approved, citable version. These unapproved articles are subject to a disclaimer.

Until the development of radar and other electronics techniques, signals intelligence (SIGINT) and communications intelligence (COMINT) were essentially synonymous. Sir Francis Walsingham ran a postal interception bureau. with some cryptanalytic capability. during the reign of Elizabeth I, but the technology was only slightly less advanced than that of the men with shotguns who, during World War I, jammed pigeon communications and intercepted the messages carried.

SIGINT became far more central to military (and to some extent diplomatic) intelligence generally with the mechanization of armies, development of tactics requiring close coordination of forces, use of submarine and commerce raider warfare, and the development of practicable radio communications. Even measurement and signature intelligence (MASINT) preceded electronic intelligence (ELINT), with sound- and flash-ranging techniques for artillery location. SIGINT is the analysis of intentional signals for both communications and non-communications (e.g., radar) systems, while MASINT is the analysis of unintentional information, including, but not limited to, the electromagnetic signals that are the main interest in SIGINT.

Analysis of written communications

This was principally the province of postal interception, as well as surreptitious entry into diplomatic pouches. Of course, once a cryptanalyst has a message in character form, it made little difference, until the advent of computerized cryptosystems operating at the stream or block level, whether the characters were copied from signaling lights or flags, elegantly written by a clerk, or intercepted from a wired telegraph.

In general, the more secure cryptosystems were codes, not ciphers. Thomas Jefferson did invent a partially mechanized system in the early 19th century, but did not realize the power of what he had created, sending out the Lewis and Clark expedition with a simple manual cipher. The system was reinvented by Bazeries in the First World War, and, changed from a set of cipher wheels on an axle (e.g., U.S. M-94), to a sliding strip system, only began to yield to machine-assisted German cryptanalysis in the Second World War. By then, however, the technique still required human pattern recognition, and simply was too slow for the volumes of traffic to be sent.

World War I

Radio communications were fairly new at this time. At the strategic level, nations gained access to commercial cable traffic. Tactically, wired telephones were in wide use, and techniques of intercepting them through ground returns were developed. These intercept techniques have had a resurgence in later wars, where radio was less available or impractical. On the declaration of war, one of Britain's first act was to cut German undersea cables, forcing them to use radio, which the British could intercept. The practice of destroying more secure wired communications, to improve the intelligence take, has been a regular practice since then. While one side may be able to jam the other's radio communications, the intelligence value of poorly secured radio may be so valuable that there is a deliberate decision not to interfere with enemy transmissions.

Russia, when preparing for the Russo-Japanese War in 1904, had established a tradition of poor communications that would last well after the fall of the Romanovs. The success of this and related salvage and rescue work persuaded the Russian Navy to install wireless sets on many of its ships. In early 1904, the Russian fleet prepared for war with Japan. The British almost immediately began to intercept their communications, with the complaint “An intelligence report on signals intercepted by HMS Diana at Suez shows that the rate of working was extremely slow by British standards, while the Royal Navy interpreters were particularly critical of the poor standard of grammar and spelling among the Russian operators.”[1] After such an embarrassing start, the great Russian mathematical tradition, and probably the national passion for chess, has made Russia, regardless of other shortages, a great innovator in communications and cryptology.

Failure to properly protect its communications fatally compromised the Russian Army in its advance early in World War I and led to their disastrous defeat by the Germans under Ludendorff and Hindenburg at the Battle of Tannenberg. Similarly, the interception and decryption of the Zimmerman telegram was an important factor in the US decision to enter the War.

Radio researchers at the British Marconi company realized that strange signals they were receiving were German naval communications, and brought them to the Admiralty. Soon, the British were operating a network of listening posts called "Y Stations", with Admiralty Room 40 doing the traffic analysis and cryptanalysis [1]. In World War II, the British referred to their traffic analysis function as the "Y service".

In contrast, battles have been lost, or not fought, when senior commanders asked the traffic analysts and direction finders, the wrong question. In World War I, British Admiral John Jellicoe, knew a little too much detail about SIGINT without fully understanding it. He asked the analysts where call sign "DK" was located [2]. DK was the headquarters of the German High Seas Fleet. The analysts answered his question precisely, telling him that it was "in the Jade River". Unfortunately, Jellicoe did not know that the High Seas Fleet commander used a different identifier when at sea. Jellicoe assumed the German fleet was also in the Jade River, and missed an opportunity for a decisive battle. When he found out the true situation, he lost faith in SIGINT, not accepting that it was his own error.

Jellicoe's faith in cryptographic intelligence was also shaken by a decrypted report that placed the German cruiser Regensburg near him, during the Battle of Jutland. It turned out that the navigator on the Ravensburg was off by 10 nmi in his position calculation. During Jutland, there was limited use of direction finding on fleet vessels, but most information came from shore stations.

France had significant SIGINT in World War I. While the key intelligence achievement in blunting the German drive on Paris in June 1918 was the cryptanalysis of Georges Painvin, had French intercept personnel not captured the message in the ADFGVX cipher, there would have been nothing to cryptanalyze.

Between the World Wars

There was substantial SIGINT work between the World Wars, although the secrecy surrounding it was extreme. While it was primarily COMINT, ELINT emerged with the development of radar.

Both sides developed direction-finding (DF) and communications interception stations during the war, although those programs often began with naval search & rescue.

Canada

Canada's first SIGINT intercept site, Special Wireless Station #1, was built in 1939, in Ottawa. "#2 SWS was located at Grande Prairie, Alberta and #3 SWS at Victoria B.C. Victoria also had a remote High Frequency Direction Finding (HF/DF) site in Nanaimo approximately 60 mi to the northwest." [3] 1 Canadian Special Wireless Group (1CSWG) deployed to Australia in January 1945.

Germany

By the mid-twenties, German Military Intelligence Abwehr was intercepting and cryptanalyzing diplomatic traffic. Under Hermann Goering, the Nazi Research Bureau (Forschungsamt or “FA”) had units for intercepting domestic and international communications. The FA was penetrated by a French spy after 193, but the traffic grew to a point that it could not easily be forwarded. In addition to intercept stations in Germany, the FA established an intercept station in Berne, Switzerland. It penetrated most cryptosystems other than the UK and US. [1]

German Condor Legion personnel in the Spanish Civil War ran COMINT against their opponents.

United Kingdom

After the First World War, British Army and Navy COMINT merged and formed a new organization, reporting to the Admiralty, called the "Government Code and Cypher School", with Alastair Denniston as its first head.

While it was operational in 1919, it was realized most of its current work was diplomatic, so it was transferred to report to the Foreign Office. Both GCCS and the Secret Intelligence Service reported to Hugh Sinclair, in London. In May 1927, Prime Minister Stanley Baldwin made public some GCCS solutions of Soviet intercepted message, causing a massive Soviet cryptographic change.

By 1940, GCCS was working on the diplomatic codes and ciphers of 26 countries, tackling over 150 diplomatic cryptosystems [4].

United States

Naval direction finding and message interception

US communications monitoring of naval signals started in 1918, and continued, but was used first as an aid to naval and merchant navigation. In October 1918, just before the end of the war, the US Navy installed its first DF installation at its station at Bar Harbor, Maine, soon joined by five other Atlantic coast stations, and then a second group of 14 installations. [5]. These stations, after the end of World War II, were not used immediately for intelligence. While there were 52 Navy MF DF stations in 1924, most of them had deteriorated. The Navy transferred, in July 1941, the remaining stations to the US Coast Guard.

As tension with the Japanese grew, the COMINT situation was being reviewed. In the early 1930s, the Navy started implemented HF/DF. Eleven locations were planned, primarily on the Atlantic Coast, and beginning with Bar Harbor, Maine, early 1936. The first operational intercept came from what would later be called Station CAST, at Cavite in the Philippines. There were still technical problems, a development program started, and the first advanced station created at Winter Harbor. In July 1939, the function turned from training and R&D to operations, and the Navy officially established a Strategic Tracking Organization under a Direction Finder Policy.

By December 1940, the Navy's communication organization, OP-20-G, had used HF/DF on German surface vessels and submarines. Training continued and cooperation with the British began. In April 1941, the British gave the US Navy a sample of their best HF/DF set from Marconi.

All remaining navigational DF stations were transferred to the Coast Guard in May 1941, and the Navy concentrated its efforts on COMINT, reporting to OP-20-G under Commander Laurance F. Safford. By December 1941, the Navy established a strategic HF/DF and intercept station, with Atlantic, Pacific and West Coast net control stations managing 20 sites. Increasingly, new site selection emphasized COMINT value over HF/DF. The prototype intercept station had been in Maine, initially in Bar Harbor but relocated to Winter Harbor in 1935. It principally intercepted European traffic to Tokyo, but also had a section intercepting Soviet traffic. Intercept reorganization came during the first week of September, with Atlantic stations reemphasizing HF/DF, and interception at Jupiter, Florida and Cheltenham, MD. The Cheltenham station was replaced by Chatham, Massachusetts as the primary intercept station. [5]

Ship platforms

Beginning in 1937, US naval ships started intercepting communications, beginning with the destroyer USS Hatfield (D-231). It anchored at La Rochelle-Paris and started operating, but lack of intercept training quickly became evident. The Director of Naval Communications established policies and procedures; it should be noted that COMINT reported Communications, not Intelligence.

The European squadron, 40-T, with USS Raleigh (CL-7) as flagship, originally was assigned to evacuate civilians from the Spanish Civil War, but a secondary COMINT duty became evident, with a unit established on the USS Omaha (CL-4) in 1938, soon designated as Station F, intercepting German, and Italian traffic, later in the Mediterranean.

Station F moved to the new flagship, USS Trenton (CL 11) in June 1939. They noted significant communications changes two days before the German invasion of Poland, and the intelligence significance was noted and forwarded to Washington. In 1939, the Atlantic was the priority, with a very short belief Japan was not a threat. In 1940, formal liaison began with the British, under the terms of a highly secret policy accepted in 1937. The Special Naval Observer in London was the point of contact, and formal COMINT exchange began in November 1940.

Station F, still on ships, concentrated on Italian traffic in 1940. OP-20-G began cryptanalytic work in July 1940. OP-20-G was acutely aware of British ship losses in the Battle of the Atlantic. Focus remained on the Atlantic, with Agnes Driscoll, the chief cryptanalyst under LT Lee W. Parke, worked on Italian systems; German system remained secure.

Increased pace in 1941

In 1941, the sensitivity of COMINT increased when the US gave Britain a PURPLE machine. The British did not reciprocate with full ULTRA information on ENIGMA, but the US received paper information on ULTRA as well as current British HF/DF procedures.

Operational priority increased when Winter Harbor and Amagansett received teletypes for faster relay of intercepts to the analysts in Washington. While the emphasis was now on Japanese traffic, other traffic of interest was still studied.

COMINT against the inter-war Japanese

COMINT of Japanese traffic proved invaluable to the Allies at the Washington Naval Conference in 1921, through cryptanalysis by Herbert Yardley. Then-Secretary of War Henry Stimson closed the US Cipher Bureau with the words "Gentlemen do not read each other's mail." Luckily for US COMINT, the Army offered a home to William Friedman after Stimson closed the Yardley operation.

Friedman's team had three analysts that would become bright figures in American cryptology: Solomon Kullback, Frank Rowlett, and Abraham Sinkov. Kahn's memorable comment "If Yardley was the star of American cryptology, Friedman was the Sun" remains apt. They developed largely manual cylindrical and strip ciphers, but, as a result of Friedman's advances in cryptanalysis, machine ciphers became a priority, such as the M134, also known as the SIGABA. While the SIGABA was a rotor machine like the German Enigma, it was never known to be cracked. It was replaced by electronic encryption devices.

SIS, in contrast with Yardley's dependence on cable companies, set up its own radio intercept organization. Eventually, the training and intercept functions were separated for both administrative and security reasons, when, a centralized signals intelligence unit, the 2d Signal Service Company, was set up at Fort Monmouth on 1 January 1939 to control all Signal Corps personnel at the permanent monitoring installations. In this period, SIS paid less attention to tactical SIGINT.

SIS Intercept Stations, 1939
Station Primary role Secondary role
W (Winter Harbor) Italian Naval Axis Diplomatic
M (Cheltenham) German Naval Axis Diplomatic
G (Amagansett) Diplomatic None
J (Jupiter) Diplomatic None

References

  1. 1.0 1.1 1.2 Bartholomew Lee. Radio Spies -- Episodes in the Ether Wars.
  2. David Kahn (1996). The Codebreakers - The Story of Secret Writing. Scribners. ISBN 0684831309. 
  3. Troyanek, Jim. The "Special" Wireless Stations.
  4. David Alvarez, GC&CS and American Diplomatic Cryptanalysis
  5. 5.0 5.1 Clancey, Patrick. Battle of the Atlantic, Volume I. Allied Communications Intelligence, December 1942 to May 1945 [SRH-005]. HyperWar: A Hypertext History of the Second World War. HyperWar Foundation.