During our vacation in London two years ago, my wife and I stumbled upon a terrific museum in Whitehall - the Cabinet War Rooms. It was an underground command center for Churchill and selected military and civilian advisors. The only evidence of what I'd found was a low-key brass plaque one often sees at "historical places of interest." It gave few details, but my adroit military mind filled blanks, interpolated and immediately developed a sparkling clear image of what I should find. After we finished reading the plaque, I excitedly suggested that we should check it out. Nancy, unimpressed, suggested we continue our independent walking tour to Downing Street not far from where we stood. Her nonmilitary mind obviously did not conjure up the same picture. This was no time for negotiation or explanation. I led her, kicking and screaming, down the short flight of concrete stairs in the outside stairwell to the steel door entrance of the War Rooms. A cashier relieved us of several British Pounds Sterling. We donned cassette recorders and headphones (self-paced guides) and began our tour. It was obvious to me - immediately - that my earlier image was correct. I absorbed the details around me, frozen at WWII time. I felt a powerful connection. In minutes, Nancy also became engrossed. There was every appearance, and the narrator on the tape so stated, that at the close of the war, they merely sealed the "centre" intact for later disposition. No reconstruction or restoration had been necessary. We were looking at precisely the paint, furnishings and the like as Churchill had seen it. Charts, status boards, plots, classified papers on desks, the visitor's log and scratch pads were complete with the final notations, doodles and notes. Everything was in perfect "moment in time" preservation. It looked like someone had sounded a fire alarm, and we were the first to return. You could feel the presence of those plucky Brits. (Nancy now talks about the visit to the War Rooms as the highlight of the trip.) Enough of that; this article is only indirectly related to the Cabinet War Rooms. You see, I have always had a subliminal attraction to "the war years," particularly from the British viewpoint. So, when I find something relating to that era, particularly if it's SIGINT related - as happened recently - I'm attracted to it like a moth to a floodlight. Especially since our trip to London.
Reset your calendar to May 23, 1943 and picture yourself in London. There was a select group of British civil servants, Wrens and Officers celebrating in the Cabinet War Rooms, in the Operational Intelligence Centre (OIC) of the Admiralty in London, and in the Government Code and Cypher School (GC&CS) in Bletchley Park, located fifty miles northwest of London.
Cause for celebration came infrequently in early 1943. Convoys, the lifeblood of the European defense against Germany, were suffering terrible losses. An intelligence report of cumulative merchant ship losses to U-boats in March 1943, listed 108 ships sunk. On May 10, 1943 the Operational Intelligence Center estimated that there were 128 operational U-boats at sea - "the highest ever known." Discussions concerning the viability of the convoy concept were approaching feverish pitch. Voices in Parliament were calling for an investigation on the failures of intelligence and convoy operations.
Winston Churchill was quoted as saying, "The only thing that ever frightened me during the war was the U-boat peril." That statement sets the dynamics and emotion of the moment and places the U-boat threat into perspective. Coming from a man who led Great Britain through V-1/2 attacks and the Battle of Britain, Churchill's statement provides insight into the intensity of the U-boat threat in early 1943. It also shows the state of the perceived threat to British logistic supply lines and the resulting effect on the outcome of the war.
Let's return to the question of why there was celebration. Festivity was in the hearts of the select few British authorized to work with German Navy Enigma cipher system decryptions - known collectively by the codeword ULTRA. The joy was due to a decision made by Grand Admiral Karl Doenitz, Commander in Chief of the German Navy. At that time, he also wore the hat of "Befehlshaber der Unterseeboote (BdU)," Commander of the U-boats. Doenitz had just transmitted directives (decrypted by GC&CS) to his U-boat captains which precipitated the celebration. He reassigned them from the North Atlantic convoy offensive to an area southwest of the Azores. A few U-boats remained in the North Atlantic to carry out a deception plan to mask the U-boat reassignments. They received elaborate orders (also decrypted by GC&CS) to transmit dummy traffic from various locations with varied callsigns to create the impression of the continued presence of the treacherous wolf-packs.
That gives rise to another question. Why on earth would Admiral Doenitz pull his wolf-packs out of the target-rich North Atlantic, especially after enjoying such success? Doenitz was simply overwhelmed with failure! Sounds contradictory and incredible huh? Well, tactics and technology were rapidly changing the nature of the war. He sat helplessly in his Berlin headquarters, watching his fleet of U-boats rapidly become inefficient and the Allied convoy escorts and air patrols become deadly. In the three weeks preceding his decision on May 23, he vectored many wolf-packs to convoys as large as seventy vessels. Some wolf-packs ranged in size from several to an incredible forty U-boats, sometimes providing a 1:1 ratio of U-boats to convoy ships! However, in those three weeks over 500 merchant ships "crossed the pond" successfully with only a handful of losses. Admiral Doenitz, on the other hand, lost about one-third of his submarines on patrol - forty-one U-boats! He was perplexed. German "B-Dienst" radio intelligence analysts had provided him with "silver bullets" which he, and those above him, thought would win the war. In September 1941, B-Dienst cryptanalysts broke the Royal Navy's Cypher No. 2. This was the system used generally throughout the Royal Navy. GC&CS figured out that the Germans had broken No. 2 and replaced it with No. 4 in January 1942, cutting off that source of information for the rest of the war. On/about December 1941, B-Dienst analysts broke Naval Cypher No. 3, which was used for North Atlantic convoy operations control and was jointly used by the U.S. and Canada for that purpose. It was a year before the British learned Cypher No. 3 was compromised and replaced it. Doenitz thus had a year of extensive operational information about Allied convoys which he used to position his U-boats most effectively for intercept and attack. However, naval warfare involves a complexity of factors. Admiral Doenitz was beaten mainly by effective Allied signal intelligence (SIGINT), medium and high frequency direction finding (MFDF, HFDF), radar technology improvements, proliferation of Allied DF and radar systems, and maritime air patrol. I mentioned SIGINT first for two reasons: 1) I suffer from cryptologists' bias, and 2) there is an interesting series of cat-and-mouse events between GC&CS (obviously more than just a school) and B-Dienst/Doenitz which inspired this article. I assimilated some gems of the war pertinent to SIGINT operations against U-boats which you might enjoy. Let me take you back from 1943 to 1941.
As it turns out, GC&CS had its own source of silver bullets; during May 1941 they successfully broke the Enigma system which had been specially modified for the submarine force. Wait a minute, you say! If this is true, why did over a hundred ships get deep-sixed by U-boats? The answer is not as straight forward as you might think!
An Enigma machine, product name Schlussel-M, came into the hands of the British through some sympathetic Polish engineers before Germany invaded Poland. GC&CS at Bletchley Park, with the help of those engineers, developed an electromechanical monster to function as an analytical device to provide a quasi-brute- force solution to Enigma enciphered messages. Success came quickly for most German traffic, but the German submarine system, employing modified logic, resisted attack. GC&CS assigned the codename DOLPHIN (not very original) to the submarine version of Enigma.
Some interesting "acquisitions" led to success against DOLPHIN. On May 7, 1941 British Navy Commandos captured a German weather ship. Codes, cipher equipment and related materials from the ship were sent to Bletchley Park. On May 9, 1941 a bonanza of Enigma machines, operation manuals and setup data was captured from the crippled submarine, U-110. Bletchley Park analysts opened their May-Day presents from U-110 and found 'regular channel' and 'officer-only' setups valid through the end of June 1941. Having actual DOLPHIN equipment provided GC&CS the information they required. Overnight, BdU HQ and U-boat traffic decrypts began flowing through ULTRA channels.
British intercept operations were adequately staffed for interception of HF signals from the U-boats, as well as from their shorebased facilities. However, the capacity to decrypt all the traffic they were collecting, not to mention the backlog, was inadequate. Quality, nature and potential of the ULTRA/DOLPHIN products justified prompt approval for increased resources to expedite exploitation of German submarine communications. By June 1941, the processing time of DOLPHIN encrypted messages to and from U-boats closely paralleled that of the Germans themselves. The "Submarine Tracking Room" in the OIC at the Admiralty had a current and complete map and status board of deployed German submarines. It was probably identical to that displayed in Admiral Doenitz's BdU HQ, which was located close to the submarine base at Lorient, France.
BdU HQ messages were a rich source of intelligence. Admiral Doenitz maintained strict operational control of his submarine captains and he kept them well informed on the tactical situation. This, in turn, kept the Allies well informed on the extent of his knowledge of convoys as well as the locations, status and missions of his submarines.
U-boats were not in the least bit deterred from communicating. They made status reports minimally on a daily basis, but some reported every two to four hours, particularly when they were shadowing a convoy or were functioning as "weather reporting stations." Captains were ordered to include their current positions in every message to BdU HQ. (That certainly came in handy!) It appeared that the U-boats were rarely permitted to attack on initiative or to deviate from orders without permission from headquarters. Wolf-pack commanders communicated extensively with his subs on the medium-frequency (MF) band. MF homing signals were used to help new pack assignees and far-flung pack members to converge. One can plainly see the tactical DF bonanza this represented once the convoy escorts had DF equipment installed.
All that hardly explains how so many Allied ships went to the bottom! In fact, the water is probably more muddy than before, so to speak! Well, even in the '40s, there was no free lunch. Many diverse factors caused the losses. Convoys were attacked without warning before GC&CS began breaking DOLPHIN messages. Once they began breaking submarine ciphers, track modification orders to convoys sometimes resulted only in minimizing their exposure, rather than eliminating it. This was due to the tactical geometry and inherent processing and warning delays involved in each instance. For example, wolf-packs were normally picketed along a line perpendicular to and centered about the expected convoy track. Distances between individual U- boats were normally from visual range to twenty miles. In many cases, track modification orders resulted in flanking movements which were sometimes detected by the 'end' U-boat. Complicating matters was the fact that, as you recall, B-Dienst was breaking the convoy cipher. BdU HQ responded with appropriate tactical orders to its U-boats when convoy track modifications were seen in the Allied convoy communications. BdU HQ was cautious and clever in this regard and concealed their success with Cypher No. 3 for a year!
It is also necessary to touch briefly on some factors causing ship losses which are non-SIGINT in nature. There was not only a learning curve on convoy escort operations, but also a limited number of warships available for that duty. This resulted in escorts taking the convoys to some distance from the port of departure, then breaking off to pick up a convoy moving in the opposite direction. Unfortunately, this caused the convoys to be unescorted during the middle portion of the transit, beyond range of air patrols. Before July 1942, DF equipment was not available on the escorts. Radar on the smaller combatants and maritime patrol aircraft used for escort duty was either not available or was of a less effective vintage. U-boats were equipped with radar warning receivers and used them skillfully. A dizzying delivery rate of U-boats was filling the Atlantic; thirty new subs per month were sliding down the ways in the last half of 1942. Also, in early 1942, Admiral Doenitz began launching specially designed "tanker submarines," nicknamed "milchcows." Through these submarine supply depots, the German's doubled the at-sea periods of U-boats. Milchcows provided fuel, food, spare parts, torpedoes, medical needs, even crew rotations. In 1943, the Royal Navy began equipping their ships and aircraft with newer radars operating on frequencies outside the bandwidth of the German warning receivers. They then began surprising U-boats on the surface.
Back to cryptology. Here are some interesting anecdotes - in timeline relationship - pertinent to SIGINT exploitation of the U-boat force. You'll recognize the cat-and-mouse game we know so well.
In May 1941 the British began breaking DOLPHIN; the backlog for processing the messages ran about seven days. This resulted in the unescorted convoy, designated HX-126, to be attacked by a wolf-pack. The attack came six days after BdU HQ transmitted information about HX-126 to the attacking wolf-pack. That BdU message was decrypted the day following the attack. Nine ships were sunk in the middle Atlantic portion of the trip between Halifax, Nova Scotia and Great Britain. This incident helped bring resources to bear on DOLPHIN processing and illustrated the need for significant changes in escort policy and procedures.
BdU HQ noted the sudden commencement of convoy rerouting in the decrypts of Navy Cypher No. 3, often resulting in avoidance or reduction in efficiency of their wolf-packs.
Germans also had recurring instances of processing delays and were unable to react to all the convoy reroutings.
British convoy controllers had been careful not to infer reasons for changing tracks of convoys. Just the same, when the Germans did become suspicious, it is surprising that they concluded Enigma was unbreakable. They attributed their problems to internal security breaches and began broadly investigating procedures, shore staff and shipboard personnel.
On June 16, 1941 Admiral Doenitz made a procedural change in the method of defining geographic coordinates in messages. Before the change, they were encoded by a classified grid system before encryption. Unknown to BdU HQ, that grid system had been recovered from the U-110 and was used by GC&CS to derive the coordinates from the DOLPHIN message decrypts. BdU HQ replaced the grid with a system in which arbitrary points on the North Atlantic chart were defined in presailing orders; each point was assigned a man's first name. Positions were indicated in messages as bearings and distances from these 'men.' This resulted in a one-month degradation in ULTRA products.
OIC estimated in June 1941 that there were 198 operational U-boats, including those under sea trials.
In July 1941 GC&CS solved the coordinate reference scheme and resumed reporting the positions of U-boats in their ULTRA products. Convoy controllers were able to resume modifying the tracks of convoys. BdU HQ noted this resumption and implemented additional security measures. They stopped referring to their U- boats by the arbitrary U-numbers and began using only the captain's names, which achieved little. Other procedural changes of little impact were also made.
On September 11, 1941 BdU HQ ceased using the system of arbitrary point references. They returned to a system of encoding grid coordinates of positions before giving the messages to the communications operators for encryption. They apparently rushed copies of the new coordinate codes to the milchcows. Each U-boat captain on patrol could then receive a copy when he came alongside the milchcow for resupply. Again, the ULTRA product was degraded for a period. BdU HQ analysts saw the immediate results of their new system; a dramatic reduction in reroutings was reflected in the Allied convoy message decrypts. But, GC&CS broke the German's new coding scheme within the month. Can you picture the scene at BdU HQ when the British convoy messages again, so quickly, resumed indications of wolf-pack location knowledge? Paranoia set in; another round of evaluating the possibility of an Enigma compromise produced a clean bill of health. They began work on a more complex, highly classified geographic coordinate coding system. SS agents desperately sought the internal spy who had access to the codes. It appears that they did not trust delivery of the new codes to the captains at sea by a third party. Most likely, each captain was given a copy of the new codebook when he picked up his orders before departure from port.
By the end of November 1941, most of the U-boats were using the new system for encoding coordinates. It was a dilly! GC&CS was unable to break it!
No doubt, there must have been someone in the German command or intelligence structure who suspected Enigma might have been compromised. But, can you imagine anyone advising Hitler that such was the case? My guess is that someone merely deemed it prudent to improve the system as a preventive measure! GC&CS began seeing references to "TRITON," the German codename for a new cipher system. Soon, they learned TRITON was being distributed throughout the German Navy, including the submariners. British intelligence learned that TRITON was a four- wheel Enigma variant that was replacing all three-wheel Enigma systems currently in use. GC&CS dubbed the new system SHARK; they waited for the dark day it would become effective and slam the door on incisive information.
One bright day in December 1941, - there are few bright days in London during December - a U-boat transmitted a message which was encrypted with the SHARK system before that system had been authorized for use. Later realizing this error, the U-boat reencrypted the same message with DOLPHIN and retransmitted it. This single procedural security faux pas saved untold effort in solving the SHARK four-wheel logic. Having the two encrypted versions of identical text was all that was needed to reverse engineer the SHARK system logic.
On February 1, 1942 SHARK was implemented by the German submarine forces and was immediately read with impunity by GC&CS. Early SHARK decrypts revealed the B-Dienst success against Cypher No. 3; it was promptly replaced, much to the chagrin of B-Dienst cryptanalysts, who never broke that system. Although it is not clear, it appears that the internal coding of position coordinates ceased when SHARK became effective, or shortly thereafter. Thus, comprehensive intelligence on U-boat operations was restored to the British. And owing to the replacement of Royal Navy Cypher No. 3 with a more robust system, Admiral Doenitz was denied knowledge of both the Allied convoy operations and success with SHARK.
In November 1942, OIC estimated the number of operational U-boats at 200, with another 170 in trials and training; German losses in action at that point were 135 U-boats. Allied convoy losses were also reported - 1,160 ships. Germany's alarming rate of submarine construction and high Allied losses tugged hard on Churchill's trouser leg, begging for action. He decided to divert precious resources from the ground and air campaigns in order to provide sufficient long-range maritime patrol aircraft for convoy escort and submarine detection and attack, including the latest equipping them with the latest radar technology. This decision was momentous and was the crowning action [no pun intended] in the war against the German submarine forces.
Admiral Doenitz was selected by Adolph Hitler for promotion to Grand Admiral and to relieve Grand Admiral Raeder as Commander in Chief of the Navy. This became effective on January 30, 1943. Doenitz retained command of the U-boats and moved his submarine force command headquarters from Kernevel, near Lorient, to Berlin.
On March 10, 1943, the U-boats began using a new coordinate codebook. Doenitz was again suspicious; too many U- boats were being preemptively attacked and too many convoys were escaping. It took GC&CS only nine days to break that new code!
April 4, 1943 appears to be the first indication that BdU HQ had begun to consider Communications Security. Returning U- boat captains had long complained of an apparent relationship between their radio transmissions and attacks by planes, ships or both. By this late date, BdU HQ agreed with them. Doenitz admonished a wolf-pack commander for meager convoy sinkings, despite many opportunities, and the loss of two U-boats in the pack. Further, Doenitz blamed unnecessary radio transmissions for the commander's misfortunes. Too little - too late, as they say!
Well, in terms of the timeline, we've come full circle to the celebrations in London on May 23, 1943. I wish I had been an analyst at Bletchley Park, instead of learning how to walk and talk in Atlantic City!
Signal Intelligence was not the singlehanded slayer of Unterseeboote dragons, but it was one helluva silver bullet. It will always be that way!