The Secret Sentry [History of the NSA]
Posted: 2012-02-10 03:44am
I've been reading The Secret Sentry by Matthew M. Aid, on the National Security Agency.
It's currently at $2.99 on Amazon's Kindle Store, and I can really highly recommend it just on reading it for an hour or so.
So here goes my quick condensation of the "money shots" of the first 7-10% of the book, to give you an idea of what's in it:
----------------------
On V-J Day, the US Army's Signal Security Agency (SSA), along with the US Navy's Communications Intelligence Organization (OP-20-G), had about 37,000+ personnel around the world cracking codes.
The SSA alone was able to read 350 diplomatic code and cipher systems belonging to sixty countries.
However, demobilization hit the US crypto establishment hard, with about 80% of manpower being lost in the 120~ days following V-J Day.
It was pretty grim.
Then we started to break into Soviet codes.
The effort began on January 1943 when BGEN Carter Clarke issued orders for the Army's Cryptanalysts to start attacking Soviet Ciphers.
By October 1943, we had made a significant breakthrough, when we discovered something really crucial about a lot of Soviet cryptography during the wartime period.
In the chaos following the invasion of Russia in June 1941, the NKVD had to take increasing shortcuts to fill the demand for cryptographic materials. In the Winter of 1941-42, the NKVD printed duplicates of 25,000 pages of one time ciphers, bound them into OTP books and sent them around the world.
Replacing the Ciphers was not easy due to German U-Boat activity which limited the reach of the Soviet Merchant Marine.
This provided us with our early war-time era decrypts of the VENONA program.
But far more significant breakthroughs would occur in the post war era.
On 1 March 1946, the US Army at Arlington hall decrypted it's first message from a Soviet SAUTERNE coding machine, which was used by Soviet Army Far East radio networks. By the end of March, the USN had found a way to determine the daily rotor settings for encipering, resulting in regular decrypts of SAUTERNE messages by 4 April 1946.
Around the same time that SAUTERNE was cracked, the British GCHQ cracked into COLERIDGE, which was used for Soviet Army radioteletype networks in the European half of the Soviet Union.
Due to the BRUSA Agreement, we got the COLERIDGE decrypts, which had reams of administrative trivia, like orders of battle, training activities, and logistical stuff.
In February 1947, a team of British cryptos led by the US Navy in Washington DC cracked LONGFELLOW, which was used by the Soviet Army.
By the spring of 1947, all three systems (SAUTERNE/COLERIDGE/LONGFELLOW) were being exploited heavily, with 341 decrypts a day on average coming out of Arlington Hall.
By early 1949, we had 12,500+ decrypted Soviet Army messages.
That was just the Soviet Army. In early 1947, we broke into several Soviet Navy ciphers; which were kind of easy to listen in onto, because the two Soviet fleets in the Pacific had to use radio to communicate with Moscow, instead of secure landlines.
By February 1949, the USN had produced 21,000+ decrypts of Soviet Naval Message Traffic.
Also in 1947, the US Army cracked one of the operational cipher systems used by Soviet Air Force HQ in Moscow to communicate with it's subordinate commands. In one such example, US Army cryptos in Japan were reading the encrypted traffic of the Ninth Air Army at Ussurijsk/Vozdvizhenka and the Tenth Air Army at Khabarovsk.
All this was used during the Berlin Crisis/Airlift. We knew from our decrypts of Soviet message traffic that the Soviets were just messing with us and not serious about war, and would not seriously impede the Berlin Airlift.
....then came Black Friday.
On 29 October 1948, the entire Soviet Government and Military changed pretty much every cryptographic system in use and switched to secure landlines wherever possible; along with implementing severe radio discipline regulations, and harsh punishments for breaking them.
From that point on, we were unable to break into secure Soviet message traffic until the late 1970s.
What caused this?
William Weisband, a 40 year old Army linguist had told the KGB everything he knew about the Army's efforts at Arlington Hall. In order to not reveal lots of stuff in public about our codebreaking effort, we did not prosecute him for Espionage, and he died in May 1967 of a heart attack at 59.
Due to this, there was a massive shift in US COMINT allocation. By Summer 1949, about 71% of all radio intercept personnel and 60% COMINT processing personnel were working on the Soviet problem.
The Armed Forces Security Agency (AFSA), the precursor to the NSA, cut back on the number of people working on Asian problems from 261 to 112 by the end of 1949.
There was no COMINT coverage at all of North Korea from 1946 to the Korean War. Period.
The AFSA's Korean section was paper only -- the two people assigned were actually assigned to the Chinese section and only worked on North/South Korean codes in their spare time.
There was also no support equipment at all, like Korean dictionaries, typewriters, etc.
However, things got better.
On 29 June 1950, the first Nork signals traffic began arriving at Arlington Hall from Japan; and by 3 July 1950, we began to translate it.
It turned out that the North Koreans were transmitting highly detailed situation reports, battle plans and troop movements in the clear.
By 14 July 1950, we made our first break into North Korean ciphers, and then from there on began to break into more and more ciphers.
By the end of July 1950, we were translating and solving 1/3 of all North Korean intercepted traffic, limited only by a shortage of Korean linguists.
This enabled 8th US Army at Pusan to hold on and slaughter repeated North Korean assaults as we knew when they were coming and where.
It also enabled MacArthur to land at Inchon, as he had the North Korean OOB in hand with detailed information on all the North Korean units around Pusan and showed that there were no large North Korean units around Inchon.
Mac also knew from SIGINT that the North Koreans were worried about a US Amphibious assault, and had started pulling back units to protect the East/West Coast of South Korea, but had picked Mokpo or Kunsan as the likely invasion point.
We reached 90% Decryption rates of North Korean traffic in December 1950, which enabled us to basically keep from being totally slaughtered by the NKPA/PLA following Chinese intervention.
Unfortunately, in the first week of July 1951, the Norks changed virtually all of their codes, and began to do smart communications security practics, like using encrypted radio callsigns, and not transmitting unencrypted plaintext radio traffic, along with switching to landlines wherever possible.
Current NSA consensus is that this was caused by Soviet advisors coming in and going "lol, whut?" at North Korea's poor communications security, and then telling them what NOT to do.
----
There's a lot more in the book, particularly on how we got caught flatfooted by the CHINESE INTERVENTION in Korea; and a lot on Vietnam war activities.
For example, did you know that USS Maddox had a portable NSA cryptologic container on her middeck during the famous Tonkin gulf incident, and that she was receiving near-real-time decrypted North Vietnamese radio traffic relating to operations against her?
The book is worth getting, particularly at that low low price of $2.99 for Kindle.
It's currently at $2.99 on Amazon's Kindle Store, and I can really highly recommend it just on reading it for an hour or so.
So here goes my quick condensation of the "money shots" of the first 7-10% of the book, to give you an idea of what's in it:
----------------------
On V-J Day, the US Army's Signal Security Agency (SSA), along with the US Navy's Communications Intelligence Organization (OP-20-G), had about 37,000+ personnel around the world cracking codes.
The SSA alone was able to read 350 diplomatic code and cipher systems belonging to sixty countries.
However, demobilization hit the US crypto establishment hard, with about 80% of manpower being lost in the 120~ days following V-J Day.
It was pretty grim.
Then we started to break into Soviet codes.
The effort began on January 1943 when BGEN Carter Clarke issued orders for the Army's Cryptanalysts to start attacking Soviet Ciphers.
By October 1943, we had made a significant breakthrough, when we discovered something really crucial about a lot of Soviet cryptography during the wartime period.
In the chaos following the invasion of Russia in June 1941, the NKVD had to take increasing shortcuts to fill the demand for cryptographic materials. In the Winter of 1941-42, the NKVD printed duplicates of 25,000 pages of one time ciphers, bound them into OTP books and sent them around the world.
Replacing the Ciphers was not easy due to German U-Boat activity which limited the reach of the Soviet Merchant Marine.
This provided us with our early war-time era decrypts of the VENONA program.
But far more significant breakthroughs would occur in the post war era.
On 1 March 1946, the US Army at Arlington hall decrypted it's first message from a Soviet SAUTERNE coding machine, which was used by Soviet Army Far East radio networks. By the end of March, the USN had found a way to determine the daily rotor settings for encipering, resulting in regular decrypts of SAUTERNE messages by 4 April 1946.
Around the same time that SAUTERNE was cracked, the British GCHQ cracked into COLERIDGE, which was used for Soviet Army radioteletype networks in the European half of the Soviet Union.
Due to the BRUSA Agreement, we got the COLERIDGE decrypts, which had reams of administrative trivia, like orders of battle, training activities, and logistical stuff.
In February 1947, a team of British cryptos led by the US Navy in Washington DC cracked LONGFELLOW, which was used by the Soviet Army.
By the spring of 1947, all three systems (SAUTERNE/COLERIDGE/LONGFELLOW) were being exploited heavily, with 341 decrypts a day on average coming out of Arlington Hall.
By early 1949, we had 12,500+ decrypted Soviet Army messages.
That was just the Soviet Army. In early 1947, we broke into several Soviet Navy ciphers; which were kind of easy to listen in onto, because the two Soviet fleets in the Pacific had to use radio to communicate with Moscow, instead of secure landlines.
By February 1949, the USN had produced 21,000+ decrypts of Soviet Naval Message Traffic.
Also in 1947, the US Army cracked one of the operational cipher systems used by Soviet Air Force HQ in Moscow to communicate with it's subordinate commands. In one such example, US Army cryptos in Japan were reading the encrypted traffic of the Ninth Air Army at Ussurijsk/Vozdvizhenka and the Tenth Air Army at Khabarovsk.
All this was used during the Berlin Crisis/Airlift. We knew from our decrypts of Soviet message traffic that the Soviets were just messing with us and not serious about war, and would not seriously impede the Berlin Airlift.
....then came Black Friday.
On 29 October 1948, the entire Soviet Government and Military changed pretty much every cryptographic system in use and switched to secure landlines wherever possible; along with implementing severe radio discipline regulations, and harsh punishments for breaking them.
From that point on, we were unable to break into secure Soviet message traffic until the late 1970s.
What caused this?
William Weisband, a 40 year old Army linguist had told the KGB everything he knew about the Army's efforts at Arlington Hall. In order to not reveal lots of stuff in public about our codebreaking effort, we did not prosecute him for Espionage, and he died in May 1967 of a heart attack at 59.
Due to this, there was a massive shift in US COMINT allocation. By Summer 1949, about 71% of all radio intercept personnel and 60% COMINT processing personnel were working on the Soviet problem.
The Armed Forces Security Agency (AFSA), the precursor to the NSA, cut back on the number of people working on Asian problems from 261 to 112 by the end of 1949.
There was no COMINT coverage at all of North Korea from 1946 to the Korean War. Period.
The AFSA's Korean section was paper only -- the two people assigned were actually assigned to the Chinese section and only worked on North/South Korean codes in their spare time.
There was also no support equipment at all, like Korean dictionaries, typewriters, etc.
However, things got better.
On 29 June 1950, the first Nork signals traffic began arriving at Arlington Hall from Japan; and by 3 July 1950, we began to translate it.
It turned out that the North Koreans were transmitting highly detailed situation reports, battle plans and troop movements in the clear.
By 14 July 1950, we made our first break into North Korean ciphers, and then from there on began to break into more and more ciphers.
By the end of July 1950, we were translating and solving 1/3 of all North Korean intercepted traffic, limited only by a shortage of Korean linguists.
This enabled 8th US Army at Pusan to hold on and slaughter repeated North Korean assaults as we knew when they were coming and where.
It also enabled MacArthur to land at Inchon, as he had the North Korean OOB in hand with detailed information on all the North Korean units around Pusan and showed that there were no large North Korean units around Inchon.
Mac also knew from SIGINT that the North Koreans were worried about a US Amphibious assault, and had started pulling back units to protect the East/West Coast of South Korea, but had picked Mokpo or Kunsan as the likely invasion point.
We reached 90% Decryption rates of North Korean traffic in December 1950, which enabled us to basically keep from being totally slaughtered by the NKPA/PLA following Chinese intervention.
Unfortunately, in the first week of July 1951, the Norks changed virtually all of their codes, and began to do smart communications security practics, like using encrypted radio callsigns, and not transmitting unencrypted plaintext radio traffic, along with switching to landlines wherever possible.
Current NSA consensus is that this was caused by Soviet advisors coming in and going "lol, whut?" at North Korea's poor communications security, and then telling them what NOT to do.
----
There's a lot more in the book, particularly on how we got caught flatfooted by the CHINESE INTERVENTION in Korea; and a lot on Vietnam war activities.
For example, did you know that USS Maddox had a portable NSA cryptologic container on her middeck during the famous Tonkin gulf incident, and that she was receiving near-real-time decrypted North Vietnamese radio traffic relating to operations against her?
The book is worth getting, particularly at that low low price of $2.99 for Kindle.
