May 31, 2013:
The CIA recently had another embarrassing reminder that it remains a spy agency without many competent spies. This incident occurred in Russia where a CIA agent was expelled from the country on May 14th, after getting caught while trying to carry out an embarrassingly amateur operation. The man (operating as a junior State Department employee at the U.S. embassy) had been arrested earlier while trying to recruit a senior Russian security official. That offer failed and the CIA man did not detect a trap. Russia later revealed that this also happened back in January but was kept quiet so as to maintain good relations with the U.S. But now the Russians saw an opportunity to use these sloppy CIA operations for domestic propaganda, to remind Russians that in one area at least they are better than the Americans. To add insult to injury, the Russians also showed their displeasure by revealing the name of the senior CIA official (the “resident” in spy-speak) in Russia. This fellow will also have to leave now, which is what the Russians apparently want. They hold the CIA resident responsible for this sloppy and embarrassing use of spy craft. By mutual consent, Russia and the U.S. usually keep the names of their own and the other nation’s residents secret. That rule is only breached when you want to send an important message to the other side.
The real embarrassment is that, since the 1970s, U.S. spies have become rare and competent ones even more scarce. Normally this does not bother the CIA or the U.S. government. But it is a problem when there is an emergency. So, since September 11, 2001, the U.S. has been forced to rely more on contractors, but even skilled freelancers are reluctant to take on the Russians, who have dominated the traditional spy business for most of the last century.
The decline of American spy craft is an aftereffect of the Church Committee. This was an investigative operation sponsored by Congress in the late 1970s that sought to reform and punish the CIA. The reforms were mainly about eliminating CIA spying inside the United States, or doing stuff for the president that Congress did not approve of. There was also a desire to avoid any CIA connection with foreign unpleasantness (like using unsavory people as spies or informants, paying foreign politicians for information, or using contractors to run informant networks). This led to a growing list of restrictions on what the CIA could do overseas and at home. Congress was out to make sure no future president (the CIA works for the president) could use the CIA, as had been done during the Vietnam War and before.
The CIA interpreted all this as "no more James Bond stuff." After the 1970s, the CIA relied more on spy satellites and other electronic monitoring for their reports on what was going on in the world. The Church Committee insured that the CIA became a much less interesting place to work for practitioners of traditional (on the ground, up close and personal) espionage. A lot of the most capable people got out over the next two decades. Recruiting became difficult. But after September 11, 2001, the CIA was tossed a huge pile of money and told to staff up and get going and save us all from the Islamic terrorists. The Church Committee restrictions were largely, if not completely, ignored. But long lists of things-you-couldn't-do were still on the books. After a decade of doing whatever it took, the rules are being enforced again.
One of the more successful operations hurt by this temporary retro thinking was the use of contractors to run Pakistan intel operation. Things were not done by the book, but results were demanded, especially efforts to find Osama bin Laden. Everyone looked the other way while the deed was done. Now Congress is again calling for investigations and “rogue operators” to punish. This sort of thing makes it very difficult to recruit and keep competent spies, even as contractors.
But it's not just paper bullets intelligence operatives have to worry about these days. The post-9/11 world dramatically altered the way that national intelligence services do business. For one, the craft of espionage and military intelligence has become inherently more dangerous for case officers and agents in an age of terrorism and insurgency than it was during the Cold War.
This is a complete turnaround from the way business was done during the Cold War in the '60s, '70s, and '80s. Many case officers on all sides, whether CIA, SIS, or KGB, served out their entire 20 or 30 year careers as professional spies without ever having touched a firearm after their initial tradecraft courses. After all, getting into gun battles was not their job. Collecting information was. Furthermore, the case officers themselves, often operating under official diplomatic cover, didn't really have anything to fear if they were caught or their covers blown, except a ruined career and expulsion from whatever country they operated in. The ones in real danger were always the informants, or "assets", that the case officers recruited, who were liable to face execution if they were found out. Simply put, spying really wasn't that dangerous for the case officers.
After the War on Terrorism began, the Cold War rules began to rapidly disappear. For one thing, Afghanistan, Pakistan, and Iraq, along with most places that CIA officers operate today, are actual war zones with nothing "cold" about them. During traditional peacetime case officers don't really have to worry about their own safety, just that of their informants. Once you get involved with terrorists or an actual shooting war starts, all of that changes, and intelligence officers (whether CIA, SIS, or Army Intelligence) become major high-value targets for terrorist and insurgents. Since 2001, over a dozen (the exact number is classified) CIA officers have been killed in the line of duty. In short, the espionage business has gotten far more dangerous in a very short period of time.
This has necessitated a number of dramatic changes in the way the Americans, British, and other professional intelligence services do business where they are needed most (in war zones). For one, the spooks are getting strapped. Case officers working in places like Pakistan, Afghanistan, and even Egypt routinely carry handguns everywhere they go to defend themselves should the need arise. During the Cold War this was unnecessary and generally considered a stupid liability since being caught with a weapon would probably get you booted out of the country you operated in. Not anymore.
Besides carrying guns, agencies and case officers are paying extra attention to things like counter-surveillance, disguises, and evasive driving. Carrying a sidearm is necessary for a case officer working in a city like Karachi or Kabul, the truth remains that getting into a gunfight is still the last resort and should be avoided at all costs. Case officers know that the most effective way to avoided being a terrorist target is to avoid following the same routines every day, varying routes to and from work/meetings, never sleeping in the same safe house for too long, and generally making one's life as varied and unpredictable as possible. Experienced spies know that if you can't be found, you can't be a target. The best game plan is to be as invisible as possible. Using contractors to run your informant networks is the best cover of all, unless Congress is looking for someone to prosecute.
Russia is different, as the Russians always had the best spies (because of superior recruiting, training, and management). A lot of those spies were cut loose after the Soviet Union dissolved in 1991, and some of them offered to talk (if the price was right). What these guys revealed was chilling for Western intel agencies, a decades long tale of successful old-school espionage operations. The KGB was so good that most of these ops were not even suspected. But the new information enabled the U.S. to roll up a number of well-placed Russian agents and moles and provide evidence supporting calls for a return to traditional espionage. Congress was still hostile to that and the September 11, 2001 attacks were one result. The current comedy of errors in Russia is another. There will be more.