At the turn of the 20th century European governments required increasing amounts of strategic intelligence to compete in power politics, to support their foreign empires, and to keep up with advances in military and communications technology. Accordingly, intelligence bureaus spread throughout the European continent, resulting in a corresponding growth in counterintelligence. Nevertheless, when World War I broke out in 1914 the intelligence services of most European countries were inadequate. The war, which none of the combatants intended, is often cited in hindsight as a tragic failure of intelligence. The French intelligence service, which already had been weakened by the Dreyfus affair (see Alfred Dreyfus), was torn by internal intrigue, and other services had been shaken by scandals. One spectacular failure of French intelligence was its gross miscalculation of German military strength in 1914, when it underrated German technical and tactical capabilities. German intelligence also had deteriorated, and by 1914 the German general staff evidently placed little faith in the information its intelligence officers supplied. Nevertheless, the Germans carried out successful intelligence activities in Persia and scored limited successes in the United States. The Russian intelligence service initially enjoyed great success against the Austrians because of the treason of an Austrian general staff officer, but it subsequently performed no better than the services of other countries involved in the war. The British succeeded in breaking German naval codes, and they were able to use the information they obtained to hasten U.S. entry into the war by exposing German efforts to involve Mexico in a war with the United States. However, tactical intelligence provided to British commanders on the Western Front was fraught with optimistic and misleading assessments of Germany’s military capabilities.

Unlike the European countries, the United States had no central intelligence organization. Indeed, at the beginning of the war, army intelligence was only a small section of the general staff, comprising two officers and two clerks. By the end of the war this service had grown to 1,200 officers and civilians. Overall, the American intelligence community at this time was staffed by amateurs and was quite deficient.

The intelligence lessons of World War I, along with advances in technology—especially electronics and aircraft technology—resulted in a proliferation of new intelligence agencies in the 1920s and ’30s, particularly in totalitarian states (Italy, Germany, and the Soviet Union) but also in some democratic European countries. The expansionist policies of the Soviet Union, Italy, Germany, and Japan in the 1930s, and especially the outbreak of World War II in 1939, precipitated the creation and expansion of intelligence services throughout the world. In 1942 the United States, which had virtually no peacetime intelligence services, created its first full-fledged organization for intelligence and secret operations, the Office of Strategic Services (OSS). The war imposed intelligence requirements never before faced by the major warring powers, primarily as a result of rapidly advancing military technology. Air warfare in particular required vast new offensive and defensive intelligence operations. Air force commanders needed information on possible bombing targets, as well as on enemy fighters and antiaircraft artillery. In the first days of World War II, the United States relied on the insurance records of German industries and on aerial reconnaissance to identify bombing targets. The growth of radio broadcasting enabled the development of the new art of psychological warfare, whose effects demanded study by intelligence services.

Yet despite its rapid development, intelligence forecasting remained a precarious trade. Many key events in the war—including the German invasion of the Soviet Union in June 1941, the Pearl Harbor attack against the United States by Japan in December 1941, the Battle of the Bulge in December 1944–January 1945, and the Allied bombing campaigns against Germany (1942–45), in which the Germans showed unexpected resilience—were marked by the failure of decision makers to profit from their elaborate intelligence networks.

However, there was one area of enormous success. Perhaps the most significant intelligence achievement of the war was the Ultra project, in which the British, using a German Enigma encoding machine obtained from the Poles and relying on earlier decryption efforts by the Poles and the French, intercepted and deciphered top-secret German military communications throughout much of the war. In essence, the Ultra project enabled the Allies to read the mind of the German high command. As the war progressed, Hitler’s increasingly centralized control of operations on all fronts made German military operations especially vulnerable. Ultra was particularly important in the defeat of the German U-boat fleet and the German surface navy. When the Allies were caught by surprise, such as in the American defeat at Kasserine Pass, the Allied defeat at Arnhem, and the Battle of the Bulge, the Germans had used land lines for communication or Ultra intercepts had been misused.

During the Cold War intelligence became one of the world’s largest industries, employing hundreds of thousands of professionals. Every major country created enormous new intelligence bureaucracies, usually consisting of interlocking and often competitive secret agencies that vied for new assignments and sometimes withheld information from each other. The United States established the Central Intelligence Agency (CIA) in 1947. Among other well-known intelligence organizations created during this period were the United Kingdom’s MI5 and MI6, the Soviet Union’s KGB (Committee for State Security), France’s SDECE (External Documentation and Counterespionage Service), China’s MSS (Ministry of State Security), and Israel’s Mossad. By the 1970s every regional power and many relatively small states had developed intelligence services. At the same time, the exploits of spies and counterspies became a staple of the entertainment and publishing industries. In books, movies, and television, intelligence agents were portrayed in roles that were sometimes comic but often deadly serious. All these accounts tended to glamorize an occupation that was often painfully tedious and sometimes (in the opinion of some) distasteful and immoral.

In the last two decades of the Cold War the United States relied heavily on imagery and signals intelligence, including satellite photography, to collect information on Soviet weapons of mass destruction. Its emphasis on these sources of intelligence, however, may actually have weakened its ability to combat terrorist organizations, which by their nature are not easily penetrated through technical means.

Since the disintegration of the Soviet Union and the end of the Cold War, nonstate actors (e.g., terrorist organizations, militias, and drug cartels) have developed sophisticated intelligence and counterintelligence capabilities that rival those of some states. The Islamic terrorist organization al-Qaeda, which organized the September 11 attacks against the United States, had an intelligence infrastructure that maintained safe houses in the Middle East, Europe, and North America. Evidence uncovered after the U.S. and British military campaign in Afghanistan indicated that al-Qaeda had purchased sophisticated computer hardware that enabled it to send enciphered communications to terrorist cells and to track U.S. photographic reconnaissance satellites. Today, terrorists and drug traffickers from the jungles of Colombia to the streets of western Europe employ advisers drawn from the intelligence services of the former Soviet Union, East Germany, and Yugoslavia and use criminals of various kinds to bribe or terrorize their opponents and protect their organizations. Accordingly, since the end of the Cold War the targets of intelligence activity have been just as likely nonstate actors as states. Operations against such organizations require smaller and more-flexible intelligence services capable of combining technical intelligence (i.e., imagery and signals intelligence) and human intelligence; operations officers and analysts; and various military, intelligence, and security organizations.