Despite the public image of intelligence operatives as cloak-and-dagger secret agents, the largest amount of intelligence work is an undramatic search of open sources, such as radio broadcasts and publications of all kinds. Much of this work, which also includes sifting reports from diplomats, businessmen, accredited military attachés, and other observers, is performed by university-trained research analysts in quiet offices.

Covert sources of intelligence fall into three major categories: imagery intelligence, which includes aerial and space reconnaissance; signals intelligence, which includes electronic eavesdropping and code breaking; and human intelligence, which involves the secret agent working at the classic spy trade. Broadly speaking, the relative value of these sources is reflected in the order in which they are listed above. A photograph, for example, constitutes hard (i.e., reliable) intelligence, whereas the report of a secret agent may be speculative and difficult to prove.

Methods of intelligence gathering

Good intelligence management begins with the proper determination of what needs to be known. Unless precise requirements are set, data will be collected unsystematically and the decision maker ultimately left without pertinent information on which to act. Collected data must be evaluated and transformed into a usable form (and sometimes stored for future use). Evaluation is essential, because many of the wide variety of sources are of doubtful reliability. A standardized system is used to rate the reliability of sources and the likely accuracy of the information they provide (e.g., information may be classified as confirmed, probably true, possibly true, or unlikely to be true).

Information obtained from open sources probably constitutes more than four-fifths of the input to most intelligence systems, though this proportion varies with the number of state secrets a country may have. Clandestine collection methods from covert sources provide the basis for much of the drama and romance attributed to intelligence work in fiction. Although the classic espionage agent will never be completely obsolete, some observers have suggested that the role largely has been taken over by machines, including orbiting reconnaissance satellites, long-range cameras, and a variety of sensing, detecting, and acoustical instruments. With this kind of technology, it is now possible to see in darkness, to hear from great distances, and to take detailed photographs from altitudes of hundreds of miles. Nevertheless, only spies can produce information about the attitudes and intentions of foreign leaders or international terrorists and other criminals. Indeed, a lack of adequate human intelligence was cited by some critics as a factor in the failure of U.S. intelligence and law-enforcement agencies to prevent the devastating terrorist attacks on New York City and Washington, D.C., on September 11, 2001 (see September 11 attacks).

Techniques of aerial reconnaissance have advanced dramatically since the 1940s, when the United States drifted balloons carrying special cameras across Soviet territory to photograph military and industrial installations. Today aerial reconnaissance is conducted by satellites, aircraft, and unmanned drones, which can orbit a battlefield for 24 hours. The U.S. U-2 aircraft and its higher-flying successors are capable of taking photographs that experts can read with great accuracy. Imaging satellites, which can produce accurate information about the number and location of a country’s nuclear missiles and other weapons, made possible the arms control treaties between the United States and the Soviet Union. High-altitude photographs are also used to diagnose environmental catastrophes, to locate terrorist training camps, and even to detect human rights abuses.

Intelligence organizations often employ electronic scavengers (from ships, planes, listening posts in embassies and military installations, and orbiting satellites) to collect information about a country’s radio communications and its naval equipment and operations. An individual submarine, for example, can be identified by the telltale and unique noises it makes (its “signature”). During the Cold War the United States collected sensitive signals intelligence by tapping communications lines in Soviet territorial waters. It also used satellites and special planes for conducting missions close to the borders of potential adversaries. Similarly, the Soviet Union (and later Russia) collected signals intelligence from listening stations in diplomatic and consular missions and from large “fishing trawlers” that shadowed the U.S. fleet.

The use of computers to analyze data on complex phenomena such as industrial production, missile launches, and rates of economic growth has created vast amounts of information that threaten intelligence systems with inundation, making the filtering of useless information a key task. Since World War II great efforts have been made to develop efficient means of cataloging, storing, and retrieving the gigantic volume of data being amassed. Although some observers believe that data collection, especially in the Internet age, has been overemphasized at the expense of analysis, computer technology and the application of artificial intelligence, which allow computer programs to organize mammoth amounts of raw material for analysts, promise to make the tidal wave of information manageable. For example, such techniques can be used at border crossings to quickly compare the image of a suspected terrorist with thousands of pictures of known criminals.

Premodern intelligence

The ancient soothsayers, who claimed to be able to communicate with the gods and were often said to have the power of predicting the future, were perhaps the earliest counterparts of the modern intelligence agency. As in modern times, their reports were often ambiguous and frequently ignored by decision makers.

The Bible says that God advised Moses (Numbers 13) to send agents to “spy out the land of Canaan.” After 40 days, 12 agents returned to report that the people in the land flowing with milk and honey were more powerful than the Israelites—and on the basis of this intelligence the Israelites rebelled and were punished by God.

The ancient Chinese author Sun Tzu (fl. 4th century bc), whose Ping-fa (The Art of War) is said to be widely read by contemporary Chinese strategists, identified five kinds of secret agent; their modern counterparts are the agent in place (who has access to enemy secrets), the double agent (who is recruited from an enemy’s intelligence and security service), the deception agent (who provides disinformation to confuse the enemy), the expendable agent (whose loss can enable other more important agents to operate), and the penetration agent (who has access to an enemy’s senior leadership). Sun Tzu stressed the importance of good intelligence organization, and he also wrote of counterintelligence and psychological warfare.

In Europe during the Middle Ages, intelligence was systematically used but crudely organized. Although it was usually impossible to conceal the massing of troops or ships, communication was slow, making the achievement of strategic surprise a difficult matter of balancing the time required to assemble large forces against the time needed by enemy agents to discover and report them.

In the 15th century the Italian city-states began to establish permanent embassies in foreign capitals. The Venetians used such outposts as intelligence sources and even developed codes and ciphers by which information could be secretly communicated. By the 16th century other European governments had followed suit.

Intelligence and the rise of nationalism

The rise of nationalism was accompanied by the growth of standing armies and professional diplomats as well as by the establishment of organizations and procedures for procuring foreign intelligence. Queen Elizabeth I (reigned 1558–1603) of England maintained a notable intelligence organization. Her principal state secretary, Sir Francis Walsingham (c. 1532–90), developed a network of intelligence agents in foreign countries. He recruited graduates of Oxford and Cambridge, developed the craft of espionage, including tools and techniques for making and breaking codes, and engaged in much foreign political intrigue. Later, Armand-Jean du Plessis, cardinal et duc de Richelieu (1585–1642), and Oliver Cromwell (1599–1658)—whose intelligence chief, John Thurloe (1616–68), is often cited as an early master spy—developed notable intelligence systems. The intelligence operations of the Great Powers also included secret channels of communication, the penetration of émigré circles, and the assassination of enemies of the state.

Not until the late 18th century, however, did there arise sharp divisions between organizations devoted to internal security (a counterintelligence function) and those concerned with external foreign intelligence. As populations began to give their allegiance to the state rather than to dynasties or religious leaders, national leaders paid increasing attention to the opinions of foreign publics, resulting in both a new diplomacy and new intelligence needs. Major innovations in organization and doctrine have been credited to the Prussian king Frederick the Great (reigned 1740–86). Frederick, and later Wilhelm Stieber, an aide to the Prussian prime minister and later German chancellor Otto von Bismarck (1815–98), organized the intelligence-gathering functions of the general staff. Under Stieber, a single military intelligence agency—the world’s first large-scale espionage organization—was established to serve as the country’s eyes on the outside world.