Week 8 Readings and Video
What I Learned in 40 Years of Doing Intelligence Analysis for US Foreign Policy Makers; by Martin Peterson
In the First Person
Martin Petersen
“Every intelligence product must be rooted in a strong understanding of the audience it is written for.”
An advantage of getting older is increased perspective. I have been doing, thinking and writing about intelligence and intelligence analysis for almost 40 years now. The business we are in has changed a great deal in that time, but more in its form than in its fundamentals.
I want to focus on three broad topics: understanding the customer, the importance of a service mentality, and the six things I learned in doing and studying intelligence analysis during my career in the DI. While these experiences are drawn from work in the CIA, I believe the principles apply across the Intelligence Community (IC).
Understanding the Consumer: Five Fundamental Truths
I believe every intelligence product must be rooted in a strong understanding of the audience it is written for, and I believe there are five fundamental truths about the analytical products and their consumers.
Truth number one: the product is “optional equipment” for many key consumers.
The most precious commodity in Washington is not information—there is an overabundance of information, data, opinion, and secrets—but time. The “future” in Washington is four years at its longest point and every day it is one day shorter. It is not surprising then that consumers of our services are in a hurry and that they are very busy people; the president’s day is actually planned in five minute increments. These people have many, many sources of information, and many of the people we serve believe they are better plugged into the world than we are. And in many cases, they are.
Our customers in the policymaking realm often do not understand our mission, our values, or our standards. They tend to be skeptical of intelligence, especially if they are new to the policymaking world. They formed their views about who we are, what we do, and how we do it from the same sources other Americans do: popular media, the press, and congressional reports—not always the most accurate or sophisticated of sources and generally not the most flattering. Our consumers have strong world views and clear policy agendas, and they often assume we have a policy agenda, too.
It is not surprising then that policymakers do not always see how we can help them: “After all, I, the policymaker, am smart and have excellent sources of information (including all the ones you have), and I am very busy, so why should I spend some of my most precious commodity on you?” The reality for intelligence officers is that we must woo them, sell them on the need for our services, and demonstrate the value of our material daily through its timeliness and its sophistication. If you are an intelligence officer, the title will often get you in the door, especially the first time, but it will not keep you there. Newcomers to the IC may not realize that the CIA presence in the Oval Office during the George W. Bush administration was the exception, not the rule.
If the IC is going to be part of the regular routine in the White House, not only must we have something to say that people there cannot get somewhere else—which has to be more than having secrets—but we have to be mindful of how we deliver it. We are not only optional equipment; we are also guests at their dinner party. If we spill the wine, insult the host, and overstay our welcome, we will not be invited back.
Speaking truth to power first requires access to power. My personal experience is that our consumers will take frequent bad news and unhappy assessments as long as they are well-reasoned, supported by data and argument, and presented without rancor, value judgments, or arrogance.
Truth number two: the written product is forever.
A colleague who spent half his career in the DI and half in the National Clandestine Service (NCS) once said only half jokingly, “You know what the DI’s problem is? You guys write things down. In the NCS we believe in the oral tradition.” He was right in the sense that the written word is forever. Once it is printed, there is no taking it back or modifying it.
Briefings and background notes are important parts of doing the mission, but they leave no permanent record. One can fight over what was said in a briefing, but the written word is in black and white. It is the WorldIntelligenceReview (WIRe) article, the serial flyer, the intelligence assessment, and the national intelligence estimate (NIE) that end up in the archives, and it is the paper product that gets held up at a congressional hearing or eviscerated on an editorial page.
And when I say forever, I mean forever. Relatively few people have read the now infamous NIE done in 2002 on Iraq’s weapons of mass destruction (WMD), but everyone knows what it says. And everyone years and years from now will know what it said, because it is viewed—rightly or wrongly—as fatally flawed and responsible for the second Iraq war. It will never go away, and it joins the pantheon of other real and imagined CIA failures. Every time we publish, we go “on the record” and the record is there forever, for the second guessers, the hindsight experts, and anyone with an agenda. Thus, our judgments need to be as precise as we can make them, supported by evidence and argument, and accurately reflect our level of confidence every time.
Truth number three: the public does not segregate success and failure.
Critics of intelligence, our customers, and the general public do not say that the products of a certain office in CIA or DIA are really great, but that the products of another office in that agency are awful. Nor do they say that one type of analysis, say political, can be trusted, but that our work on something else, say S&T is unreliable. Nor will they say that although they were wrong last time, we can trust them this time.
No, customers remember, and they question. Sometimes they question fairly, but often they do not, especially those customers who find what is being said to be inconvenient or “unhelpful” in advancing a policy position they favor. From the CIA alone, I can produce a list of what I call “everybody knows”: everybody “knows” the CIA failed to predict the fall of the Shah of Iran in 1979 or the collapse of the Soviet Union in 1991 or the Indian nuclear test in 1995 or this or that. The facts are often far more complex, but they have entered the popular mythology. And the consumers of intelligence say out loud “Why should I trust you on this issue when you were wrong on that one?” Weak performance in one DI area immediately calls into question all work in the CIA.
President Kennedy famously said of the CIA that its successes will be secret and its failures will be trumpeted. To which I add my own corollary: in the intelligence business success is transitory, and failure is permanent.
Truth four (closely related to truth three): our individual and collective credibility —and thus our ability to do the mission—rides on every piece of finished intelligence that goes out the door.
Sad to say, no one cares what I think about a particular issue—and no one cares what you personally think either. They do care tremendously about what the CIA or DIA—or name the IC organization—thinks. The finished intelligence products that go out the door are not personal products but corporate ones.
IC products have brand names, and they are important and powerful ones. They can open doors, but they will not keep any analyst inside circles of power if that brand name is devalued by shoddy work. Our customers read our products for many reasons: to learn, to make better decisions, to know what the President’s Daily Briefing tells the president, to look for ammunition in a policy fight, or to discredit what the IC says.
Every poorly-reasoned piece of finished intelligence tarnishes a brand name a bit and over time can produce cracks in the trust they place in us to live up to our tradecraft. When that happens there is nothing one can say and eventually the broader trust is lost. Ask BP and Toyota. One bad oil well and a few sticky accelerators undid years of excellent performance, and shouting “but our record is still better than that of [someone else]” makes no difference. We do not drill oil wells or build cars. We do the mission—the mission of protecting the United States. Our ability to “raise the level of the debate” or to “help policymakers make the best decisions possible” or to “speak truth to power”—however one defines the mission—rests on one thing and one thing only: our reputations for analytic rigor, objectivity, and total integrity. Lose that and we lose everything.
Truth five: our customers are smarter and more sophisticated than we give them credit for; they have their own independent sources of information and analysis with which we are competing.
And these customers are continually changing. We have to establish our credibility and usefulness individual by individual, administration by administration. There is no down time when it comes to quality.
These five truths demand tradecraft excellence, they demand exacting standards, and they demand the pursuit of perfection. They demand that we learn from our past, and they demand that we ask the best of ourselves every time. To do the mission; to serve the policymaker; to protect the nation—requires nothing less.
The Importance of a Service Mentality
Excellence requires more than a standard of quality. I believe it also demands a specific approach to the craft of intelligence analysis: it requires a service mentality. A service mentality is the opposite of a product mentality, which often seems to drive the work of intelligence analysis, and the difference is easiest to explain by comparing the two. In a product mentality, the focus is on the producer, who thinks of a product as his or hers. It is also about packaging that product and disseminating it widely. Success is measured in numbers—how many units were produced or how many received each unit. It is about filling a book or producing a product to demonstrate that an analyst is ready for the next big step in a career.
In a service mentality, the focus is on the customer—the consumer of our services—and specifically on how best to meet the customer’s needs. It is not about the author or the producing component; it is about the recipient. It is about helping that customer understand an issue. It is about being timely, relevant, expert, and corporate in our approaches to providing service, intelligence analysis. Success is measured not by the number of units produced, but by how well the product addresses and answers the specific concerns of an identified and targeted audience.
Product and service are not mutually exclusive. Ideally every product we produce should be infused with a service mentality—although we often act like we are in the product business. What difference does it make? When the product is more important than the service it provides, we relax our standards to get the product—another unit of production—off the assembly line and out the door. Close enough becomes good enough, and the brand name suffers.
To infuse every product with a service mentality requires two things of intelligence analysts: One is a set of standards—the DI Quality Framework in CIA’s case; the other is mastery of a simple technique—asking two questions before writing or briefing: who is the primary audience for this piece and what is the specific intelligence question they need help with?
It is very hard for the author of a piece to have a service mentality when he or she is focused on a broad intelligence topic rather than a specific intelligence question. It is the difference between “we need a piece on the demonstrations in Tunisia” and “we need a piece on the options the Tunisian government has for addressing the cause of the demonstrations.” A good intelligence question has the following properties: it bounds or narrows the subject matter to be addressed; it generally contains a what, who, why, or where is it going element; it is specific as to the topic or event being addressed; and it is a question and generally not a “yes or no” question.
It is possible to have many different intelligence questions for the same event. Current intelligence pieces generally work best when they are organized around one central question, although they may touch on others. Which question to focus on is determined by who is selected as the primary audience and what that audience is most interested in or most needs to understand.
Forty years of experience have taught me that failing to identify a specific audience and an intelligence question up front is often at the root of the weakest analytic efforts. In the Art of Review Seminar we talk about “The Road to Ruin,” the first step on which is not clearly defining the issue to be addressed. This in turn easily leads to other, too common, failings in analytical writing:
A failure to present a clear basis for judgments.
A weak piece typically speculates on what happens next but seldom provides the reason an analyst believes the speculation is correct. The most underused word in CIA DI analysis is “because.” Every “may” and “likely to” and “could” requires a “because” statement or its equivalent—the reason we believe what we believe. Absent the “because,” or its equivalent, that article is just another opinion in a town full of opinions.
The use of imprecise language.
It is not so much that language in a work of analysis is opaque but that the point it is trying to make does not come through. It is stating that “X benefits from Y” without providing a standard by which to measure the benefit or spelling out precisely how and why X benefits. Words like “limits,” “benefits,” “suggests,” and all adverbs need a “because” or “why” or “how” to convey precise meaning. Internal inconsistencies, not surprisingly, are often rooted in imprecise language.
The Six Things I Learned
We all learn the craft of intelligence analysis by doing. The lessons are iterative and frequently opaque, and they generally come slowly. Often they are only clear in looking back. Now looking back over nearly 40 years, I think I have learned the following six things.
First, how one thinks about the mission affects deeply how one does the mission.
I think the intelligence analyst’s mission is less about “connecting the dots” (although sometimes it is) or predicting the future (although sometimes it is) or speaking truth to power (although we often do) than it is about understanding the world. Dots and prediction and truth can cause us to narrow our focus in a world of intelligence challenges that are characterized by their complexity and most important, by their dynamic nature. In 40 years I learned that quite often the most important piece of the puzzle, and often the hardest one to get a handle on, is what the United States is doing in a given situation—or, in military intelligence terms, understanding the “Blue” component of a situation.
I always thought of my job as “bounding uncertainty” and by doing so helping make my guy smarter than their guy, whether it was across a conference table or across a battlefield and enabling our policymakers to make the best decisions possible given the time and information available. Sometimes that involved connecting dots or predicting courses of action or providing warning, but it always meant understanding the forces at work in any situation—the key variables and drivers and our adversary’s perspective. It is the difference between strategic understanding and tactical command of an issue.
Second, intelligence failures come from failing to step back to think about underlying trends, forces, and assumptions —not from failing to connect dots or to predict the future.
When our focus becomes too tactical we fail to see the strategic. We must learn to step back from time to time and ask ourselves: what are we not seeing that we would expect to see if our line of analysis were correct. The IC’s 24-hour production cycle often makes this hard to do, but because it is hard to do, it is essential that we do it.
An understanding of history and culture is key to coming to grips with the assumptions that underpin much of our analysis. And I am not talking about our history and culture, but the history and culture of the countries we work on as the people and leaders of those countries understand them. Every analyst—regardless of discipline or role—needs a deep appreciation of how a people see themselves, their historical ambitions, and their grievances. For analysts focused on foreign leaders, or politics, or economics, it is essential that they understand how power is acquired, the preferred way of exercising power, and the acceptable and unacceptable uses of power, as well as the defining life experiences of the key actors in the countries they specialize in.
Third, good analysis makes the complex comprehensible, which is not the same as simple.
The key to making the complex comprehensible is having in mind a specific audience and a very precise intelligence question for the analysis to tackle. Data dumps and murky analysis almost always are rooted in trying to write about a development without first asking, “Who is my audience and what specific question does it need answered?” It is that difference between “we need a piece on the rioting in Athens” and “we need a piece on the government’s options for addressing the underlying cause of the rioting.”
We do very well as a rule in responding to questions from policymakers. We come up short when we have to supply the audience and the question ourselves and we start to write before we have done all the thinking. If we think in terms of answering well defined questions, we can make complex situations comprehensible, and we also stand a better chance of making clear what we know and do not know accurately, conveying our level of confidence, and presenting a convincing basis for our judgments.
Fourth, there is no substitute for knowing what one is talking about, which is not the same as knowing the facts.
Former CIA Director Michael Hayden once famously said, “If it is a fact, it ain’t intelligence.” The business of intelligence analysts is more about putting facts in perspective than it is having command of the facts. We are paid not for what we know, but for our ability to think about what we know—or think we know. It is about knowing what is important. It gets back to those assumptions, drivers and variables I dwell on.
Sources—clandestine, open source, technical, diplomatic, etc.—are not the same as knowledge. Sources are not the equivalent of, or a substitute for, expertise, the type of knowledge I talked about in the second thing I learned. All sources are best thought of as opinions, some more authoritative than others, but all should be subject to careful reflection and comparison to what we know and believe. The dangers in sources are three-fold:
-
- We tend to give greater credence to those that support what we already believe.
- Sources are not a scientific sample but a small slice of a much larger and more complex information picture.
- They never answer the critical question of what are we not seeing but should see if our analysis were correct.
During one of the most challenging times in my analytical career, I worked for the finest analyst I ever knew. In the middle of the Tiananmen Crisis in 1989—when everyone’s hair was on fire—I found him late one afternoon going through a stack of musty old reports. I asked him what he was doing. He said, “I am looking for things that did not make sense then, but do now.” He found some, and it profoundly affected our line of analysis.
Fifth, intelligence analysis starts when we stop reporting on events and start explaining them.
Our production cycle puts a premium on being agile, quick, and smart. It is often 24 hours or less. The DI is one place where a consumer can ask a question and get an answer—a thoughtful and considered one—overnight. It is one of the DI’s greatest strengths. It is also one of its great vulnerabilities. It makes it harder to step back and think about underlying causes, drivers, and variables, especially in a crisis situation. My Tiananmen story is the exception. My career as an analyst taught me that lesson one (how we think about the mission) and lesson two (understanding forces at work) are the key to operationalizing lesson five—the need to explain events.
Sixth, managers of intelligence analysts get the behavior they reward, so they had better know what they are rewarding.
This is a message for all managers and all who aspire to management. It is my experience that if you have clear standards and are seen as consistent and fair in applying them, your unit will live up to the standard. And, you must also hold yourself to the same standards. If you value analytic tradecraft, talk about it and practice it. If you want open communication where different interpretations are considered, invite it. If you want honesty, be honest. And reward the behavior you profess to value.
There is a Chinese proverb: “If your vision extends one year, grow wheat; if it extends 10 years, plant trees; if it extends 10,000 years, grow and develop men.” Managers, your job is to grow men and women who can do the mission. The standard of success, I believe, is uncompromisingly simple: “Did I leave the unit I led stronger than I found it?”
Why It All Matters
If there is an underlying reality to all that I have learned, it is the obvious: we are in a very difficult business. It is more life and death now than it was in my heyday. The consequences of getting analysis wrong are much greater now. Intelligence is also more “political” now in the sense that what is done today is more open than it has ever been and as a result more subject to partisan sniping.
There are some who say the United States is a declining power or that it is the source of many of the world’s problems. Time will tell on the first question, but I believe the United States is a force for good in the world, and how powerful a force depends as much on our knowledge as on our military and economic might. I tell intelligence analysts I teach that more often than not they are the source of that knowledge. It is their professionalism and tradecraft that provide checks on the system, light the way, and leverage US power. All the dollars spent on intelligence—the collectors in the field, the technical systems, and the lives at risk—are for naught, unless that knowledge comes together in what analysts do every day.
As the deputy executive director at the CIA, I addressed each class of just-promoted CIA Senior Intelligence Service officers, and each time I asked for a show of hands of those who believed they would never see WMD used on US soil in their lifetimes. The question always startled them, and I never saw a single hand raised. We cannot afford to accept anything less than the pursuit of perfection. We cannot accept anything less than holding ourselves to the highest standards. We cannot accept anything less than our best effort every time, every day. The potential consequences are too great.
And I know it is damn hard. Intelligence analysis is less fun than a policy rotation or an overseas assignment. It is less honored and romanticized than other aspects of the Great Game. It is frustrating. It is exhausting. And even the best efforts will be picked at. The analyst’s work will be criticized by the knowledgeable and the ignorant alike. It will even be demonized at times—independent of its quality—and it will always be hostage to the politics of the moment.
But—and I say this with my four decades of perspective—what intelligence analysts do has impact. It matters. I have seen the quiet victories of intelligence and the mistakes averted, and I have seen critics become advocates because of what analysts do every day.
We all chose careers in intelligence for the same reason: to make a difference, to do the mission. The colleague who teaches the Kent School’s Art of Review Seminar with me tells a story about Abraham Lincoln, who in one of the darkest hours of the Civil War attended a Sunday service in that little church that still stands across from the White House. On his way back, he was asked by a fellow parishioner what he thought of the young reverend. Lincoln replied that he had a strong voice and clear message, but that he failed to do one thing; he failed to ask us to do something great.
I am asking every analyst who reads this to do something great. Do what brought you here. Do the mission every day to the best of your ability. And, may God bless you for doing it.
All statements of fact, opinion, or analysis expressed in this article are those of the author. Nothing in the article should be construed as asserting or implying US government endorsement of an article’s factual statements and interpretations.
