No Truth Here

When I was in grad school, one of my more famous professors usually began any discussion of new material by calling it "crapolla," including his own contributions to the discipline. As a grad student, I was anxious to prove myself and to rise above my peers by working harder and contributing more than anyone around me. Therefore, at first I took the crapolla comment as a sort of cynical sarcasm as I consumed the cool-aid as quickly as it could be served. But once I really internalized things and began applying all this new knowledge in practice, I recognized that he wasn’t being cynical or sarcastic. He was expressing a basic truth about social science methodology: not that it has no value, but that its value must be scrutinized closely before drawing firm conclusions and generating policy prescriptions. It all, truly, is crapolla.

Take Sean Gourley, whom I’ve criticized at length already but who again serves as the quintessential bad example. In making observations, gathering data, and drawing inferences, he has made many policy prescriptions that are based upon severely flawed analysis. The unwary policy makers who takes him seriously will certainly create more problems dealing with insurgency than they will solve.

But the problem that Gourley represents is more insidious and wide spread than that. If you’ve been keeping up with the news, you’re aware of the recent flap over the use of PowerPoint as a communications tool. PowerPoint briefings can be and often are a complete waste of time. But that is not a symptom of the application, but a symptom of the incompetence of the people doing the briefings. A PowerPoint slide is a prop, and should never be the focus of a briefing. That concept is lost upon most people who make PowerPoint presentations. Often, their goal is more to overwhelm the audience with visuals and complexity. The point of that is a very simple one: it avoids difficult questions by making the material too complex and overwhelming to absorb, and therefore to critically analyze. Thus, the briefer can, more often than not, simply be taken at his word, leaving behind a room full of bewildered and amazed onlookers, too confused (or bored) to question anything that they just heard. The tool isn’t the problem, it’s the practitioner.

PowerPoint briefers have no monopoly on this particular technique. The social science community often acts similarly; and the motivation for it is quite clear. Social science modelers come in two basic flavors: those that drank the cool-aid and actually believe what they’re saying, and those that spit it out, but still have to earn a living.

As G.E.P. Box said, "All models are wrong but some are useful." Their usefulness is the insight they can provide into certain types of problems. But they are models, and all models suffer from omitted variables, and always will. As a community of practitioners, whether that practice is wargames, systems models, statistical analysis, whatever, we are perversely incentivized to sell our product as the best representation of ground truth possible. The danger is that we often inspire more confidence than the tool actually deserves. And that can be dangerous indeed. But the point here must be clear: our tools are not the problem, we are. Our methodologies are quite good, often very advanced, and certainly useful. But we must hold onto the awareness that they are also wrong, and where they are wrong is the part that we must pay closest attention to. Technology and our capacity t exploit it is a wondrous thing, but our dependence leaves us vulnerable, and as time goes on, that problem will only get more difficult to address and contend with. The first step to coping may lie in the understanding that it cannot be solved.