OPB logo

The DARPA Model for Transformative Technologies: Perspectives on the U.S. Defense Advanced Research Projects Agency
More info and resources at: https://doi.org/10.11647/obp.0184


16. Lessons from DARPA’s Experience

Richard Van Atta, Patrick Windham and William B. Bonvillian

© Chapter’s Authors, CC BY-NC-ND 4.0 https://doi.org/10.11647/OBP.0184.16

DARPA has been considered unique in having successfully promoted transformative innovation for more than six decades. Its ability to do so is based on several key features, which have been elaborated in the chapters of this book. It should be noted that these features have varied in emphasis over the years. Not all of these features existed when DARPA was created—indeed they evolved as the agency evolved. DARPA initially began with little explicit structure, organizational architecture, or management processes. It was largely ad hoc. Its first programs were large collections of projects aimed to tackle various aspects of the three Presidential issues it was given as its first assignments: (1) get the U.S. into space; (2) missile defense; (3) nuclear test detection. These were large, umbrella tasks for which there was no well-defined path—they were all highly exploratory and required multiple approaches. Soon after it was established, DARPA took on another area of research—Project AGILE, which was to provide technical support to counterinsurgency in Vietnam. This began a decade-long on-going program which in many ways, in retrospect, had features contradictory to almost all of those identified by the chapters in this book. AGILE also was an ignominious failure. This failure could be due to the fact that it was trying to develop technical solutions to intrinsically political problems. But it was also due to the fact is it was run without any of the discipline and clear management focus that have become associated with DARPA programs. There is perhaps an important lesson here: just being ambitious and taking on a major problem, such as “counterinsurgency” is not enough—it is necessary to bound that problem and apply well-founded management principles.

Important Features

In this Conclusion, we want to highlight some of the most important of these features:

Ambitious technical goals. DARPA focuses on high-risk, high-potential projects. Moreover, it is not simply a research agency. Its focus and goal is to create and demonstrate new significant technologies and systems. DARPA works on both basic technologies and components and on prototype systems that use these advanced technologies to demonstrate new and valuable equipment, processes, and other systems. It therefore both helps to create new technologies and applies them in useful and novel ways.

Organizational independence. DARPA takes on problems that are beyond or outside those of other defense organizations. One way this has been phrased is that if there is a defined “requirement” for something—that is, there is a known approach for accomplishing a specific objective—then it is not a job for DARPA. The Defense Department chartered DARPA to focus on new technologies and approaches, not incrementally improve on what exists.

Freedom from bureaucratic procedures. Moreover, the agency is free of day-to-day bureaucratic interference from other parts of the Defense Department. It can move quickly, without having to check with outside organizations or committees. While the Director and the Office Directors may take overall direction from the Secretary of Defense and some other high-level OSD executives, this is almost entirely focused on stating defense priorities and not how to do the research itself.

Highly-talented managers and a lean management structure. DARPA focuses on some of the hardest and most important technical problems in the U.S., and will succeed only if both DARPA and the R&D performers DARPA funds are among the very best technical people in the country. The agency hires excellent program managers and then lets them propose programs, run competitions, select R&D performers, and work with those performers. The agency has only two layers of management above the program manager: the office director and deputy and the agency director and deputy. Moreover, these managers are themselves technical experts and can make informed technical judgments quickly.

Temporary R&D teams. DARPA does not have its own internal laboratory and instead funds outside R&D performers, usually for projects that last three to five years. As a result, the agency has great flexibility. If progress is strong in a particular program area, DARPA can extend funding over several generations of programs. If a program fails, then it is terminated and funds are used for other work. In addition, once a new technology is created and demonstrated, DARPA can move into other, newer areas.

A technically-sophisticated and well-funded customer. The Defense Department is a sophisticated customer, which makes technology transfer practical. Moreover, DARPA is most successful when senior Defense Department officials not only support the agency’s independence, but also work to transfer its new technologies to the military services.

Continuous management, not post hoc evaluation. Because DARPA program managers are themselves technical experts, they can quickly judge whether R&D projects are succeeding or not and, equally important, can work with R&D performers to change projects when surprises and changes inevitably occur. DARPA therefore has a process of continuous learning. And if a project fails even after changes, then it is terminated. DARPA does not perform post hoc assessments of its research program as is done by many other government research organizations. In this sense it is run more like a business where the “evaluation” is in results, recognizing that not all its projects will succeed.

A credible process for accountability. R&D quality and agency accountability are ensured by picking an excellent director and excellent program managers, by this process of continuous evaluation, and by having oversight (but not heavy paperwork) from senior government officials. There is no need for supervision by committees, outside evaluations and audits, or other bureaucratic steps that would actually slow down the agency’s work.

An effective political design. DARPA has built strong, enduring political support based on its performance, and therefore has had stable budgets and continuing independence. It has this support because it performs a vital mission (keeping the U.S. military technologically advanced), focuses on long-term challenges and opportunities facing the Defense Department, does not threaten the budgets of other DOD agencies, has won the respect and support of the U.S. technical community, and has credible procedures for tracking program progress and maintaining quality.

The chapters in this book present greater detail on these features and also other aspects of its management and operations that are seen as useful for it to fulfill its special mission. Some of these mechanisms have been introduced to assure that DARPA does not get bogged down in stultifying bureaucratic processes that inhibit its flexibility and adaptability. We have emphasized that a key feature is the ability to bring on technically expert program managers for explicit, short term appointments. Another feature is the ability to quickly undertake, but also if needed, quickly cancel specific projects. To make these feasible, DARPA uses flexible hiring and contracting authorities.

Creating New Technical Communities

By funding multi-disciplinary teams that both compete and cooperate with each other, DARPA often helps create new technical communities and new academic fields. Examples over the years include materials science and engineering, computer science, and, more recently, artificial intelligence, autonomous systems, and synthetic biology/engineering biology. In fact, one can argue that DARPA actually makes two very important contributions: it not only helps create and demonstrate new technologies, but also helps create important new technical communities.

These researchers then can perform additional R&D, teach students, and contribute further ideas to DARPA, as well as commercialize the technologies. Indeed, DARPA-funded communities are a primary means for transitioning the newly developed technologies to the military and to commercial companies.

DARPA and the Future

DARPA has existed for over sixty years and has had massive impacts on many areas of defense capabilities. It has also produced much broader, revolutionary advances in information technologies, microelectronics, materials, and other areas, that have had profound economic and societal impacts. DARPA has garnered a reputation as the innovation icon—often pointed to as the most successful U.S. innovation agent. DARPA has changed over time in response to the changing security, technological and governmental landscapes.

DARPA’s higher-risk, longer-term R&D agenda distinguishes it from other defense R&D organizations. Perhaps the most important effect of DARPA’s work is to change people’s minds as to what is possible. DARPA’s sixty-year history reveals an institution driven by a constant imperative to create novel, high-payoff capabilities by pushing the frontiers of knowledge. DARPA has many of the same features as its research. DARPA began as an experiment aimed at overcoming the usual incremental processes of technology development. Like the research it is chartered to develop, DARPA consistently has been purposively “disruptive” and “transformational”. Over the decades, there have been various efforts to tone down DARPA, make its research more compatible and integrated into the rest of DOD R&D, and have it focus more heavily on nearer term, more incremental applications—that is, to shift its focus away from disruptive possibilities.

Additionally, there have been efforts to broaden its charter into prototyping systems beyond the proof-of-concept demonstrations DARPA traditionally has carried out. However, with strong internal leadership, both within DARPA and in the OSD, as well as with support from Congress, DARPA has been able to perform a truly unique role for six decades. It has been, and continues to be, DOD’s “Chief Innovation Agency”, pushing the frontiers of what is possible for the benefit of national security and the nation.

DARPA remains an impressive “opportunity farm”. For example, DARPA helped move “artificial intelligence” (AI) from an inchoate notion with almost no technological underpinnings into pervasive capabilities affecting our everyday lives and supporting real-time military operational decision-making. It is now pursuing similar advances in cognitive computing and robotics. It is pursuing fundamental advances in materials such as biomaterials, and accelerated materials development. The Agency has revolutionized the realm of distributed sensing. Among many current DARPA research topics that populate the opportunity farm are heterogeneous electronics, engineering biology, agile access to space, and hypersonic systems.

Looking to the future, the question is not whether DARPA can still pursue new change-state prospects. The question is this: how can DARPA and the Department of Defense identify and focus on what these should be in the changing geopolitical and technological environments? DARPA has been adroit in addressing emerging technological prospects—but, in today’s world, it has to be yet more focused on where it can have leverage, as others are investing, often massively, in the very technologies that DARPA initially championed.

With global investments in robotics, AI, synthetic biology, quantum computing, and advanced materials, on what should DARPA focus? From a military applications perspective, what should DARPA do to harness and promote the potential use of such emerging technologies into defense uses? To what needs, as opposed to today’s defined requirements, should DARPA seek to employ these technologies? Crucially how should DOD achieve the fruition of these efforts?

Today DARPA faces new challenges that raise a key issue concerning its future success—the ability to draw upon extraordinary technical talent for program managers. The commercial high-tech sector, particularly in such areas as information technology, autonomous systems, advanced biology—areas in which DARPA is focused—is aggressively spending vast sums and hiring the very best. These firms attract this talent with high salaries, relatively unfettered work environments, in locations far from Washington, DC, with foreign nationals making up a growing proportion. Moreover, many leading tech companies are now outside of the U.S., in Asia and Europe. Thus, there is greater competition for technical talent and greater competition worldwide in advanced technologies. Even as DARPA now must confront a tougher recruiting context that it has in the past, it still presents prospective program managers unique opportunities to affect the future that few other organizations can offer. These talent dynamics are crucial to understanding today’s DARPA and its ongoing mission to identify, demonstrate and develop the technologies of the future.

These questions emphasize a crucial point—DARPA does not succeed by itself. Its success resides in the opportunities it creates that others bring into fruition. Thus, its success must build upon the larger U.S. innovation infrastructure. That innovation ecosystem has changed fundamentally over the past twenty-five years. For DARPA research to be successful, it must eventually culminate in transition, whether in an operational military capability or a new field of technology that expands frontiers for decades. DARPA itself is not responsible for executing transitions, but it depends on effective transition paths being there. These paths need to be better understood, and other stakeholders, beyond DARPA, need to support the measures that foster transition—whether within DOD or within industry. Some worry that military transition mechanisms within the DOD have eroded. In the broader commercial economy, transition paths have become more uncertain and diffuse. For DARPA to continue to have transformative impacts, it must exist within an economic and policy environment that encourages implementation. These are critical technology policy concerns that the U.S. must address to ensure that DARPA can continue to deliver breakthrough technologies in the decades to come.