Why Software Reuse has Failed and How to Make It Work for You

Douglas C. Schmidt
Department of Electrical Engineering and Computer Science
Vanderbilt University
http://www.dre.vanderbilt.edu/~schmidt/
d.schmidt@vanderbilt.edu

An earlier version of this article appeared in the C++ Report magazine, January 1999.


Introduction

Although computing power and network bandwidth have increased dramatically in recent years, the design and implementation of networked applications remains expensive and error-prone. Much of the cost and effort stems from the continual re-discovery and re-invention of core patterns and framework components throughout the software industry. The heterogeneity of hardware architectures, the diversity of OS and network platforms, and stiff global competition are making it increasingly infeasible, however, to build networked applications from scratch with the following qualities: Developing software that achieves these qualities is hard; systematically developing high quality reusable software components and frameworks is even harder [GoF:95]. Reusable components and frameworks are inherently abstract, which makes it hard to engineer their quality and to manage their production. Moreover, the skills required to develop, deploy, and support reusable software have traditionally been a ``black art,'' locked in the heads of expert developers. When these technical impediments to reuse are combined with common non-technical organizational, economical, administrative, political, and psychological impediments, achieving significant levels of software reuse throughout an organization becomes decidedly non-trivial.

During the past decade, my colleagues and I have worked with hundreds of telecommunication, aerospace, and medical companies and written millions of lines of code while developing widely reusable middleware components and frameworks [Schmidt:01,Schmidt:97] for networked applications. We've also had the opportunity to document several dozen patterns [Schmidt:00] and architectures [POSA:96] that guide the design of these components and frameworks. In addition, we've taught hundreds of tutorials and courses on these topics for thousands of developers and students. In spite of formidable non-technical and technical challenges, we've identified a solid body of knowledge, experience, and software artifacts that can significantly enhance the systematic reuse [Griss:96] of networked application software.

In this article, I outline common reasons why reuse has failed in the past. I then discuss proven steps that organizations, projects, and individuals can take to avoid these traps and pitfalls. The key themes pervading this article are:


Why Software Reuse has Failed Historically

Reuse has been a popular topic of debate and discussion for over 30 years in the software community. Many developers have successfully applied reuse opportunistically, e.g., by cutting and pasting code snippets from existing programs into new programs. Opportunistic reuse works fine in a limited way for individual programmers or small groups. However, it doesn't scale up across business units or enterprises to provide systematic software reuse. Systematic software reuse is a promising means to reduce development cycle time and cost, improve software quality, and leverage existing effort by constructing and applying multi-use assets like architectures, patterns, components, and frameworks.

Like many other promising techniques in the history of software, however, systematic reuse of software has not universally delivered significant improvements in quality and productivity. There have certainly been successes, e.g., sophisticated frameworks of reusable components are now available in OO languages running on many OS platforms. In general, however, these frameworks have focused on a relatively small number of domains, such as graphical user-interfaces or C++ container libraries like STL. Moreover, component reuse is often limited in practice to third-party libraries and tools, rather than being an integral part of an organization's software development processes.

In theory, organizations recognize the value of systematic reuse and reward internal reuse efforts. In practice, many factors conspire to make systematic software reuse hard, particularly in companies with a large installed base of legacy software and developers. In my experience, non-technical impediments to successful reuse commonly include the following:

As if these non-technical impediments aren't daunting enough, reuse efforts also frequently fail because developers lack technical skills and organizations lack core competencies necessary to create and/or integrate reusable components systematically. For instance, developers often lack knowledge of, and experience with, fundamental design patterns in their domain, which makes it hard for them to understand how to create and/or reuse frameworks and components effectively.

I've also observed that developers often put too much faith in language features, such as inheritance, polymorphism, templates, and exception handling, as the primary means to foster reuse. Unfortunately, languages alone don't adequately capture the commonality and variability of abstractions and components required to build and systematically apply reusable software in complex application domains.


How to Make Systematic Reuse Work for You

Although the track record for systematic software reuse has been rather spotty historically, several key trends bode well for software reuse in the future:

These trends are particularly evident in markets, such as electronic commerce and data networking, where reducing development cycle time is crucial to business success.

Over the past decade, I've worked with many companies, including Boeing, Cisco, Ericsson, Iridium, Kodak, Lucent, Motorola, SAIC, Siemens, and Sprint, building reusable networked applications using OO design techniques and programming languages [Schmidt:00,Schmidt:01]. These projects have applied a range of reusable middleware tools including CORBA, the ACE framework [Schmidt:01], which is a C++ framework that implements many patterns for concurrent networked applications, and TAO [Schmidt:97], which is a high-performance, real-time implementation of CORBA that leverages the framework components in ACE.

The following discussion summarizes the lessons my colleagues and I have learned making systematic reuse work with ACE and TAO, which have been deployed in a wide range of commercial applications in many domains, including aerospace, data- and tele-communications, medical, process automation, simulation, and financial services. ACE and TAO are freely available, open-source software packages that consist of frameworks, components, examples, and regression tests containing over one million lines of C++ code. More than 100 person-years of effort have been invested to develop, validate, optimize, and document these reusable assets.

Although there is no magic methodology or process that's guaranteed to foster systematic reuse, I have personally seen the recommendations below applied successfully numerous times over the past decade on many projects at many companies around the world.


Strive to Create the Prerequisites for Successful Systematic Reuse

In my experience, systematic software reuse is most effective when the following prerequisites are met:
  1. The market is competitive -- In a competitive business environment, such as financial services or wireless networking, time-to-market is crucial. It's therefore essential to leverage existing software to reduce development effort and cycle time. When a market is not competitive, however, organizations tend to reinvent, rather than reuse, software.

    The defense industry in the 1980's is a good example of the ``reinvent rather than reuse'' phenomenon. During the defense buildup in the Reagan administration, defense-related R&D funding was abundant. Not surprisingly, little systematic reuse was achieved, even though the Ada programming language was designed explicitly to support software reuse. Starting in the early 1990's, however, the substantial decrease in DoD funding had a powerful impact on the defense industry. Contemporary aerospace and defense contractors now aggressively seek to integrate "commercial-off-the-shelf" (COTS) software and hardware in order to remain competitive in their changing marketplace.

  2. The application domain is complex -- Components that are relatively easy to develop, such as generic linked lists, stacks, or queues, are often rewritten from scratch rather than reused. In contrast, developers working in highly complex domains, such as distributed, real-time systems are often willing to reuse components, such as dynamic scheduling frameworks or real-time ORBs, when building equivalent solutions from scratch proves too error-prone, costly, or time-consuming.

  3. The corporate culture and development process are supportive -- Not only is it hard to develop high-quality reusable components and frameworks, it's even harder to reap the benefits of reuse immediately. Significant investment must be expended up-front to produce efficient, flexible, and well-documented reusable software assets before they can be leveraged in subsequent generations of a product line. Therefore, organizations must support an appropriate software development process that allows systematic reuse to flourish.

    Ideally, an organization's software process should reward developers who invest the time and effort to build, document, and reuse robust and efficient components. For instance, a reward system could be built into project budgets, with incentives based on the number of software components reused by individuals or groups. I still find companies, however, whose processes measure programmer productivity solely in terms of the number of lines of source code written from scratch, which penalizes developers who attempt to reuse existing software.

  4. Attractive ``reuse magnets'' exist -- To attract systematic reuse, it crucial to develop and support ``reuse magnets,'' [Foote:97] i.e., well-documented framework and component repositories. These repositories must be well-maintained so that application developers will have confidence in their quality and assurance that any defects they encounter will be fixed promptly. Likewise, framework and component repositories must be well-supported so that developers can gain experience through hands-on training and mentoring programs.

    In my experience, ``open-source'' development processes are an effective process for creating attractive reuse magnets. Open-source processes have yielded many widely used software tools and frameworks, such as Linux, Apache, GNU, ACE, and TAO. The open-source model allows users and developers to participate together in evolving software assets. One of the key strengths of this model is that it scales well to large user communities, where application developers and end-users can assist with much of the quality assurance, documentation, and support [Schmidt+Porter:01].

    As a general rule, software development does not scale up as the number of developers increases. The main problems stem from the increased human communication and coordination costs associated with large projects. A team of 10 good developers can therefore typically produce much better quality software systems with much less effort and expense than a team of 1,000 developers.

    In contrast, however, software debugging does scale up as the number of developers helping to debug the software increases. The main reason for this, of course, is that all other things being equal, having more people test the code will identify its ``error-legs'' much more quickly than having just a few people testing code. A team of 1000 testers will therefore typically find many more bugs than a team of 10 testers.

    Moreover, open-source development efforts tend to have short feedback loops between the point when a bug is discovered and the bug is fixed. This increases the incentive for the user community to help with the debugging process since they are ``rewarded'' by rapid feedback and fixes once bugs are identified. In addition, because the source code is available for inspection, developers in the user community can often help fix any bugs they find, which further amortizes the overall debugging effort and improves software quality rapidly.

  5. Strong leadership and empowerment of skilled architects and developers -- I've observed that the ability of companies and projects to succeed with reuse is highly correlated with the quality and quantity of experienced developers and effective leaders. Conversely, reuse projects that lack a critical mass of developers with the necessarily technical and leadership skills rarely succeed, regardless of the level of managerial and organizational support.

    In general, the level of experience required to succeed with systematic reuse depends largely on whether programmers are trying to develop reusable components or to use them. I've found that developing reusable frameworks and components for complex domains, such as telecom or avionics, requires highly experienced and skilled architects and developers. These individuals must be trained and empowered to create, document, and support horizontal middleware platforms that reduce the effort required to develop vertical applications.

    In general, the more complex the domain, the greater the skills and leadership required to develop effective reusable middleware that can encapsulate complex communication protocols and mechanisms for concurrency, locking, persistence, fault tolerance, connection management, event demuxing, and service configuration. When middleware architects and developers are successful, they create component abstractions that hide these error-prone and tedious mechanisms and protocols. Application developers therefore needn't be as experienced with complex systems-level technologies since they can program to these higher-level component abstractions.

    I've also observed, however, that horizontal middleware platform efforts generally fail when application developers are (1) too inexperienced, (2) the domain is sufficiently challenging, and (3) the middleware team lacks sufficient training, resources, time, or empowerment to create a stable platform. It's therefore important that developers at all levels improve their technical skills and learn how to apply good software principles, patterns, and practices.

Unfortunately, many organizations lack the five prerequisites described above. As a result, these organizations often fall victim to the ``not-invented-here'' syndrome and redevelop many software components from scratch. However, deregulation, global competition, and the general dearth of experienced application and middleware developers is making it increasingly hard to succeed by building complex networked applications from the ground up.


Support Iterative Development Processes and Grow Reuse Assets Incrementally

For systematic reuse to succeed in-the-large, organizations must recognize that good components, frameworks, and software architectures require time to design, implement, optimize, validate, apply, maintain, and enhance. Creating reusable software assets requires a mature organization whose developers and architects can distinguish key sources of variability and commonality in their application domain. Identifying and separating these concerns for complex networked applications requires an iterative development process since it's hard to design reusable artifacts correctly the first time using a top-down ``waterfall'' software lifecycle model.

In my experience, many companies have a hard time getting even relatively small systematic reuse efforts to work right the first time. It's therefore essential to resist the urge to institutionalize ambitious corporate-level reuse initiatives before the people and the processes have reached the proper maturity. I've found that a much more effective process in practice is to iteratively grow reuse efforts out of smaller-scale successes that allow developers and managers to build the necessary skills and confidence.

For systematic reuse to succeed therefore, management must have the vision and resolve to support the incremental evolution of reusable software assets. Fred Brook's observation that ``Plan to throw the first one away, you will anyway'' [Brooks:75] applies as much today as it did 25 years ago. Increasingly, an 80% solution that can be deployed and evolved incrementally is preferable to waiting for a 100% solution that never ships. This phenomenon is referred to as the ``piecemeal growth'' model of software deployment, sometimes referred to by the more colorful name ``Worse is better'' [Gabriel:98]. According to this model, ``the best is the enemy of the good'' since striving for perfection makes it hard to hit tight market windows and gain initial market share.


Maintain a Close Feedback Loop Between Middleware Developers and Application Developers

Most useful middleware components and frameworks originate from solving real problems in particular application domains, such as telecommunications, medical imaging, avionics, or electronic commerce. A time-honored way of producing effective reusable middleware, therefore, is to generalize and refactor them from working systems and applications.

In contrast, reuse efforts that try to work top-down, e.g., from high-level domain analysis, are highly likely to fail. The culprit is often the lack of close feedback loops between developers of reusable middleware and developers of applications. For instance, I've observed that it's usually counter-productive to create reuse teams that build middleware frameworks and components in complete isolation from application teams.

Since reuse efforts rarely have sufficient resources to please all possible customers, it's important to be goal-directed, rather than exhaustive, in determining which assets to develop, enhance, and maintain. Without intimate feedback from application developers, therefore, the software artifacts produced by component teams rarely address core business problems and won't be reused effectively.

It's also important that the relationship between application developers and reuse groups be mutually synergistic. For instance, reuse teams should be responsive to fix problems that inevitably arise in their middleware. Likewise, application developers should actively help to improve reusable artifacts, rather than waiting passively for the reuse team to find and fix all the problems. While it's possible to depend upon developer altruism and good will to achieve these goals, it's usually more effective to institutionalize a reward system with incentives to encourage effective relationships between developers and users.


Buy, Rather than Build, System Infrastructure and Middleware Integration Frameworks

Frameworks can be classified according to their scope, as follows [Schmidt+Fayad:97]:

In general, system infrastructure and middleware integration frameworks focus largely on internal software development concerns. Although these frameworks are essential to creating high quality sofware rapidly and cost-effectively, they typically don't generate substantial revenue. It's therefore often more cost effective to buy system infrastructure and middleware integration frameworks rather than build them in-house.


Develop Software Based on Architectures, Rather Than on Particular Middleware Technologies

It's very risky to expect that industry standards, such as CORBA, J2EE, or .NET middleware, will eliminate the complexity of developing networked applications. No single middleware technology is a panacea. Moreover, industry standards for middleware are not ubiquitous, nor are they implemented consistently.

For large-scale, long-lived networked applications it's essential to design and use architectures that transcend any specific technology or middleware standard. For instance, programming directly to a proprietary middleware API is risky since these APIs can rapidly become obsolete. I've therefore found it's more fruitful to develop networked applications based on a common architecture that can be instantiated on a range of middleware and OS platforms.

An effective pattern I've seen applied repeatedly to organize a common software architecture is to use (1) frameworks in the horizontal middleware platform layers and (2) components in the vertical application layers. Components are self-contained instances of abstract data types (ADTs) that can be plugged together to form complete applications. Common examples of components include COM+ controls and CORBA Object Services.

The relationship between frameworks and components is highly synergistic, with neither subordinate to the other. Frameworks can be used to develop components, whereby component interfaces provide Facades for internal class structures inside a framework. Moreover, components can be used as ``pluggable strategies'' within a framework.

Compared with frameworks, components are less tightly coupled and can support binary-level reuse more readily. For example, application developers can reuse components without having to subclass from existing base classes. In my experience, application developers are generally more comfortable and successful programming with components than they are customizing frameworks. Conversely, frameworks are useful for middleware teams because they help to simplify the development of horizontal platform software. Naturally, components can also be used to develop infrastructure and middleware.


Avoid One-dimensional ``Solutions'' Complex Software Development Problems

Trying to apply one-dimensional technical solutions to complex software development problems is an exercise in frustration and a recipe for costly project failures. For instance, attempting to translate software implementations entirely from high-level SDL specifications or from abstract ``analysis rules'' rarely succeeds for complex networked applications. Likewise, using the latest design methodology, modeling notation, programming language, or middleware technology fads can't guarantee success.

The urge to apply one-dimensional solutions to complex problems isn't limited to technologists, however. For instance, there is a school of thought that claims only the non-technical impediments to reuse are worth addressing since systematic reuse fails solely for economic and organizational reasons, not technological ones. According to this perspective, investing in education or training to improve the technical skills of developers is pointless because it has no impact on success.

Unfortunately, one-dimensional non-technical solutions are no better than one-dimensional technological solutions. Managerial and organizational support is certainly desirable and essential for large-scale adoption of systematic reuse across an enterprise. In my experience, however, this support is not sufficient, nor even always necessary, to succeed with systematic reuse, particularly within smaller parts of organizations.

Moreover, focusing solely on organizational and economic impediments at the expense of technology and skills-building, can yield a corporate culture of ``learned helplessness.'' Developers suffering from this malady often postpone improving their design and reuse skills until the entire organization is ``cured.'' This approach is as futile as waiting for all the customer requirements to solidify before engaging in architecture and design phases.

Failing to invest in technology and education can greatly hamper a company's ability to compete effectively, particularly when time-to-market is crucial to success. It can also cause companies to become dangerously out of touch with contemporary software practice, where an increasing number of companies are in fact succeeding with systematic reuse and COTS technology adoption. Not surprisingly, many of the large companies that suffered the most during the economic downturn in 2001 were also companies that most strongly resisted adopting systematic reuse and COTS.

I therefore believe that we must not wait passively for organizational and economic problems to be resolved completely before building the technical skills and experience level of developers. Instead, we must initiate and support skills-building education now and sustain them over time. These skills are ultimately required to succeed with systematic reuse, in particular, and high-quality software development, in general.


Respect and Reward Top-Notch Developers and Architects

Developing robust, efficient, and reusable networked applications requires teams of people with a wide range of skills. We need experienced managers who know how to properly evaluate risks and opportunities in order to navigate their teams through the constantly changing landscape of technology and business drivers. Likewise, we need expert analysts and designers who have mastered design patterns, software architectures, and communication protocols in order to alleviate the inherent and accidental complexities of networked applications. Naturally, we also need seasoned programmers who can implement these patterns, architectures, and protocols to form reusable frameworks and components.

For systematic reuse to succeed, it's crucial that developers who build reusable assets are empowered to own, maintain, and enhance these assets. Ultimately, reusable components and frameworks are only as good as the people who build and use them. In my experience building complex networked applications, there's simply no substitute for hiring high-quality, experienced software developers. The key to success is to hire developers whose skills and knowledge of patterns and design principles have withstood the test of time.

In practice, of course, it's hard to find top-notch software developers. Ironically, many companies---particularly large ones---still treat their developers as interchangeable, ``unskilled labor,'' who can be replaced easily. I'm increasingly noticing, however, that companies who respect and reward their top-notch software developers consistently outperform those who don't.


Invest in Continual Training, Mentoring, and ``Hands-on'' Experience

The principles, methods, and skills required to develop reusable software simply cannot be learned by generalities or platitudes. Instead, developers must learn through experience how to design, implement, optimize, validate, maintain, and enhance reusable software components and frameworks. Only by repeatedly engaging in these activities over time will developers truly internalize good development principles, patterns, and practices.

Systematic reuse is largely a by-product of good designs and experienced developers. Education is crucial to help improve developers' design skills. Fortunately, developing good reusable software requires many of the same set of skills, such as knowledge of architectures, patterns, frameworks, and components, necessary to develop good software in general. The time and effort spent on education will pay off therefore, whether or not developers actually write reusable software artifacts.


Keep the Faith

I have repeatedly witnessed organizations that initiate systematic reuse efforts with the best of intentions, only to lose faith when various impediments arise or schedules slip. Inevitably, they then fall back onto familiar processes, i.e., developing their software from scratch. I've observed that reuse-in-the-large is best achieved when development and management leaders are unwavering and evangelistic. Moreover, this faith must be propagated up to, and echoed by, the highest levels of the organization.

Ultimately, organizations that attempt systematic reuse without providing an incubation environment will lose their faithful. Many of these faithful will be the most experienced developers or those most capable of coming up to speed quickly. In markets driven by ``Internet cycle times,'' the loss of valuable developers can devastate an organization's long-term competitive viability.

Keeping the faith requires keeping abreast of external R&D developments and global technology trends. In my travels throughout the software industry, I am continually amazed by the rate at which reuse and COTS middleware is being adopted in many businesses and application domains. I suspect that the pundits who dismiss reuse as a myth simply haven't spent enough time ``in the trenches'' lately to recognize the speed at which the software industry is moving away from programming applications from scratch to integrating applications out of reusable frameworks and components.


Concluding Remarks

Over a decade's worth of experience developing and deploying reusable networked application artifacts has taught me the importance of increasing developers' knowledge of patterns, as well as improving their skills at creating and supporting reusable components and frameworks. For more information on using patterns to build reusable networked application frameworks and components like CORBA, ACE, and TAO, I recommend you check out the following URLs:

www.dre.vanderbilt.edu/~schmidt/patterns.html
www.dre.vanderbilt.edu/~schmidt/ACE.html
www.dre.vanderbilt.edu/~schmidt/corba.html
www.dre.vanderbilt.edu/~schmidt/TAO.html

I want to stress, however, that these technological solutions alone are not silver bullets [Brooks:87]. I firmly believe the promise of systematic reuse for networked applications will not be fully realized until we address both technical and non-technical impediments effectively. I also strongly believe, however, that there's no virtue in waiting for organizational and managerial maladies to be resolved completely before improving the education and experience of software developers. Fortunately, most software professionals are eager to hone their technical skills, so future impediments to successful reuse will be largely self-imposed.


Acknowledgments

I would like to thank Jim Coplien, Nynke Fokma, Beki Grinter, and Pat Sciacca for highly spirited discussions that helped to shape many of the key points in this article. Thanks also to Chris Cleeland and David Levine for comments on earlier drafts of this article.


References

[Brooks:75] Frederick P. Brooks, ``The Mythical Man-Month,'' Addison-Wesley, Reading, MA, 1975.

[Brooks:87] Frederick P. Brooks, ``No Silver Bullet: Essence and Accidents of Software Engineering,'' IEEE Computer, Volume 20, Number 4, April 1987, 10-19.

[Foote:97] Brian Foote and Joseph Yoder, ``The Selfish Class,'' in Pattern Languages of Program Design 3, eds. Robert C. Martin, Dirk Riehle, and Frank Buschmann, Addison Wesley, 1997.

[Gabriel:98] Richard P. Gabriel, ``Patterns of Software, Tales from the Software Community,'' Oxford University Press, 1998.

[GoF:95] Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley, Reading, MA, 1995.

[Griss:96] Martin Griss, Systematic Software Reuse: Architecture, Process and Organization are Crucial, Fusion Newsletter, Oct 1996.

[POSA:96] Buschmann et al., Pattern-Oriented Software Architectures, Wiley & Sons, 1996.

[Schmidt:97] Douglas C. Schmidt, David Levine, and Sumedh Mungee, ``The Design and Performance of Real-Time Object Request Brokers,'' Computer Communications, Volume 21, No. 4, April, 1998.

[Schmidt+Fayad:97] Douglas C. Schmidt and Mohamed Fayad, ``Object-Oriented Application Frameworks,'' Communications of the ACM, Special Issue on Object-Oriented Application Frameworks, Vol. 40, No. 10, October 1997.

[Schmidt:00] Douglas C. Schmidt, Michael Stal, Hans Rohnert, and Frank Buschmann, Pattern-Oriented Software Architecture: Patterns for Concurrent and Networked Objects, Wiley and Sons, 2000.

[Schmidt:01] Douglas C. Schmidt and Steve Huston, C++ Network Programming: Mastering Complexity with ACE and Patterns, Addison-Wesley, 2001.

[Schmidt+Porter:01] Douglas C. Schmidt and Adam Porter, Leveraging Open-Source Processes to Improve the Quality and Performance of Open-Source Software, 1st Workshop on Open Source Software Engineering, ICSE 23, Toronto, Canada, May 15, 2001.


Back to Douglas C. Schmidt's home page.

Last modified 18:06:19 CST 25 January 2019