The following document is the foreword to the book Design Patterns in Communication Software, edited by Linda Rising and published by Cambridge University Press, 2000.

The Consequences of Information Technology Commoditization

The commoditization of IT has a number of consequences:

• Greater focus on integration rather than programming -- There is an ongoing trend away from programming systems from scratch towards integrating them by configuring and customizing reusable frameworks and components. While it is possible in theory to program systems from scratch, economic and organizational constraints--as well as increasingly complex requirements and global competitive pressures--are making it infeasible economically to do so in practice. In the future, therefore, most systems will be configured by integrating reusable commodity components implemented by different suppliers.

• Increased technology convergence and standardization -- To leverage inexpensive commodity hardware and software effectively, application developers are converging towards fewer general-purpose tools, platforms, and methods than they've used in the past. For example, rather than choosing from dozens of programming languages, operating systems, network protocols, and middleware, newer systems are being developed with a relatively small number of common tools and platforms, such as C++, Java, UNIX, Windows NT, TCP/IP, and CORBA, many of which are industry standards. Likewise, there's a general consolidation of development methods and modeling notations away from ad hoc technologies towards industry standards, such as UML.

• Growing economies of scale for mass market technology and personnel -- Due to technology convergence, there is an increasing abundance of good technology and personnel available at competitive prices for mainstream mass markets, such as e-commerce or consumer electronics built using general-purpose operating systems, languages, networks, and middleware. However, for niche markets--such as real-time embedded systems written using proprietary platforms, languages, and tools--commodity hardware and software artifacts and personnel are much more expensive--if they can be found at all. Thus, the further an industry diverges from the mainstream mass market (1) the greater the costs it incurs to develop, deploy, and maintain systems and (2) the more its customers must pay for the niche technology.

• More opportunities for disruptive technologies and global competition -- Another consequence of IT commoditization is that industries long protected by high barriers to entry, such as the telecommunication and aerospace industries, will be much more vulnerable to disruptive technologies and global competition. For example, advances in high-performance COTS hardware and fault tolerant middleware are making it possible to build dependable network elements ranging from PBXs to high-speed backbone routers using standard hardware and software components that cost much less than today's highly proprietary systems. In turn, the commoditization of network elements will continue to erode the profit margins that market leaders of communication equipment have enjoyed historically.

• Lower priced, but potentially lower quality components -- According to John Chambers, CEO of Cisco Systems, consumers are the big winners of IT commoditization because ``everything gets cheaper forever.'' Clearly, John must not have purchased a house in Silicon Valley recently ;-). Nevertheless, global competition and time-to-market pressures are generally increasing productivity and driving commodity IT prices to marginal cost for low- and middle-end systems. What's not clear at this point, however, is how well commodity hardware and software can support the dependability and fidelity requirements of high-end systems, such as carrier class central office switches or flight-critical avionics control systems. Unfortunately, there will be little incentive to improve the quality of commodity artifacts as long as:
  1. Most users emphasize price and features over quality and scalability;
  2. Vendors can continue to make money selling lower-quality products inexpensively in mass quantities; and
  3. There's no credible competition or alternatives.

• The decline of internally funded R&D -- Shrinking profit margins and increasing shareholder pressure to cut costs are making it hard for companies to invest heavily in long-term research that doesn't yield short-term pay offs. As a result, many companies can no longer afford the luxury of internal R&D organizations that produce proprietary hardware and software components with customized quality-of-service support. To fill this void, therefore, commodity hardware and software researched and developed by third-parties is becoming increasingly strategic to many industries. This trend is requiring companies to transition away from proprietary architectures toward more open systems, which can reap the benefits of externally funded R&D and open-source development processes.

• Potential complexity cap for next-generation complex systems -- Over the past five years there has been a steady flow of faculty, staff, and grad students out of the traditional research centers, such as universities and research labs, and into startup companies and other industrial positions. While this migration has helped to fuel a global economic IT boom, it's unclear whether this trend bodes well for long-term technology innovation. In particular, without an investment in fundamental R&D it will be hard for developers of next-generation systems to master the complexities associated with the move towards large-scale distributed ``systems of systems'' in many domains. Thus, as the current generation of technology transitions run their course, the systemic reduction in long-term research funding relative to short-term venture capital funding may be limiting the level of complexity of systems that can be developed and integrated using commoditized hardware and software components.

How Patterns are Helping Us Succeed in a World of Information Technology Commoditization

Identifying, studying, and applying patterns and pattern languages can help us ensure that COTS components are created and integrated into high-quality communication systems by addressing the following system development and evolution challenges:

• Communication of architectural knowledge among developers -- Patterns help capture essential properties of software architectures, while suppressing certain details that are not relevant at a particular level of abstraction. For example, patterns help to document software architectures by expressing the structure and dynamics of participants at a level higher than (1) source code or (2) object-oriented (OO) design models that focus on individual objects and classes. Patterns also provide software organizations with shared vocabularies and common architectural models, thereby helping to improve communication within and across developers, teams, and projects.

• Accommodating new design paradigms or architectural styles -- Historically, the adoption of COTS and other technology advances has been limited by the effort required to transition developers trained in traditional techniques to newer paradigms, such as OO analysis, design, and programming. Fortunately, many core patterns have originated in non-OO contexts, such as operating system kernels, I/O subsystems, and databases, that are familiar to developers of legacy systems. Thus, developers who study these patterns can more readily migrate to new paradigms by learning how to leverage and extend domain expertise they gained from their prior experience and express it effectively in newer languages, platforms, methods, and notations.

• Resolving key non-functional forces associated with accidental complexity -- A surprising amount of software complexity is accidental and arises from limitations with development tools and platforms. For instance, the vestiges of legacy hardware and software platform diversity has made it hard to develop portable applications and tools that can run across multiple operating environments, such as POSIX, Win32, and embedded real-time operating systems. These accidental complexities are being addressed today by identifying key patterns and pattern languages and reifying them into reusable middleware, such as CORBA or Real-time Java, that shields developers from many tedious, error-prone, and non-portable programming details.

• Avoiding development traps and pitfalls that were historically learned by costly trial and error -- Developers of communication systems must address many recurring design challenges related to key properties, such as efficiency, scalability, robustness, and extensibility. For decades, successful architects and developers have learned to resolve these challenges by trial and error during a lengthy apprenticeship process. At any point of time, however, the knowledge gained from this process resides largely in the heads of domain experts or buried deep within complex system source code. By externalizing this expertise in the form of patterns and pattern languages, new generations of developers can learn how to avoid common traps and pitfalls in their domains without requiring such an extensive apprenticeship.

Manifesting the Maturation of the Patterns Community

Much of the patterns literature has focused on a few well-traveled domains, however, such as graphical user-interface frameworks or business applications. As the scope and criticality of communication system requirements continues to expand, our most pressing need now has become elevating the focus from relatively small-scale, stand-alone software mechanisms, protocols, and patterns to large-scale, distributed policies, architectures, and pattern languages. Developers of communication systems face many vexing inherent complexities, such as partial failures, distributed deadlock, and end-to-end QoS enforcement, that contain many interlocking aspects. Therefore, unless the panoply of commodity hardware and software point solutions can be consolidated into integrated frameworks, their value will be diminished and can in fact make matters worse instead of better.

Until now, no single book has focused on patterns and pattern languages for communication systems with mission-critical requirements, such as telecommunications systems with 24x7 high-availability requirements or real-time middleware that can provide stringent end-to-end latency and jitter guarantees to applications. The material in this book exemplifies the maturation of the patterns community and is a major contribution to the study of patterns and pattern languages for communication systems. By learning this material you'll be able to better design and implement communication systems that can't be bought off-the-shelf, thereby improving the productivity and quality of your software solutions and staying ahead of your competition.

We are fortunate that Linda Rising has found time in her busy life to assemble the work of an outstanding group of authors for this book. During the past ten years I've had the honor to meet and work with most of these authors, many of whom are internationally renowned experts in their fields. If you want thorough coverage of the patterns and pattern languages that are shaping next-generation communication software read this book. I've learned much from it and I'm confident that you will too.

Douglas C. Schmidt
University of California, Irvine

Back to my editorials home page.

Last modified 18:06:18 CST 25 January 2019