Managing Director, BCG Platinion
Companies across many industries are struggling with the transition to Agile, a fast, iterative software-development method. Too many companies have the appearance of Agile—with, for example, hastily converted, brightly colored meeting rooms and daily stand-up meetings—but they achieve little of tangible impact.
At software start-ups in which Agile is commonly used, the development team is at the heart of the business, so buy-in, sustained commitment, and collaboration come fairly naturally. It is not easy, however, to integrate self-directed, cross-functional Agile teams into the existing hierarchy of large companies.
Some large companies, however, are figuring out how to make Agile work. Rather than impose its specific methodologies, they apply its general principles, paying special attention to the integration of Agile teams into the rest of the organization.
When large companies get Agile right, the results can be stunning. Productivity can improve by a factor of three. Employee engagement, measured in quantitative surveys, increases dramatically too. New product features can be released within weeks or months rather than quarters or years. Rates of innovation rise, while the number of defects and do-overs declines. In the first year after going Agile, one bank’s development team increased the value delivered per dollar spent by 50%, simultaneously cutting development time in half and improving employee engagement by one-third.
As the quality of software rises and the responsiveness of processes improves, some companies are applying Agile principles to activities other than software development. For such companies, Agile can become a journey of continuous improvement.
Agile grew out of a desire to improve traditional methods of software development. Customarily, software has been developed sequentially, with the waterfall serving as a rough metaphor for its progression. Separate groups conceive, design, build, test, put into operation, and maintain software, each group waiting for the preceding group to complete its work. The method is inefficient. In many cases, participants spend more time sitting in meetings and managing handoffs across organizational boundaries than writing and testing code. According to the often-cited The Chaos Report less than 10% of large software projects come in on time and on budget.1
The waterfall method comes from engineering, but writing many types of software is different from building a bridge. A river doesn’t change its course, but software users have frequently changing and unpredictable needs. Consequently, Agile relies on bringing together many different points of view and supporting back-and-forth dialogue between developers and business executives.
Many forms of Agile have been developed, but at its heart, Agile is a set of beliefs. It is iterative, empirical, cross-functional, focused, and continually improving.
Putting the Agile set of beliefs into practice can be difficult in large companies, given layers of processes and structures, such as HR, finance, and legal functions. Rather than viewing Agile as yet another new process, companies should integrate Agile values into their own software-development organization and culture, making reasonable modifications when necessary.
There are several best practices that help activate the Agile set of beliefs at large companies. These practices—which embrace iterative, empirical, cross-functional, focused, and continually improving approaches—accommodate the realities of large organizations while staying true to Agile principles.
There are five secrets of success for large-scale Agile transformations.
Transformative change requires support from the top. Senior leaders need to be actively involved in fundamental decisions about the business purpose of going Agile and the cultural barriers and root causes that might stand in the way of success. Without this commitment, legacy approaches to, for example, capital allocation, HR processes, and portfolio management will doom Agile. That’s why business—not just tech—leaders must be accountable.
Agile transformations are different from other transformations: leaders must mobilize management to march in an unfamiliar new direction. The fast pace and cross-functionality of Agile can put many executives out of their comfort zone. Without strong and steady support from the top, many executives and team members revert to the norm. The CEO of a large European bank told us that he wants his organization to operate as a technology company that deals with financial services products.
In a large organization, Agile pilots are necessary in order to determine whether Agile will work there and whether the organization will accept Agile principles. Pilots are critical to a company’s making the necessary adaptations to Agile.
For example, in a scrum, a single product owner takes responsibility for managing the relationship and interactions between developers and customers. This role requires a careful mix of technical and business skills. Companies may need to have even two or three people collectively serving in that role until the organization develops people who have the required multifunctional skills.
Likewise, it might be difficult to fully implement iterative development in all instances, but frequent feedback between developers and business executives ought to be the norm.
Staged rollouts in waves create momentum by building relevant capabilities and ensure that Agile principles and culture are embedded across the organization. (See Exhibit 1.)
The pilot phase is followed by steps that must be executed with some delicacy to avoid unnecessary tension: it’s time to scale up Agile in an organization that may be theoretically willing to accept it but, practically, is challenged to do so.
HR processes, such as performance management, may not be set up to handle fully dedicated cross-functional teams where team—not individual—results matter most. Agile’s flexibility will almost certainly strain budgeting processes even if Agile is ultimately less costly than traditional development activities. An organization’s IT infrastructure may not be set up to accommodate continual integration and deployment because of lengthy provisioning times. Furthermore, traditional development teams may be resentful, and certain activities may be outsourced.
These are all real technology and organizational concerns that will not resolve themselves on their own. Executives must actively manage the integration, and the enterprise almost certainly will have to invest in training and development to encourage the right culture and behaviors.
Several successful approaches exist for scaling up Agile within organizations. At one extreme, the music-streaming service Spotify has fundamentally changed its organization structure. The company’s product-delivery organization is made up of squads, tribes, chapters, and guilds. The primary unit is the squad, a multidisciplinary team that works toward a shared purpose and is run by a product owner. Tribes are groups of squads that work on related areas. Chapters are groups of people with similar expertise across squads, and they form the line organization. Guilds are interest groups that anyone can join. (See Exhibit 2.) Other companies have simply overlaid cross-functional teams above existing hierarchy.
The ultimate goal of Agile is to improve the business. Therefore, the ultimate measurement should relate to business performance. If the goal of a bank’s Agile project is to reduce the dropout rate in credit card applications, then the dropout rate should be the most important metric. But in order to improve the business, companies also need to track software reliability, security, complexity, and size.
That’s where software measurement tools enter the picture. These tools allow companies to demonstrate empirically the productivity and quality improvement of Agile development and the overall performance of Agile teams.
Agile development is an exercise in continuous improvement. It is not a one-off exercise. Agile requires constant monitoring to ensure proper functioning. Companies need to take steps to bake the Agile principles into the organization. There are many ways to ensure that Agile endures. Many companies, for example, create teams consisting of the leaders of each Agile project, and they share best practices.
At its heart, Agile is about creating the right context in which your people—specifically your developers—can do their best work. It is often thought of as a method for writing software, but ultimately, it is a way to run and continually improve your business.