Managing Director & Senior Partner, Chairman of the BCG Henderson Institute
San Francisco - Bay Area
Adam Smith’s “invisible hand” famously guides market economies, in which unplanned transactions by various agents lead to collectively beneficial outcomes. This logic governs most interactions between enterprises and consumers and between enterprises today. But work within enterprises is organized mostly according to the very different concept of bureaucracy, the principles of which were codified and popularized by a later philosopher, Max Weber. In this model, activities are planned, mandated, and guided by the very visible hand of organizational hierarchy.
Despite massive changes in the business environment that have put a premium on agility and innovation—and despite “bureaucracy” having become a synonym for inertia and stagnation—the concept has shown remarkable staying power. How many of us could name more than a handful of enterprises that are not fundamentally organized as bureaucracies? We explore this seeming paradox by examining the nature and functions of bureaucracy, why it has remained the dominant paradigm, and whether and how emerging challenges will require leaders to reinvent their organizational models.
How Bureaucracy Works
According to the transaction cost theory of the firm, companies exist because they can eliminate certain transaction costs that would otherwise be incurred in free markets, such as the effort required to discover prices or renegotiate contracts. As a result, companies can often better coordinate tasks that require a high degree of alignment between multiple parties or involve nuance or context. For example, car manufacturers typically outsource the production of car parts, because the exact shape and functionality of each part can be specified in advance. In contrast, car assembly typically happens within a firm, because the assembly process has many highly interdependent steps and can therefore be error-prone and require nuanced judgment.
Bureaucracy, according to Weber, emerged as an organizational form to create stability and predictability, making enterprises more efficient. In particular, he identified six essential characteristics of bureaucracy that set it apart from other forms of organization:
Most large companies today fundamentally fit Weber’s definition of a bureaucracy. They task employees with job descriptions to facilitate the division of labor and arrange them in multilayered hierarchies through which directives are transmitted, executed, and monitored. Hiring and promotion decisions are based on objective criteria specific to each role. And extensive and relatively static rule sets authorize some actions and forbid others, giving only limited leeway for judgement and initiative based on circumstances.
This organizational form thrived in the stable, predictable environments in which most businesses historically developed. Hierarchical management enabled detailed planning at all levels of the organization; clearly defined roles allowed tasks to be decomposed and executed efficiently; and comprehensive, unchanging rules allowed new hires or resources to be easily integrated into the organization in pursuit of economies of scale.
Bureaucracy’s Staying Power
Despite the model’s popularity, the effectiveness of bureaucracy has increasingly been called into question because of structural changes in the business environment. Uncertainty is rising, disruption is increasing, and what it takes to succeed is changing faster than ever. As a result, companies increasingly must compete on adaptiveness, learning, and innovation. But bureaucracies are inherently ill-suited for these new imperatives: static organizational rules inhibit adaptation; top-down hierarchies are predicated on forecasting and planning rather than experimentation and learning; and highly codified tasks often do not leave sufficient room for discretion or imagination. Indeed, bureaucracy’s demise has been predicted for decades: “The conditions of our modern industrial world will bring about the death of bureaucracy,” wrote leadership scholar Warren Bennis in 1966.
Why, then, has bureaucracy remained the dominant organizational paradigm? The bureaucratic model is simple and well-codified, allowing it to be easily understood and applied by managers in any industry. The things it does well—such as planning what needs to be done, decomposing this objective into specific actions, and coordinating those actions among employees—are still valuable in many situations. Bureaucracy is also a convenient mental model that is often employed even if the actual network of the organization departs somewhat from the formal model.
Some companies have boldly experimented with alternative organizational forms, moving away from the planned, top-down nature of bureaucracy. (See the exhibit, “Alternative Organizational Models.”)
These philosophies share some characteristics, including a focus on avoiding top-down hierarchy and allowing teams to self-manage, with the aims of increasing adaptiveness through decentralized action and increasing innovation through greater employee engagement and motivation. Zappos has gone further than most companies in breaking free from the restrictions of bureaucracy. By fully empowering each team to determine the value propositions it offers and make its own investment decisions, the company effectively brings market forces inside the organization. (See the sidebar, “Zappos’ Journey to Bring the Market into the Organization.”) However, such alternative organizational models are still not proven, scaled, or codified to the degree necessary to make them accepted alternatives for mainstream companies today.
Tony Hsieh joined Zappos as an advisor and investor in 1999 and eventually became CEO. After ten years of rapid growth, Zappos was bought by Amazon in 2009, but as part of the deal Hsieh negotiated that Zappos would retain significant independence in operational control. As a result, Zappos has had the freedom to experiment with its management philosophy, which has increasingly broken with the bureaucratic model.
Hsieh believes that bureaucratic organization models ultimately limit adaptability and innovativeness and are not conducive to human flourishing and happiness. This is in contrast to the market-based models found, for example, in cities and economies. As Hsieh says, “Companies don’t innovate, markets do.” That is why Zappos is increasingly introducing market-based elements into its organizational model.
Zappos’ first major departure from the standard bureaucratic model came in 2015 when it formally converted to holacracy, a model focused on breaking down hierarchy and enabling self-organization. The company has since evolved its organizational model into a form it calls “market-based dynamics”—which extends the logic of self-organization to decentralize financial and investment decisions as well—because the budgeting decisions in holacracy were still centralized, which ultimately limited the capacity of teams to self-manage. Zappos is still experimenting with its new model, but it has demonstrated that there are viable alternatives to bureaucracy.
Structurally, holacracy breaks a firm into independent teams called “circles.” They function like startups with their own independent purpose, but each purpose aligns with the overall organizational purpose. Although the goal of each circle is prescribed, circles are free to organize however they want to achieve their goals. Holacracy does have an element of hierarchy; circles can be nested inside other circles. However, in contrast to typical bureaucracies, circles can have overlapping responsibilities. Additionally, the lines of accountability between supercircle and subcircle are bidirectional and consent-based, so supercircles don’t control subcircles.
Employees are highly autonomous in this model—they can choose which circle or circles to join, and while they agree to deliver specified outcomes for each role they take on, they are completely free to decide how to achieve those outcomes. Unlike bureaucracies, circles don’t have full-time managers; instead, decision making happens through structured group conversations. Consequently, career development works in a very different way. Career paths are characterized by either expertise development or skill set broadening, in contrast to the typical hierarchical career ladder. As lead organizational designer John Bunch says, “Career progression moves from a ladder to a jungle gym. Don’t look up; rather, look around for opportunities and do what fits you best.”
Market-based dynamics extends self-management by not only giving circles the freedom to determine how they achieve their goals but also allowing them to decide what their goals are, as long as they are profitable and consistent with Zappos’ purpose and values. Circles decide on their goals by specifying the “menu of services” they offer (customer-facing services like sales, for example, or internal services like development), and they are free to sell these services to both internal and external stakeholders. Each circle is free to invest profit as it wants, within the constraints of the purpose of Zappos. As Hsieh says, “If an employee wants to start up a circle to set up a cupcake bakery, that would be great. The only demand is that this bakery abides by our purpose to deliver the very best customer service, customer experience, and company culture.”
Additionally, if a circle needs to use specific services to deliver its offering, it can source these services internally or externally, except for a small set of centralized services—like payroll, core IT systems, or legal—that each circle must use and for which it pays a contribution margin. Recruitment and salary levels are currently centralized, but the company plans to decentralize these aspects too over time, which will give circles complete freedom in recruiting new employees and setting salary levels, within the bounds of their own profitability.
New tools are needed to support this decentralized model, because the typical tools assume standard hierarchical structures and decision rights. For instance, Zappos has its own internal platform that allows for decentralized budgeting, and an HR tool that allows for people having multiple roles in overlapping circles.
Success is defined as a balance between efficiency, stability, and resilience. Business metrics typically focus heavily on efficiency and stability but neglect resilience. In order to address this gap, one of Zappos’ primary metrics is the percentage of circles’ income that comes from new sources, because increased income from new sources means that circles are more entrepreneurial, which makes the larger company more resilient.
Eventually Zappos aspires to blur the boundaries between the firm and the market even further by allowing nonemployees to set up Zappos circles as well. These external circles would leverage the basic set of services that Zappos offers, for a fixed percentage contribution margin. Just as software as a service and computing as a service (cloud computing) lowered the barrier to developing new technology, Zappos hopes that this model will lower the barriers to start a business by offering “entrepreneurship as a service,” while allowing Zappos to scale faster. This would move its model even further away from the typical bureaucratic command-and-control structure; it would incorporate people that are not directly managed by the company.
Zappos acknowledges that it does not yet have firm data to prove that its model is superior to bureaucratic alternatives. However, it does see encouraging signs that this model could in the long run deal much more effectively in dynamic business environments. As Hsieh says, “Typically, reorganizing is a long process that is fraught with failure. In contrast, our organization chart changes multiple times a day—that’s our new normal. This allows us to respond to changes incredibly quickly.” Zappos is still experimenting at a rapid clip, and it hopes to eventually develop a full-fledged, market-based organizational philosophy.
Zappos’ experimentation teaches us valuable lessons about the power—and the limits—of current alternative organizational forms. It is a clear demonstration that the tenets of bureaucracy are by no means the only way to operate a successful business, and even quite radical departures from the bureaucratic model can deliver commercial success. It also shows that the process to develop an alternative is by no means rapid or straightforward; Zappos has experimented with alternative models for half a decade and still hasn’t achieved its full ambition.
Toward a New Organizational Paradigm
So is bureaucracy’s dominance here to stay, in spite of its frequently heralded demise? We believe not, as further changes in the business environment will likely push it past its breaking point. Uncertainty and dynamism continue to rise, which puts an even higher priority on learning, resilience, adaptiveness, and innovation. In addition, AI’s rapidly increasing power presents new challenges, because organizational models must encompass both human and algorithmic decision making. Technology also expands the range of possible organizational solutions, because algorithmic decision making allows activities to be coordinated on much shorter timescales and across a much wider set of participants (within and beyond the company).
Leaders must therefore reconceive the organization—from a static hierarchy and rule set to a continuously evolving model; from being human-centric to encompassing both humans and algorithms; and from being contained within company boundaries to encompassing connections and activities with external partners. They also need to reframe how their enterprises compete—from serving a relatively standardized and static set of offerings more efficiently to competing on the rate of learning, in order to discover and act on new opportunities. For that they will need a new generation of organizational models, which we collectively call the “hybrid learning organization.”
Although we don’t have a blueprint for the hybrid learning organization, we can already identify several emerging design principles:
Perhaps not surprisingly, the first glimpses of organizations that embody such principles can be found in some technologically advanced companies. Some of the new ways of working can be seen at Zappos, as described earlier. Alibaba has developed the “self-tuning enterprise” concept, in which algorithmic learning principles are applied throughout the organization. Its decision engines autonomously experiment, modulate, and improve over time, relying on information from the company’s vast data ecosystems, and decentralized teams are free to trigger new initiatives when they see market potential. Whereas most organizations have a static vision and organizational structure, Alibaba continuously evolves its vision and organization design. Amazon (Zappos’ parent company) demonstrates similar elements, such as data systems that are interconnected throughout the organization, allowing all parts of the company to react to new information without guidance from a centralized authority. Employees often take a “hands off the wheel” approach, validating algorithmic decision processes and setting guardrails rather than intervening directly, which allows them to focus on more creative efforts such as imagining new businesses and business models.
The challenges facing businesses today are evolving rapidly, yet most firms are still organized along the entrenched 19th century paradigm of bureaucracy. Recognizing the new imperatives they face, winning organizations will embrace bureaucracy where useful but also boldly experiment with new organizational models that harness both technology and human ingenuity where needed. The exact shape of these new models is still undetermined, but enterprising leaders are currently developing them. In some of the companies described in this article we can see hints of a revolution that will likely affect all companies and sectors eventually.
The authors thank Tony Hsieh and his colleagues at Zappos for allowing them to closely observe and discuss the market-based dynamics model. The views in this article are the authors’ own.
The BCG Henderson Institute is the Boston Consulting Group’s strategy think tank, dedicated to exploring and developing valuable new insights from business, technology, and science by embracing the powerful technology of ideas. The Institute engages leaders in provocative discussion and experimentation to expand the boundaries of business theory and practice and to translate innovative ideas from within and beyond business. For more ideas and inspiration follow us on LinkedIn and Twitter: @BCGHenderson.