Partner and Vice President, Deep Tech
Software is eating the world, as Marc Andreessen famously put it more than a decade ago. Now, generative AI (GenAI) is transforming software development—and, by extension, how big companies are run today.
Software and data are expressed in languages, and GenAI is very good at translation. This makes the tech function a natural place for early application.
GenAI’s impact will be felt in multiple areas. It will improve the quality and speed of software developers as it enables business users to develop prototypes of what they want. GenAI vastly expands access to data because it “understands” data from a wide array of sources and automates data cleansing, structuring, and management. Software implementation costs will drop because GenAI can update legacy code at low expense, enabling full migration to the cloud and end-to-end processes automation. And the speed with which GenAI is beginning to make itself felt is unprecedented.
Company tech functions are already leveraging off-the-shelf coder augmentation solutions, such as GitHub Copilot (Microsoft), CodeWhisperer (Amazon), and Codey (Google), to boost their efficiency and creativity. Our research estimates that these tools enable software development productivity gains of 30% to 50%. Altogether, efficiency gains could reach 10% of the IT cost baseline—up to 65% of costs can be directly affected—translating into hundreds of millions of dollars for most Fortune 500 companies. (See Exhibit 1.)
GenAI presents CIOs with four new responsibilities:
Big tech companies have long deployed their innovations internally first. This article, which kicks off a series on how CIOs and their companies can tap the power of GenAI, focuses on how companies outside the tech sector can deploy GenAI for the tech function.
AI’s adoption has been far from even, but because of its plain-speaking capabilities, GenAI is a game-changer with many use cases along the tech value chain. Large language models (LLMs), such as GPT-4, show impressive results when trained on vast quantities of data. IT makes AI’s job even easier because it relies on standardized and structured documents and artefacts, such as code, specifications, data models, tests, operations knowledge bases, user tickets, build scripts, and system logs. Commercial tools are available for myriad use cases that fall into four areas, helping to optimize IT processes and workflows end to end. (See Exhibit 2.)
Partnering with the Business. GenAI tools may form the basis of a new win-win relationship between IT and the business. Prototyping applications in a few days can lead to better specifications, faster iterations, and stronger business adoption. GenAI will accelerate the “no code” movement that is already underway in many companies. All employees—not just engineers—will be empowered to build business dashboards, generate insights from data sets, and design their own workflows just by typing what is needed into a chatbot supported by an LLM.
Building and Maintaining Apps. All steps of the software development lifecycle can be augmented with commercial tools available today. For example, one software company saw developers’ completed tasks jump more than 40% in the three months after implementing an AI productivity tool. (See “Augmenting Software Development.”)
IT Operations. Basic operations are typically responsible for more than half of IT costs. Many processes can be automated or managed more efficiently. Consider the following estimates, which are based on BCG research and client experience:
Enhancing Data. Multiple GenAI use cases can accelerate data transformation. For example, GenAI can help automate data cleansing, consolidation, augmentation, and integration; it can generate synthetic data; and it can automate data labeling and classification, data anonymization, and data management. (See Exhibit 3.)
Here is a six-step plan that, based on our experience, can realize the opportunities offered by GenAI for IT. (See Exhibit 4 and Exhibit 5.)
Formulate Your GenAI Vision. The first step is to define where GenAI can have early impact and how the impact can scale and spread, based on the technology’s potential and the company’s ambitions and starting point. This may require experimentation and learning by doing, but is important to think in terms of end-to-end processes and tools (such as redesigning software development and maintenance process) rather than optimizing steps separately (for example, only coding in current development languages). Management should be prepared to address fears (job loss) and constraints (legacy technology, high share of commercial off-the-shelf software packages) and explain how gains will be reallocated. As the IT function gains experience, the vision can be translated into a use case roadmap based on value and feasibility.
Fund the Journey, Establish the Prerequisites. GenAI represents a major IT transformation, which requires funding for the necessary changes in tooling, processes, and talent. A good rule of thumb for planning is that one to two dollars of funding are required to generate each dollar of annual saving. Sufficient companywide technical maturity also must be achieved in several areas prior to deployment, including a centralized code base, a well-documented configuration management database and knowledge base, and a lean software and DevOps toolchain. Since most of the tools used will be off-the-shelf solutions, the bulk of the investment will go toward people and skills, and operating model costs will be higher. Skills building and change management should receive adequate budget allocations.
Exercise Your GenAI Muscles. GenAI will gain traction by experimenting on the most feasible and high-value use cases. Tech function KPIs, such as lead time, deployment frequency, software quality, and developer satisfaction, should be monitored before and after to document efficiency saving. Derive and share the key lessons from each experiment to improve future processes.
Reshape Sourcing Strategies. External service providers will be deploying AI for themselves to improve their cost structures. Seek to share in the savings. Renegotiate contracts as soon as they come up for renewal looking to cut fees and costs. Over time, GenAI may offer an opportunity for CIOs to reduce their dependence on third parties as the value shifts to the design and architecture stages of product lifecycle, which can be beefed up internally. Companies that have already internalized resources as part of their digital transformations stand to gain more from GenAI in the near term.
Anticipate Workforce Implications. Organizations will need to adapt their operating models to accommodate the new skills, roles, and ways of working. Strategic workforce planning must go along with change management and “upskilling” campaigns.
Mitigate Legal and Ethical Risks with Responsible AI. GenAI risks are easier to measure and control in IT than in other functions, but they should not be left to chance. For code generation, most commercial tools offer the option to use models trained on code that carries explicit authorization for training purposes and does not include open-source code. Helpdesk chatbots may be susceptible to “hallucinations” (falsehoods and fabrications) and bias from their training data sets. Most commercial tools are based in the cloud and require uploading assets such as code, tests, and tickets. Even enterprise tiers with enhanced data protection can encounter restrictions from company-specific data privacy, regulatory, or compliances policies. To mitigate these risks, IT will need independent tools and processes to measure quality, service level agreements, security risks, and user satisfaction.
GenAI’s disruption in IT will likely be on a scale similar to that of the internet. Indeed, companies that apply GenAI to rein in IT overspending, along with taking other steps (such as refactoring legacy architecture, moving to the cloud, internalizing software as a service, and automating mobile device management and operations), could shift their IT budgets from an average 70%/30% run-change ratio to 50%/50%, ushering in a new era for IT.
The sheer number of potential GenAI use cases, the extent of the productivity gains, and the extension of new tools to business functions mean that the potential benefits will likely far outweigh the investments required. Forward-looking CIOs who put GenAI capabilities to work as soon as possible will be well positioned to gain the necessary knowledge and experience and reap the business value.