You Don’t Have to Start from Scratch in a Data Transformation

Related Expertise: Digital, Technology, and Data, Digital Transformation, Data and Analytics

You Don’t Have to Start from Scratch in a Data Transformation

By Andrew Arcuri and Evan Stubbs

Companies looking to transform into agile, data-driven organizations often discover a double agent in their midst: their own technology. Over the years, many IT landscapes have evolved in ways that actually hinder—or outright undermine—a move to an integrated, scalable data architecture. The main culprits: fragmented technologies, legacy systems that don’t play well with others, and data that is isolated within applications. The IT you implemented to spark growth now hampers your transformation.

Wiping the slate clean is a startup’s game. Established companies don’t have the luxury of waiting until an all-new infrastructure is ready to go. That can take years—and rack up costs while competitors rack up market share. So what’s the way forward?

It starts with the first commandment of transformations: make sure they’re cost effective, incremental, and sustainable.  That means thinking about how you can unlock data that’s currently confined to silos and identifying new ways to create value from that data. And it means defining a blueprint that allows you to steadily develop your new data and digital architectures as you go along.

The idea: be ambitious but don’t try to realize your goal in one “big bang.” Do it through use cases that, one by one, build capabilities, business value, and buy-in for the transformation.

A wide range of organizations—not just B2C and B2B companies, but also governments and other public sector players—can benefit from this approach. Careful planning is a must. Without it, technology can fragment even further, with different parts of the organization doing more of their own thing. But by building your strategy around a few key principles, you can unleash your data transformation much faster than you think.

Unexpected Pitfalls

Digital initiatives—and, more broadly, the “digital-first” strategies so many organizations are putting in place—live and breathe data. The better you are at turning data into insights, the better you become at understanding, and meeting, customers’ or users’ needs. But if your organization has been around a while, there’s a good chance your data isn’t very fluid. It’s likely stored within specific applications, corralled by controlling departments, or constrained by ineffective software that requires you to jump through hoops—or find workarounds—to get to the data, limiting the value you can generate from it.

Digital natives also suffer these problems; the difference is that their data and digital architectures are designed to flex to overcome them. These companies constantly improve their IT architectures to enable quick, efficient, and multi-purpose use of data.  But for everyone else, there’s no digital transformation without a data transformation.

This notion of a “transformation within a transformation” might conjure images of outsize to-do lists—and outsize effort and stress. But we’ve found there’s a savvy, structured way to pursue a data transformation—one that’s already creating success stories. It consists of three phases:

  • Pursue quick wins to fund the journey. Avoid the boil-the-ocean approach, where companies throw everything into a data lake.  Instead, focus on high-impact—but not overly complex—use cases that can rapidly, and noticeably, generate value from data. These quick wins build your capabilities and experience, foster support for follow-on efforts, and help fund the larger transformation.
  • Design a company-wide roadmap. Leverage the lessons from your first initiatives to create an outline for where—and how—a data transformation will change your organization. By showing how value can be generated across the company, you communicate the vision for the transformation and build support for the necessary changes.
  • Organize for sustained performance. Keep the momentum going by introducing an operating model that encourages agile ways of working and creates a data-driven culture. Embed new behaviors and mindsets with careful change management.

Yet we’ve also seen more than a few companies miss one important step: scrutinizing their technology platforms along the way.  This might seem intuitive, but in practice, the pitfalls can be hard to see. Changes that seem to further data transformation (including changes that address the three points just listed) can actually create problems, such as an overly complex architecture or cumbersome customer experiences.

Consider the experiences of two companies. One is a high-growth Asian bank that long struggled with managing sales commissions. Invariably, different channels would insist that their action should be credited with the revenue uplift. To simplify commission allocations and reduce tension, the bank realigned roles to distinguish among lead identification, lead nurturing, and lead conversion. Accordingly, the bank redesigned its CRM systems to account for these three roles. But by doing so, it unintentionally complicated the user experience. Loan seekers now had to talk to at least three people before they could borrow money. This slowed the company’s transformation into a digital bank.

The second company, a global retailer and services provider, was looking to accelerate growth in its digital business. It created distinct teams for digital marketing, assisted channels, and loyalty—and empowered each to move fast. But because each team could choose its own technology, platforms overlapped and initiatives and campaigns were largely uncoordinated (one frequent consequence: customers would receive offers they had already rejected). To avoid marketing opt-outs and a negative impact to its net promoter score, the company had to do extensive manual data collection and reconciliation. That reduced its productivity and agility.

Let the Customer Journey Lead the Way

But what does “scrutinizing technology platforms” entail? How do companies ensure that their data architecture supports, instead of hinders, their digital strategy and goals? CEOs and CIOs should follow three key principles:

  • Design around customer journeys, not internal processes. In the Asian bank example, the company followed what appeared to be a reasonable course for simplifying a troublesome internal process. Yet the solution actually complicated things for customers. Instead, companies should flip their approach: let the customer journey dictate the internal process. No doubt, this will be a big change for many companies, since legacy platforms, such as CRM systems, are usually built to support the requirements of internal users, not customers. But as roles and responsibilities evolve, fragmentation tends to increase, leading to inconsistent, prolonged, or cumbersome customer experiences (and, often, all three). Put simply: if you’re going to be customer centric in the way you do business, you need to be customer centric in the way you design your IT architecture.

  • Treat data as a discrete asset. Data needs to be portable—decoupled from, instead of restricted to, specific applications. The idea is to create pools of shared data (where users and applications across the organization can leverage the same information) instead of data islands (where “hoarded” data limits insights—and value creation—to a small sliver of what might be possible). Treating data as a discrete asset requires a lot of legwork. You need to pay careful attention to data governance (to ensure consistency, accuracy, and other dimensions of data quality) and to how you can technically open up access to data (for example, through APIs or microservices).  But the effort is no longer optional. Whether you want to understand customer journeys, manage end-to-end logistics, or ensure you know the opportunities and risks, you need to treat data as a resource in its own right—not just the byproduct of a process.
  • Align ways of working between technology and business functions. Data science doesn’t work—at least, not well—in a vacuum. You need to bring together a range of technical expertise in areas like software engineering, AI, automation, and architecture. At the same time, you need to take a commercial mindset, using data in ways that support and spark revenue generation. Many company cultures don’t foster efficient, sustainable collaboration across technology and business functions. But there are proven ways to remedy that. Agile methodologies help you build—and unleash—integrated teams. And next-generation practices like DevOps, InnerSource, and continuous integration and continuous delivery boost speed, automation, and scalability.  With such new approaches, you can develop and deploy your data use cases more quickly.

A Data Architecture That Lets You Evolve Fast

Keeping those principles in mind, consider how you might build an architectural framework that facilitates rapid, ongoing change—for example, deploying new customer experiences at digital speed.

A good starting point: reimagining journeys in light of customers’ needs (whether they be consumers, other businesses, or users of government services). Your IT stack is the bedrock—the chief enabler—of digitization. Your technology, therefore, must be designed around the “customer first” principle. The idea is to rationalize the core—the systems that support your business but don’t provide any real competitive advantage (think general ledger and payroll processes here)—while steadily adding best-of-breed applications that directly touch the customer and can spark differentiation (think personalization or incidence management).

What does this process look like? Savvy companies will work along two parallel tracks. On one side, they’ll simplify their core down to a lean, vanilla state, with just enough there to handle HR, finance, supply chain management, and other essential back-office processes. They won’t spend much time on the core, and they won’t get fancy, opting instead for proven solutions that don’t require much, if any, customization. But as they simplify the core, these companies will also start to generate value from commercial use cases. This is the second track: identifying, prioritizing, and implementing applications and experiences that resonate with customers—and in turn differentiate the company—because they’re designed around the customer.

This customer-centric approach means looking for points in existing journeys where the customer experience is clunky or complicated or just doesn’t work. You’ll want to prioritize the fixes. At the same time, be pragmatic: you might have legacy systems that you can’t easily pull out of the core or economically reengineer. So for the near-term, at least, you might be straddling two worlds—and need to make any necessary connections. (See “A Peaceful Coexistence.)


Uprooting legacy systems isn’t always easy or fast. So even as companies transform, they need a way to link the old with the new.

One approach is to put a “wrapper” around the old technology. Do this by deploying middleware that, on one end, connects to your digital application in an entirely modern fashion, but on the other end, connects to your legacy system using old-school hooks. That way you develop your best-of-breed applications without worrying about older communications protocols. The middleware does all the grunt work, at a fraction—perhaps 5%—of what you might pay to upgrade the relevant part of the core.

Another solution is to place the new platform under the purview of a separate entity (whether an internal organization or an external provider). Instead of trying to integrate legacy systems with an all-new technology stack, you decouple them, run them separately, and work out how to move only those bits of data the platforms need to share. This approach can be compelling when data-sharing needs are relatively limited and the old and new worlds largely function independently.

You’ll also want to ensure that you link investments—whether in developing customer experiences or simplifying the core—to clear benefits, such as cost reduction or revenue generation. By specifying milestones and expected results, senior leaders can maintain control and line of sight over the transformation, taking swift remedial action when something looks off.

All of this means delivering your transformation through incremental changes—not a big bang—that build out the target architecture while making a visible business impact. Along the way, new ways of working and new tools and practices help you collaborate, move fast, and decouple data—so that you can rapidly turn the reimagined journeys into integrated and relevant experiences.

To be sure, data transformation requires bold moves. But the use-case approach is a way to start the ball rolling now, instead of waiting years for a completely new architecture to come online. Best yet, it sustains itself as you gradually build the expertise, support, and funding you need to continue toward—and smartly evolve—your digital vision.