Augmented reality (AR)—a technology with enormous promise—is emerging as the key that will unlock the full potential of the Internet of Things (IoT). AR applications are using the myriad kinds of data generated by IoT devices and components to help employees be more effective and productive.

To quickly kick off a virtuous cycle of cost savings, revenue increases, and higher profits that can prompt a double-digit increase in top and bottom lines, companies should develop a joint IoT-AR strategy, rather than think about these technologies piecemeal. This report includes a downloadable IoT-AR app that illustrates how companies can capture these benefits.

Why AR Goes Hand in Hand with IoT

Almost unnoticed, the number of IoT-connected devices rose from about 6 billion in 2016 to 11 billion by 2018, possibly reaching 20 billion in 2020, according to Gartner. That increase has resulted in a veritable Big Bang of data. Cisco Systems and IDC estimated the volume of IoT-generated data to be about 22 zettabytes (or 22 trillion gigabytes) in 2016. That volume more than doubled to 52 zettabytes by 2019, and it is projected to reach 85 zettabytes by 2021.

However, data volumes are rising so quickly that many corporations are unable to use all the data they collect. Huge volumes result in slow processing times, and as a result, the data becomes stuck in data lakes. Moreover, the data sets lack a coherent sequence, and the databases have too many sources to synthesize. Consequently, many companies are finding it difficult to generate data-based insights that employees can use at the right time and in the right place.

The other, oft-overlooked challenge to effectively using big data is the standard visualization tools; they aren’t always effective at depicting data from many sources. Yet, most people need to see data in a visual form to understand it quickly.

This is where AR comes in. IoT devices capture data from the physical world so that it can be analyzed, and AR devices take that digital data and render it back on the physical world for people to view and interact with. (See “The Sweet Spots for IoT-AR Solutions.”)

Unlike virtual reality applications, which require the user to inhabit a virtual environment, AR applications overlay digital information on top of the user’s physical environment. For example, an AR application could add digital content to live camera feeds, making the content appear as if it is part of the physical world. That capability would allow users to interact with data more intuitively.

Several companies have developed AR applications that use artificial intelligence to distill vast amounts and various types of data (including IoT data) into a few critical actions, which are then presented to employees at the right time and place so that they can make informed decisions and better execute tasks. In many industries, companies that aren’t experimenting with joint IoT-AR strategies risk being left behind by rivals.

IoT and AR Usage Is Rising Rapidly

Using AR for business is becoming more widespread: increasingly, companies are expanding the technology’s use beyond the initial testing grounds of marketing and sales. (See Augmented Reality: Is the Camera the Next Big Thing in Advertising, BCG report, April 2018.) This expansion includes other functions, particularly manufacturing, operations, service, and training. (See “Rolling Out Augmented Reality in the Field,” BCG article, October 2018.) The proliferation of smartphones and tablets that can handle AR, rather than the development of dedicated devices, has prompted AR’s expansion.

To identify the prevailing trends, BCG and PTC conducted a quantitative survey of more than 200 executives from companies using IoT or AR solutions, or both. We also interviewed several senior executives. Strikingly, many see IoT and AR as complementary. Of the companies surveyed, 81% are using IoT and also exploring the possibility of using AR, while about 76% of those developing AR-only solutions believe that adding IoT to their applications would have value.

Using the combination of IoT and AR is a recent trend. Although a few companies have been experimenting with IoT-AR solutions since 2010, the number doing so has risen sharply since 2017. Moreover, more than 80% of the companies surveyed expect IoT-AR solutions to become the norm in their industry in the next five years. The survey highlighted other important trends in five areas:

• Industry. Several industries are pioneering the use of IoT and AR technologies at the same time. Heavy industries are showing the way, followed by technology, engineering, aerospace, telecommunications, and media. In functions such as equipment maintenance, we found that IoT and AR applications already represent the state of the art.
• Design. According to our survey, 47% of IoT-AR experiments started with the use of IoT; companies added AR later. However, in interviews, senior executives noted that when their companies developed solutions that used a combination of IoT and AR technologies from the outset, it was easier to deploy the technologies, and they captured more value.
• Devices. In the companies we surveyed, many users of IoT-AR applications use more than one AR device. The smartphone is the most popular (51%), followed by Microsoft HoloLens (39%), Google Glass (18%), custom devices (17%), head-mounted displays (16%), RealWear wearables (11%), and the Magic Leap display (9%).
• Organization Structure. Of the companies we surveyed, 85% already manage the development and deployment of IoT and AR applications together, and of those companies, 77% have dedicated AR budgets.
• Payback. About 80% of the companies surveyed said that they expected a payback from their investments in IoT and AR within three years, and 26% recouped their investment in one year or less. In fact, half the companies that have implemented the two technologies—or are in the late stages of testing them—have already demonstrated their value internally, while about 35% are on track to do so.

IoT and AR at Work

Our survey and interviews identified more than 30 IoT-AR use cases. When we studied them, we found that they could be meaningfully analyzed by considering how they augment human efforts and impact business. (See Exhibit 1.)

IoT and AR applications can help people in three ways:

• Visualizing Data and Interacting with the Environment. IoT-AR solutions help employees visualize, analyze, and understand data, which, in turn, enables them to better navigate their environments. People can understand data more intuitively by viewing it in three dimensions. Providing employees with relevant data or knowledge when they are solving problems is the most effective way to help them work, according to the learning sciences. Of the IoT-AR use cases that enable people to visualize data and interact with their environments, 20% help workers manage spaces.

• Diagnosing Problems. IoT-AR applications combine various types of data into a single view, helping employees better analyze objects and spaces as well as diagnose problems in physical objects and their surroundings. For example, these applications help workers identify problems with complex machinery without having to open it and locate a part without going through every rack in a shelf. Of the IoT-AR use cases that enable employees to better diagnose issues, 22% help workers manage equipment or objects.
• Taking Action. IoT-AR solutions help employees make smarter decisions by combining a high volume and large variety of context-specific data into a single view. IoT-AR applications can also connect employees with remotely located experts who provide real-time guidance. Of the IoT-AR use cases that help employees take action, 15% enhance human capabilities.

These three ways build on each other: To diagnose a problem, employees must visualize data and interact with their environments. And to take action, they need to diagnose the problem.

IoT and AR technologies also impact business in three areas:

• Enhancing Human Capabilities. Companies can use IoT-AR solutions to help employees learn how to perform complex tasks more easily. For example, by simulating products and overlaying IoT data on them, an IoT-AR application could help companies train plant employees to assemble equipment or use machinery. Similarly, hospitals could use these solutions to train health care technicians.
• Managing Spaces. IoT and AR technologies can help companies make sense of disaggregated data collected from physical spaces, such as factories, warehouses, and retail stores. Understanding disaggregated data is critical to better managing functional areas. For example, an IoT-AR application could help a company optimize the use of a warehouse by determining the amount of space to allocate to high-turnover products, monitoring inventory health, and identifying aging stock to move out. The application could also use IoT data and AR devices to guide workers through the warehouse using the most efficient route. Implementing such an application could reduce the amount of working capital needed, lower energy costs, and improve labor productivity. Similarly, companies could use AR to overlay foot traffic data—collected using beacons, mobile apps, and video analytics—on retail store configurations to identify the optimal store setup. In addition, AR headsets could use live IoT data from in-store sensors to enable managers to walk through a store’s aisles and spot problems. Alerts could pop up when products are out of stock, in the wrong place, or incorrectly barcoded, for instance.

• Managing Equipment. Companies can help employees fine-tune complex equipment and make repairs by creating an AR application that superimposes IoT data on a visual representation of a machine’s components. Such a representation enables workers to “look” inside the machine at even hard-to-see areas so that they can diagnose issues more easily. AR enables the application to overlay step-by-step instructions on the image as well as display real-time guidance from remotely located experts while workers are making a repair. In some cases, the next repair instruction may be provided to a technician only after the current step has been digitally confirmed as completed, increasing the quality of the repair. On shop floors, workers can wear AR devices that help them visualize IoT data in the context of their work. For example, when IoT sensors are embedded in machinery on a production line, technicians wearing AR glasses can walk the floor to get insights into how equipment is functioning. The closer a technician is to a piece of equipment, the more specific the information will be. IoT-AR applications also make it easier to spot issues across assembly lines. When designing equipment, companies can feed IoT data into digital models and use AR to interact with those models at scale, giving designers a dramatically more accurate representation of how a design will perform in the real world. For instance, the representation may help designers locate the most commonly used features so as to improve the design and performance of the machine. Similarly, remote IoT devices—such as cameras and temperature sensors that are installed in and around structures—enable data collection that can help companies create digital twins. When such virtual models are paired with AR devices, architects, contractors, and builders can interact with data to provide better inputs, and, among other things, improve the placement of plumbing and electricity.

Boosting Performance with IoT and AR

Deploying IoT and AR is one of the quickest ways that companies can boost the bottom line. IoT-AR solutions can help companies improve performance in three ways, often at the same time.

• Reducing Costs. Companies can decrease costs by using IoT and AR technologies to improve labor productivity, reduce material waste, shrink asset downtime, and optimize working capital—and that’s just for starters. Even a simple application—such as an IoT-AR solution that renders real-time, context-specific data to assist workers tackle equipment repair—can increase first-time fix rates and decrease errors, both of which will reduce the cost of materials and eliminate repeat visits by technicians.
• Boosting Revenue. Companies can deploy IoT-AR solutions to increase the speed of operations, and, thereby, business output. Companies can also use these technologies to develop new products and services that they may be able to monetize. For instance, creating a premium maintenance service built on IoT and AR could help a company retain customers, reduce churn, and increase the number of customers that sign up for after-sales services.
• Creating Strategic Value. The added value from an IoT-AR solution can accrue in many ways, including higher brand equity, increased customer satisfaction, and mitigated risk. For instance, using AR to guide workers through a warehouse can boost safety, which improves the work environment. 

An IoT-AR Application

To highlight how IoT-AR solutions can generate significant financial returns, we investigated two use cases and developed an IoT-AR app to bring them to life. The app demonstrates how these technologies can help reduce equipment maintenance and warehouse costs.

Reducing Costs and Increasing Revenue from Equipment Maintenance Using IoT and AR. After-sales services are a lucrative source of revenue for industrial manufacturers because most customers sign service contracts for the maintenance and management of their assets. However, manufacturers are facing increasing pressure from customers to provide not only better service at a lower cost but also faster response times. In addition, many manufacturers must deal with a growing skills gap. Baby boomers are leaving the workforce, and companies are losing experience and expertise en masse. Against this backdrop, using an IoT-AR solution for equipment maintenance can address multiple pain points:

• Unexpected Downtime. IoT technologies can monitor equipment constantly to provide early warning signals about the functioning of machines and production lines. IoT-AR solutions use IoT real-time performance data, advanced analytics, and AR technologies to deliver such warnings to staff, enabling them to schedule preventive maintenance.
• Project Delays and Higher Costs. IoT data and analytics can determine when equipment downtime will be less disruptive and which parts will be needed for service or repairs. Such guidance helps companies avoid project delays and higher costs.
• Long Wait Times for Repairs. Field service engineers usually must visit equipment sites to make repairs, extending downtime. However, remote support through AR, along with access to IoT dashboard data, can help less experienced engineers who are on site tackle some issues that they otherwise would not have been able to handle.
• More Training Time. Experienced employees are leaving the workforce. As a result, companies must allocate more time to train staff and new hires on complex tasks. AR devices that give clear step-by-step instructions superimposed on physical objects can provide employees with the necessary guidance and on-the-job training.
• Poor Service Quality. IoT and AR technologies can confirm whether equipment has been repaired or serviced properly, which, along with AR-guided work instructions, will improve service quality.

Several companies—such as Global Foundries, Howden Group, Sysmex, and Volvo Group—have deployed IoT-AR solutions to tackle maintenance issues, provide customers with self-service options, and improve quality. For these OEMs, IoT-AR solutions are transforming the speed and quality of service delivery, increasing customer satisfaction, and reducing consumer churn.

By implementing IoT-AR solutions, companies have realized sizable economic benefits:

• Increase in Labor Productivity. Providing AR-guided instructions reduces the time it takes to make repairs, increasing labor productivity. Because remote support reduces the need for experts to travel, each specialist can respond to more service calls.
• Decrease in Material Waste and Rework. AR-based instructions, together with IoT verification, can ensure that repairs are completed the first time, eliminating the need for rework and the reordering of parts.
• Lower Costs. IoT-AR training solutions reduce labor costs by enabling less capable engineers to conduct repairs. These applications also lower training costs by providing on-the-job training on demand.
• Lower Churn. IoT-AR solutions can increase how quickly companies respond to customers and improve service quality. In turn, companies can increase their renewal rate of lucrative after-sales service contracts and win new opportunities.
• New Offerings. Developing novel, value-added IoT-AR services and capabilities increases the opportunities for manufacturers to generate revenue from existing customers. Such services and capabilities also help build companies’ brand equity.

To show how a solution that pairs IoT with AR can lower costs and boost revenues, we developed a case study using real-world data. In our study, a manufacturer with annual after-sales service revenue of $1 billion, or 30% of its annual revenue, employs 1,000 service engineers.

By deploying an IoT-AR solution with 20% to 30% of its service engineers, we found that the manufacturer has the potential to generate $13 million to $30 million of net benefit for every $1 billion of revenue. We have often seen companies achieve results toward the higher end of this range. (See Exhibit 2.)

According to our calculations, the upside potential could be even larger if the company rolls out the solution to all its service engineers. The benefits would also extend beyond the manufacturer, providing significant cost savings for its customers in terms of lower asset downtimes and higher asset performance.

To be sure, companies could use other options instead of an IoT-AR solution. An alternative would be to match engineers’ skills to the right tasks, thereby reducing errors and increasing the first-time fix rate. A pure human solution is likely to cost less, but it won’t provide all the benefits that an IoT-AR solution can deliver. For instance, the former won’t reduce the travel that experts undertake, provide training benefits, or help new technicians get up to speed faster.

To bring the case study to life, BCG and PTC developed IoT+AR, an app that enables users to experience an equipment maintenance solution in 3D. The app can be downloaded for iOS and Android devices. The app puts users in the shoes of a technician to experience how IoT and AR can guide them at every step of an excavator repair.

Shrinking Warehouse Costs Using IoT and AR. The second case study we developed focused on order picking. In mammoth warehouses, selecting products for shipment or delivery is a slow process because workers need to navigate vast spaces. In most cases, a product’s location is noted only at the entrance and exit, so it’s difficult for workers in the warehouse to know exactly where products are. Employees spend a lot of time searching for products, taking suboptimal routes to find what they’re looking for. Moreover, lifting bulky products while holding a clipboard or a checklist often leads to accidents and damaged goods, as does performing a job without clear instructions.

An IoT-AR solution can increase the speed of order-picking operations and boost productivity. Several companies (such as DHL, Intel, and Boeing) have deployed IoT sensors throughout their warehouses, and head-mounted AR devices guide workers to products’ locations by superimposing instructions on users’ fields of vision. These solutions help employees take the shortest routes to the exact locations of products. When workers find the products, their headset camera scans them and checks them off the list, freeing the employees to handle the products.

Companies can benefit from IoT-AR warehouse solutions in several ways:

• Higher Labor Productivity. IoT-AR applications that provide the optimum route through the warehouse save time and reduce the number of employees that are needed to perform order-picking activities.
• Reduced Waste. AR-guided instructions reduce the chance of goods getting damaged, saving insurance costs and material waste.
• Faster Throughput. Improved operating speeds enabled by IoT-AR solutions ease traditional bottlenecks.
• Lower Risk. AR-guided navigation improves safety and reduces accidents, making working conditions in warehouses better.
• Burnished Brands. IoT location tracking enables companies to track the location of products, especially high-value products, at all times. Providing higher-touch services that are supported by faster processing speeds can unlock new business opportunities and enhance corporate reputations.

This case study focuses on a logistics company that manages the transportation of large volumes of products. It works on tight timelines and has several warehouses all over the world. Most product picking is done manually by the company’s workforce of about 90,000 employees.

According to our estimates, a company of this kind with $20 billion in revenue would spend about $3 billion in order-picking labor costs a year. With an initial IoT-AR deployment covering one-third of the workforce, the company could generate a net benefit of $3.5 million to $7 million a year per $1 billion of revenue. Those benefits would only rise over time as the solution rolls out across the rest of the workforce. (See Exhibit 3.)

An IoT-AR solution is not the only way to tackle order-picking and package-sorting issues in warehouses, of course. Another option may be to use robots. Although generally more expensive, robots can work well in industries with large warehouses, small homogenous products, and higher margins. Companies should experiment with different solutions to identify the technologies that will work best for them.

BCG and PTC’s IoT+AR app brings this to life. It enables users to experience product picking using IoT and AR. The app can be downloaded for iOS and Android devices. The app puts users in the shoes of a warehouse employee to see how IoT and AR can help employees find products quickly and safely.

Using IoT and AR in Health Care

IoT-AR solutions aren’t only about increasing revenue or reducing costs; they can deliver societal benefits as well. For example, using the two technologies together may prove to be a game-changer in health care applications. Specifically, minimally invasive surgeries account for about 35% of all medical procedures. During such a surgery, a doctor makes a small incision in the patient’s body and inserts tiny cameras and miniature instruments so as to navigate the patient’s body without having to make large cuts and incisions.

Although the techniques of keyhole surgery, as it is often called, have been honed over the past 20 years, surgeons still face several challenges. They cannot get all the insights needed solely from the 2D video streams provided by the inserted cameras. Moreover, surgeons must continually switch their focus among several sources of information, including video screens, biosignal monitors, the attending surgical team, and, of course, the patient.

By contrast, an IoT-AR solution would enable a surgeon wearing an AR headset to see a 3D model from diagnostic imaging that is superimposed on the patient; the model would be aligned with the patient’s anatomy with the help of high-precision positional trackers placed on and inside the patient. The surgeon could rotate, zoom in and out, and interact with the 3D model using simple hand gestures. The patient’s vitals and other biosignals, as well as instructions (such as the best angle and depth for a cut), could be overlaid in the surgeon’s view, eliminating the need to switch among the various factors that require attention. And the surgeon would be able to control the entire procedure through hand gestures, voice controls, and eye movements—without pressing buttons or looking away from the patient for even a second.

For surgeons and patients, there would be enormous benefits. Doctors would be able to perform surgeries with fewer errors and achieve higher first-time success rates. Patients would thus enjoy faster recovery times and a lower risk of complications. Several clinical trials are under way globally, and if they receive approval, IoT-AR surgical solutions could deliver enormous benefits to society.

Getting IoT and AR Right

IoT-AR solutions are at an inflection point, but they are likely to become table stakes soon in several industries. Companies must choose the right starting points—the largest cost centers and revenue opportunities—and identify where IoT-AR applications will have the highest value. It’s critical to scale such applications as quickly as possible across the enterprise not only to reap the financial benefits but also to ensure that the cultural change that is necessary for the new approach to succeed has time to take root. Although companies will face challenges, it’s possible to successfully harness these complex technologies. (See “Managing the Risks of Deploying IoT-AR Solutions.”)

Scope and Value. Companies should rank projects by the largest value opportunities, choosing to develop and deploy one or two IoT-AR applications that will deliver value—fast. Companies must develop business cases and articulate the value proposition of using the two technologies together.

The use of IoT technology is increasingly prevalent, so it’s important to ensure that AR has a value-adding role; AR shouldn’t be an afterthought or a gimmick. Our survey suggests that employees are five times more likely to regard AR as a core technology if their company develops an IoT-AR application from the start, rather than adding AR later.

Organization Structure and Partnerships. Cross-functional collaboration across traditional business units is critical for success. Companies must get stakeholders from all relevant groups—not only the core business units but also IT and R&D—to back the IoT-AR experiments. As we mentioned earlier, of the companies we surveyed, 85% already manage the development and deployment of IoT and AR together.

Because the development of an IoT-AR solution must be a business-level initiative, obtaining executive sponsorship is important. Having that backing from the start can ensure that these projects gain momentum quickly. Companies must be careful about choosing technology vendors, partners, and systems integrators for the IoT-AR ecosystem; they must have the ability to support the redesign of operational processes as well as the implementation of technological solutions.

Development and Launch. Companies must optimize the user experience and ensure that IoT-AR devices are comfortable and intuitive in the physical environments in which they will be used. Getting that right can drive adoption faster and ensure that employees routinely use the technologies. The best way to optimize the user experience is to encourage IoT-AR teams to develop minimum viable products, run pilots, conduct end-user tests, and iterate solutions quickly.

Combining IoT and AR technologies enhances the ways that people can interact with the world around them. But to harness that power, companies must choose where to deploy the technologies, build an ecosystem that supports IoT and AR, and develop the talent that can use both technologies. Companies that do so will be able to not only reduce costs, boost revenues, improve the customer experience, and strengthen brands but also use contextual data in ways never before imagined.