Augmented reality (AR)—a technology with enormous promise—is emerging as the key that will unlock the full potential of the Internet of Things (IoT). AR applications are using the myriad kinds of data generated by IoT devices and components to help employees be more effective and productive.
To quickly kick off a virtuous cycle of cost savings, revenue increases, and higher profits that can prompt a double-digit increase in top and bottom lines, companies should develop a joint IoT-AR strategy, rather than think about these technologies piecemeal. This report includes a downloadable IoT-AR app that illustrates how companies can capture these benefits.
Almost unnoticed, the number of IoT-connected devices rose from about 6 billion in 2016 to 11 billion by 2018, possibly reaching 20 billion in 2020, according to Gartner. That increase has resulted in a veritable Big Bang of data. Cisco Systems and IDC estimated the volume of IoT-generated data to be about 22 zettabytes (or 22 trillion gigabytes) in 2016. That volume more than doubled to 52 zettabytes by 2019, and it is projected to reach 85 zettabytes by 2021.
However, data volumes are rising so quickly that many corporations are unable to use all the data they collect. Huge volumes result in slow processing times, and as a result, the data becomes stuck in data lakes. Moreover, the data sets lack a coherent sequence, and the databases have too many sources to synthesize. Consequently, many companies are finding it difficult to generate data-based insights that employees can use at the right time and in the right place.
The other, oft-overlooked challenge to effectively using big data is the standard visualization tools; they aren’t always effective at depicting data from many sources. Yet, most people need to see data in a visual form to understand it quickly.
This is where AR comes in. IoT devices capture data from the physical world so that it can be analyzed, and AR devices take that digital data and render it back on the physical world for people to view and interact with. (See “The Sweet Spots for IoT-AR Solutions.”)
IoT-AR solutions are most useful in business situations where employees (rather than automated systems) make some or all of the decisions and where workers need digital data to more effectively interact with physical objects, such as machines and spaces.
IoT-AR applications are also very helpful when the aggregation of large volumes and various kinds of contextual data in a single view would facilitate decision making, when employees need both hands to effectively interact with objects or their environments in real time, and when employees need to interact with objects or parts that are not visible.
Unlike virtual reality applications, which require the user to inhabit a virtual environment, AR applications overlay digital information on top of the user’s physical environment. For example, an AR application could add digital content to live camera feeds, making the content appear as if it is part of the physical world. That capability would allow users to interact with data more intuitively.
Several companies have developed AR applications that use artificial intelligence to distill vast amounts and various types of data (including IoT data) into a few critical actions, which are then presented to employees at the right time and place so that they can make informed decisions and better execute tasks. In many industries, companies that aren’t experimenting with joint IoT-AR strategies risk being left behind by rivals.
Using AR for business is becoming more widespread: increasingly, companies are expanding the technology’s use beyond the initial testing grounds of marketing and sales. (See Augmented Reality: Is the Camera the Next Big Thing in Advertising, BCG report, April 2018.) This expansion includes other functions, particularly manufacturing, operations, service, and training. (See “Rolling Out Augmented Reality in the Field,” BCG article, October 2018.) The proliferation of smartphones and tablets that can handle AR, rather than the development of dedicated devices, has prompted AR’s expansion.
To identify the prevailing trends, BCG and PTC conducted a quantitative survey of more than 200 executives from companies using IoT or AR solutions, or both. We also interviewed several senior executives. Strikingly, many see IoT and AR as complementary. Of the companies surveyed, 81% are using IoT and also exploring the possibility of using AR, while about 76% of those developing AR-only solutions believe that adding IoT to their applications would have value.
Using the combination of IoT and AR is a recent trend. Although a few companies have been experimenting with IoT-AR solutions since 2010, the number doing so has risen sharply since 2017. Moreover, more than 80% of the companies surveyed expect IoT-AR solutions to become the norm in their industry in the next five years. The survey highlighted other important trends in five areas:
Our survey and interviews identified more than 30 IoT-AR use cases. When we studied them, we found that they could be meaningfully analyzed by considering how they augment human efforts and impact business. (See Exhibit 1.)
IoT and AR applications can help people in three ways:
These three ways build on each other: To diagnose a problem, employees must visualize data and interact with their environments. And to take action, they need to diagnose the problem.
IoT and AR technologies also impact business in three areas:
Managing Spaces. IoT and AR technologies can help companies make sense of disaggregated data collected from physical spaces, such as factories, warehouses, and retail stores. Understanding disaggregated data is critical to better managing functional areas. For example, an IoT-AR application could help a company optimize the use of a warehouse by determining the amount of space to allocate to high-turnover products, monitoring inventory health, and identifying aging stock to move out. The application could also use IoT data and AR devices to guide workers through the warehouse using the most efficient route. Implementing such an application could reduce the amount of working capital needed, lower energy costs, and improve labor productivity.
Similarly, companies could use AR to overlay foot traffic data—collected using beacons, mobile apps, and video analytics—on retail store configurations to identify the optimal store setup. In addition, AR headsets could use live IoT data from in-store sensors to enable managers to walk through a store’s aisles and spot problems. Alerts could pop up when products are out of stock, in the wrong place, or incorrectly barcoded, for instance.
Managing Equipment. Companies can help employees fine-tune complex equipment and make repairs by creating an AR application that superimposes IoT data on a visual representation of a machine’s components. Such a representation enables workers to “look” inside the machine at even hard-to-see areas so that they can diagnose issues more easily. AR enables the application to overlay step-by-step instructions on the image as well as display real-time guidance from remotely located experts while workers are making a repair. In some cases, the next repair instruction may be provided to a technician only after the current step has been digitally confirmed as completed, increasing the quality of the repair.
On shop floors, workers can wear AR devices that help them visualize IoT data in the context of their work. For example, when IoT sensors are embedded in machinery on a production line, technicians wearing AR glasses can walk the floor to get insights into how equipment is functioning. The closer a technician is to a piece of equipment, the more specific the information will be. IoT-AR applications also make it easier to spot issues across assembly lines.
When designing equipment, companies can feed IoT data into digital models and use AR to interact with those models at scale, giving designers a dramatically more accurate representation of how a design will perform in the real world. For instance, the representation may help designers locate the most commonly used features so as to improve the design and performance of the machine. Similarly, remote IoT devices—such as cameras and temperature sensors that are installed in and around structures—enable data collection that can help companies create digital twins. When such virtual models are paired with AR devices, architects, contractors, and builders can interact with data to provide better inputs, and, among other things, improve the placement of plumbing and electricity.
Deploying IoT and AR is one of the quickest ways that companies can boost the bottom line. IoT-AR solutions can help companies improve performance in three ways, often at the same time.
To highlight how IoT-AR solutions can generate significant financial returns, we investigated two use cases and developed an IoT-AR app to bring them to life. The app demonstrates how these technologies can help reduce equipment maintenance and warehouse costs.
Reducing Costs and Increasing Revenue from Equipment Maintenance Using IoT and AR. After-sales services are a lucrative source of revenue for industrial manufacturers because most customers sign service contracts for the maintenance and management of their assets. However, manufacturers are facing increasing pressure from customers to provide not only better service at a lower cost but also faster response times. In addition, many manufacturers must deal with a growing skills gap. Baby boomers are leaving the workforce, and companies are losing experience and expertise en masse. Against this backdrop, using an IoT-AR solution for equipment maintenance can address multiple pain points:
Several companies—such as Global Foundries, Howden Group, Sysmex, and Volvo Group—have deployed IoT-AR solutions to tackle maintenance issues, provide customers with self-service options, and improve quality. For these OEMs, IoT-AR solutions are transforming the speed and quality of service delivery, increasing customer satisfaction, and reducing consumer churn.
By implementing IoT-AR solutions, companies have realized sizable economic benefits:
To show how a solution that pairs IoT with AR can lower costs and boost revenues, we developed a case study using real-world data. In our study, a manufacturer with annual after-sales service revenue of $1 billion, or 30% of its annual revenue, employs 1,000 service engineers.
By deploying an IoT-AR solution with 20% to 30% of its service engineers, we found that the manufacturer has the potential to generate $13 million to $30 million of net benefit for every $1 billion of revenue. We have often seen companies achieve results toward the higher end of this range. (See Exhibit 2.)
According to our calculations, the upside potential could be even larger if the company rolls out the solution to all its service engineers. The benefits would also extend beyond the manufacturer, providing significant cost savings for its customers in terms of lower asset downtimes and higher asset performance.
To be sure, companies could use other options instead of an IoT-AR solution. An alternative would be to match engineers’ skills to the right tasks, thereby reducing errors and increasing the first-time fix rate. A pure human solution is likely to cost less, but it won’t provide all the benefits that an IoT-AR solution can deliver. For instance, the former won’t reduce the travel that experts undertake, provide training benefits, or help new technicians get up to speed faster.
To bring the case study to life, BCG and PTC developed IoT+AR, an app that enables users to experience an equipment maintenance solution in 3D. The app can be downloaded for iOS and Android devices. The app puts users in the shoes of a technician to experience how IoT and AR can guide them at every step of an excavator repair.
Shrinking Warehouse Costs Using IoT and AR. The second case study we developed focused on order picking. In mammoth warehouses, selecting products for shipment or delivery is a slow process because workers need to navigate vast spaces. In most cases, a product’s location is noted only at the entrance and exit, so it’s difficult for workers in the warehouse to know exactly where products are. Employees spend a lot of time searching for products, taking suboptimal routes to find what they’re looking for. Moreover, lifting bulky products while holding a clipboard or a checklist often leads to accidents and damaged goods, as does performing a job without clear instructions.
An IoT-AR solution can increase the speed of order-picking operations and boost productivity. Several companies (such as DHL, Intel, and Boeing) have deployed IoT sensors throughout their warehouses, and head-mounted AR devices guide workers to products’ locations by superimposing instructions on users’ fields of vision. These solutions help employees take the shortest routes to the exact locations of products. When workers find the products, their headset camera scans them and checks them off the list, freeing the employees to handle the products.
Companies can benefit from IoT-AR warehouse solutions in several ways:
This case study focuses on a logistics company that manages the transportation of large volumes of products. It works on tight timelines and has several warehouses all over the world. Most product picking is done manually by the company’s workforce of about 90,000 employees.
According to our estimates, a company of this kind with $20 billion in revenue would spend about $3 billion in order-picking labor costs a year. With an initial IoT-AR deployment covering one-third of the workforce, the company could generate a net benefit of $3.5 million to $7 million a year per $1 billion of revenue. Those benefits would only rise over time as the solution rolls out across the rest of the workforce. (See Exhibit 3.)
An IoT-AR solution is not the only way to tackle order-picking and package-sorting issues in warehouses, of course. Another option may be to use robots. Although generally more expensive, robots can work well in industries with large warehouses, small homogenous products, and higher margins. Companies should experiment with different solutions to identify the technologies that will work best for them.
BCG and PTC’s IoT+AR app brings this to life. It enables users to experience product picking using IoT and AR. The app can be downloaded for iOS and Android devices. The app puts users in the shoes of a warehouse employee to see how IoT and AR can help employees find products quickly and safely.
IoT-AR solutions aren’t only about increasing revenue or reducing costs; they can deliver societal benefits as well. For example, using the two technologies together may prove to be a game-changer in health care applications. Specifically, minimally invasive surgeries account for about 35% of all medical procedures. During such a surgery, a doctor makes a small incision in the patient’s body and inserts tiny cameras and miniature instruments so as to navigate the patient’s body without having to make large cuts and incisions.
Although the techniques of keyhole surgery, as it is often called, have been honed over the past 20 years, surgeons still face several challenges. They cannot get all the insights needed solely from the 2D video streams provided by the inserted cameras. Moreover, surgeons must continually switch their focus among several sources of information, including video screens, biosignal monitors, the attending surgical team, and, of course, the patient.
By contrast, an IoT-AR solution would enable a surgeon wearing an AR headset to see a 3D model from diagnostic imaging that is superimposed on the patient; the model would be aligned with the patient’s anatomy with the help of high-precision positional trackers placed on and inside the patient. The surgeon could rotate, zoom in and out, and interact with the 3D model using simple hand gestures. The patient’s vitals and other biosignals, as well as instructions (such as the best angle and depth for a cut), could be overlaid in the surgeon’s view, eliminating the need to switch among the various factors that require attention. And the surgeon would be able to control the entire procedure through hand gestures, voice controls, and eye movements—without pressing buttons or looking away from the patient for even a second.
For surgeons and patients, there would be enormous benefits. Doctors would be able to perform surgeries with fewer errors and achieve higher first-time success rates. Patients would thus enjoy faster recovery times and a lower risk of complications. Several clinical trials are under way globally, and if they receive approval, IoT-AR surgical solutions could deliver enormous benefits to society.
IoT-AR solutions are at an inflection point, but they are likely to become table stakes soon in several industries. Companies must choose the right starting points—the largest cost centers and revenue opportunities—and identify where IoT-AR applications will have the highest value. It’s critical to scale such applications as quickly as possible across the enterprise not only to reap the financial benefits but also to ensure that the cultural change that is necessary for the new approach to succeed has time to take root. Although companies will face challenges, it’s possible to successfully harness these complex technologies. (See “Managing the Risks of Deploying IoT-AR Solutions.”)
Several challenges could derail an IoT-AR initiative after it gets underway. It’s therefore important to keep them in mind from the outset. The major risks fall into the following categories:
Scope and Value. Companies should rank projects by the largest value opportunities, choosing to develop and deploy one or two IoT-AR applications that will deliver value—fast. Companies must develop business cases and articulate the value proposition of using the two technologies together.
The use of IoT technology is increasingly prevalent, so it’s important to ensure that AR has a value-adding role; AR shouldn’t be an afterthought or a gimmick. Our survey suggests that employees are five times more likely to regard AR as a core technology if their company develops an IoT-AR application from the start, rather than adding AR later.
Organization Structure and Partnerships. Cross-functional collaboration across traditional business units is critical for success. Companies must get stakeholders from all relevant groups—not only the core business units but also IT and R&D—to back the IoT-AR experiments. As we mentioned earlier, of the companies we surveyed, 85% already manage the development and deployment of IoT and AR together.
Because the development of an IoT-AR solution must be a business-level initiative, obtaining executive sponsorship is important. Having that backing from the start can ensure that these projects gain momentum quickly. Companies must be careful about choosing technology vendors, partners, and systems integrators for the IoT-AR ecosystem; they must have the ability to support the redesign of operational processes as well as the implementation of technological solutions.
Development and Launch. Companies must optimize the user experience and ensure that IoT-AR devices are comfortable and intuitive in the physical environments in which they will be used. Getting that right can drive adoption faster and ensure that employees routinely use the technologies. The best way to optimize the user experience is to encourage IoT-AR teams to develop minimum viable products, run pilots, conduct end-user tests, and iterate solutions quickly.
Combining IoT and AR technologies enhances the ways that people can interact with the world around them. But to harness that power, companies must choose where to deploy the technologies, build an ecosystem that supports IoT and AR, and develop the talent that can use both technologies. Companies that do so will be able to not only reduce costs, boost revenues, improve the customer experience, and strengthen brands but also use contextual data in ways never before imagined.