• Articles
  • Collections
  • Videos
  • Explore HP

Innovation

Stepping into the metaverse from the factory floor

Augmented reality technology is transforming manufacturing and machine repair in nearly every industry, from aerospace to outer space.

By Rose de Fremery — April 28, 2022

“It’s so surreal,” said Boon Leong. The technician was standing in front of a stalled industrial printer at an HP manufacturing partner facility in Malaysia, when the printer’s authentication device began to appear in his field of vision. “I can see everything!” 

A few minutes before, he had placed a Microsoft HoloLens 2 mixed reality headset on his head — just like you would a pair of goggles. When he turned it on, a friendly face appeared in his field of vision. It belonged to Karen Ng, an HP expert in Singapore who had remotely joined him for a live troubleshooting session using Microsoft Dynamics 365 Remote Assist AR software. “OK, here’s what we’ll do,” Ng said, as she guided him through the steps for renewing the security keys that had expired.

 

RELATED: What is extended reality (XR), and how does it work?

 

Augmented reality (AR) technology allows manufacturers to rapidly repair machines and improve production, onboard new hires, upskill current employees, retain the institutional knowledge of recently retired workers, and enable highly technical customer service from afar. They can also tap AR to improve compliance and safety, cut costs associated with travel, and reduce their carbon footprint. This type of immersive technology, which usually involves a headset or smart glasses and specialized software but can also include tablets and smartphones, is leading manufacturers into the metaverse and opening up new opportunities for the future.

Courtesy of HP

AR technology brings a support engineer right to the technician, making virtual help in a physical space possible..

According to PwC research, efficiencies and opportunities created by AR and VR have the potential to boost the global economy by 2030 by up to $1.5 trillion – and of the two technologies, AR will provide the bigger boost.  

“The fact that intricate manufacturing tasks can be carried out easily, faster, and in a more convenient way using AR is proof that it will increasingly be sought after in the near future,” says Stanley Leong, senior IT engineer, print and R&D engineering at HP. 

A factory technician wearing the Microsoft HoloLens 2.

Courtesy of HP

A factory technician wearing the Microsoft HoloLens 2.

Exploring AR’s potential in all kinds of spaces

At companies like Jaguar Land Rover, employees now simply point a tablet at the vehicle’s dashboard, and an AR app shows them all the auto components that are hidden behind the paneling. Schneider Electric leverages AR to remotely troubleshoot electrical distribution equipment and give customers virtual access to experienced field technicians. And at Boeing, the world’s largest aerospace company, AR is providing voice-enabled step-by-step instructions to assembly technicians, reducing error rates to zero and cutting wiring production time by 25 percent. 

And nowhere is the ability to get expert, virtual guidance more important than in space, where one false or delayed step in the process for repairing a piece of equipment could endanger not just the mission but the lives of the crew. NASA is exploring how AR and VR could one day help astronauts conduct repairs on the International Space Station (ISS), the moon, or Mars. The ability to make accurate decisions is crucial in these settings, where there may be a delay in communications with Mission Control, and the window of time to successfully repair a piece of equipment may be vanishingly short.

Aboard the ISS, astronaut Soichi Noguchi of the Japan Aerospace Exploration Agency (JAXA) tested the process for repairing the T2 treadmill he and his colleagues use to stay fit while orbiting the Earth using Microsoft HoloLens headsets and custom-built AR software, as part of the T2 Augmented Reality Project (T2AR). Rather than consulting a dense PDF document on a laptop or tablet which wouldn’t allow for hands-free operation in the narrow and sometimes dimly lit confines of the space station, Noguchi simply put on a HoloLens headset and saw the steps he needed to follow directly within his field of vision. 

One day, AR could be part of a technology toolkit that helps NASA astronauts find answers and solve problems on their own, making everyday operations more efficient. “​​For these future exploration missions, the purpose of technologies like augmented reality is to make the crew smart and to provide the crew decision support, so they don't have to rely on Mission Control,” says NASA’s Jeremy Frank, who led the project.

A virtual helping hand — and a view into the machine

In normal times, an HP service technician would have traveled to Malaysia to help Boon Leong renew the expired security keys in person, but because of pandemic-related travel restrictions, that wasn’t possible. AR technology provided a quick solution and in the process, revealed the potential to improve efficiency and sustainability even after the pandemic.

 

AR is not only giving manufacturers a fresh perspective on their processes today, but opening the door for transformative new applications in the future.

“Using AR in remote collaboration can reduce travel related to operation support, improve response time, and optimize resources” says Chiew Beng Ng, IT manager, print and R&D engineering IT at HP. “It can also be used for more effective cross-site knowledge transfer and retention.”

At each step, Boon Leong saw not just the printer’s actual authentication device and his own real-world surroundings, which were crucial to help him get the job done and keep him safe on the factory floor, but he also saw diagrams of the printer’s inner workings complete with little annotations that Karen Ng was drawing in real time. These visual guides made it much easier for him to figure out what he was supposed to do.

Online 3D printing service JawsTec, which uses HP Jet Fusion 3D printers, similarly uses AR to get expert virtual visual support from HP xRServices when a problem threatens to halt production. For example, when a 3D printer system displayed a pressure sensor error, a JawsTec engineer was able to get immediate access to HP service experts to help resolve the issue in under 30 minutes. With timely remote access to the right expertise via AR, JawsTec can get printing problems diagnosed and resolved faster, rapidly resume normal operations, and increase production.

"With the HP xRServices. we bring the machine to HP directly without leaving the office," says Oscar Klassen, CEO of JawsTec. "This technology allows quick problem solving, fast solutions, and multiple engineering eyes to take a look directly in the machine. The HoloLens provides more expertise for our self-maintenance support with lower costs and rapid results.” 

AR as a digital transformation enabler in the manufacturing sector

Although manufacturers have already discovered AR’s clear promise for use cases like equipment repairs and troubleshooting, they are also actively investigating how it could have a transformative impact on day-to-day operations, such as new hire onboarding, regulatory compliance, and ensuring the safety at a particular facility. 

HP xRServices — The world's first mixed reality customer support for printers

Marked by the support engineer, 3D holograms indicate the exact spots where the operator needs to adjust or activate.

“In manufacturing and industrial firms, there’s a lot of interest in utilizing AR as part and parcel of different workflows,” explains Michael Larner, principal analyst at ABI Research. “That can be on the assembly line, starting with the basic task to onboard an individual to make sure that they are following steps correctly. It’s also useful for data logging – for example, recording that a part has been moved to a different part of the facility.” 

This is especially important for the aerospace and automotive sectors, in which specific tools are strictly designated for certain tasks and work stops if a particular tool goes missing. Manufacturers can also use AR to confirm they’re complying with safety regulations, such as chemical plant verifying that it has completed the correct steps for fixing the valves on a particular piece of equipment. 

But AR is not only giving manufacturers a fresh perspective on their processes today, it’s also opening the door for transformative new applications in the future, creating opportunities in areas businesses may not have even considered yet. And as the technology advances and makes it easier to create new immersive types of AR environments or to wear headsets longer, it will become an accepted and adaptive tool for work.  

“When smartphones first entered the business world, people said ‘That’s just for tech geeks, not for me,’” Larner says. “Now, everyone in business relies on smartphones to be productive. If you’re trying AR, you should give it to as broad a range of people as possible.”
Tags
Manufacturing / Virtual Reality / Innovation / Technology and Innovation / Hybrid Work
Posted In
Innovation

Recommended Stories

Taking a scientific leap by printing one cell at a time

An HP device that dispenses single cells is empowering precision research at Oregon State University.

The new shopping experience is online and in-store

Technology is transforming the retail experience, creating engaging in-person destinations with the personalization and convenience of online shopping.

How AI is transforming the higher ed classroom

Together, teachers and students are getting first-hand experience with the potential and pitfalls of generative AI.