Augmented Reality
Data glasses for the service technician
With “Smart Assistance”, field service technicians are moving beyond their existing duties. The augmented reality application is being developed and used by Swisscom.
Text: Urs Binder,
For Swisscom service technician Markus A., it’s a first: Until now, he has been a voice specialist for routing voice services for corporate customers once another field service team employee had installed and configured the network hardware. Today, it is the first time that he is also going to install the router. And there’s no need for a long, drawn-out training session: The information he needs for individual work steps is given to him by a pair of augmented reality glasses that display the information directly in his field of vision.
The VR glasses display all information needed to install the router.
Custom-fit information at a glance
“Smart Assistance” is the name of the solution being developed by Swisscom Digital Enterprise Solutions (DES) in conjunction with the Customer Field Service organisation (CFS). In concrete terms, it looks like this: Markus A. puts on the glasses – a HoloLens from Microsoft – and looks towards the point in his field of vision where he would like the menu to appear. He uses the voice command “Place” to fix the position of the menu. He then chooses the router model that he needs to configure from the menu. To do this, he targets the menu item in question by gazing at it for half a second. His direction of vision is illustrated by a blue dot.
Markus A. now receives comprehensive information about the router, including step-by-step instructions for various parts of the task and a 3D model of the device with information about the connections and operating elements. If he still gets stuck, he can use a menu command to call an expert who knows the device well. One option is a video call – through the camera in the glasses, the expert sees exactly what the technician can see and can offer 1-to-1 support.
Lukas Zimmerli with the Microsoft HoloLens
Specific business case, quick development
One driving force behind the Smart Assistance project is Matthew Butcher, Head of Team CFS East at Swisscom. He has been looking into augmented reality (AR) since the days of Google Glass and is always on the lookout for new fields of use where a gadget such as AR glasses can be of real use in a business environment. In collaboration with the Development & Integration department at DES, a pilot project was drawn up involving pilot tests for a customer who was having network and voice services installed.
Lukas Zimmerli’s team is responsible for the technical development of Smart Assistance: “We are the doers. Three specialists from my ten-member team developed the solution using the scrum method across three sprints and within six weeks. At the end of 2016 we did the field tests, and since the start of 2017 we have been working on turning the proof of concept into a marketable solution.” One developer was responsible for the HoloLens application, another for the backend that would feed in the information, and a third for video support. In parallel with the AR glasses solution, the same team also developed a smartphone application that would provide the same information.
Zimmerli’s team has also developed an app that shows the same information as the solution intended for the VR glasses.
Technical information gained
During development, Zimmerli’s team learnt quite a bit about the development of AR applications for the HoloLens. The HoloLens can be operated using gestures, speech or simply with one’s gaze, for example. Only around half of the test users were able to get a handle on the gesture control, however. “It also doesn’t make a great impression on customers if the technician is constantly making hand gestures,” says Zimmerli. The team thus decided on gaze control. One more lesson learnt: The usage location does not always offer a reliable data connection. This means that the information needs to be stored in the AR glasses themselves, and not just in the backend.
In the future, customer and machine information will be integrated, as will big data and artificial intelligence-based options such as predictive maintenance. “Our solution is flexible enough to allow it to be used in various areas in the future.” It is also not just going to be used internally at Swisscom, but rather customers should get their hands on it too. However, according to Zimmerli, customers will not receive it as a finished and permanent product, but rather on a project basis and customised for the use in question. This is because every situation is different. And, as he adds: “Augmented reality is no longer science fiction. This technology is revolutionary and allows for a new type of interaction.”
The author is also very impressed by the application.
Don't leave humans behind
For Matthew Butcher, the organisational lessons learnt from the project are decisive. For the field tests, a conscious decision was made to select very different types of employees – both younger and more technically savvy ones, but also seasoned specialists who define themselves by their experience and skill. According to Butcher, the reactions also differed. Some participants thought it was “cool” and considered it to be a success because they were able to solve an issue straight away that they were seeing for the first time in a productive environment. Others, on the other hand, felt that they were now dependent on aids rather than being able to act based on their own expertise.
For Butcher, AR-supported applications succeed or fail with communication. He says that this is where management should come in: “We need to position solutions like this in such a way that nobody needs to feel that they are being used as robots and being overruled by technology.” On the contrary, the technology should be used to enable employees to do more than they could before.
Augmented reality is real
It is not just Swisscom that is getting to grips with augmented reality; AR applications are gaining in popularity elsewhere too. For example, 24,000 lift maintenance employees at ThyssenKrupp will, in the future, use the HoloLens on site as a digital manual and, if necessary, access expert support via video chat – as with Swisscom’s Smart Assistance, this allows both hands to remain free for working. The German paint manufacturer Bergolin is testing the glasses in logistical production processes, where heavy chemical-resistant gloves preclude the use of smartphones. Bern’s Inselspital hospital is testing a prototype to see if AR offers any benefits to planning operations.
More on the topic