Extended reality technologies are growing fast across all industries, even the maritime industry. But what is extended reality? Is it just a fad? And will extended reality really make a difference in the maritime industry?
What is extended reality?
While many people use extended reality (XR), mixed reality (MR), virtual reality (VR) and augmented reality (AR) interchangeably, they’re definitely different.
In a nutshell, VR immerses users into an entirely virtual world, while AR overlays virtual content on the real world. The virtual content in AR can ‘fit into’ the real world, but users can’t interact with it, and it can’t interact with the real world.
The classic example of augmented reality is the head-up display, but most people’s first experience of AR was Pokemon Go in 2016. If you missed it, Pokemon Go was a smartphone game that involved “throwing” virtual balls at digital creatures overlaid on the real world. However, players couldn’t reach out and interact with the virtual creatures.
In MR, you can either add digital content to the real world, or real-world elements into a virtual world. Unlike AR content, digital content in MR can interact with the real world. A virtual ball in MR can bounce off real-world walls, and a virtual spare part can align itself with real-world equipment to demonstrate a maintenance procedure.
XR is a catch-all term for AR, VR, MR, and any related future technologies.
What hardware do you need for XR?
Since cinematographer Morton Heilig created the first VR machine in 1956, technology has progressed. For years, the size and weight of VR headsets limited the practical applications of VR; however, in 2020 we have cost-effective commercially available VR and AR headsets. Even our smartphones have everything necessary for simple AR and MR applications.
To run a VR application, you need a headset and a computer. For budget VR experiences, you can get away with a mid-range smartphone and a Google Cardboard headset, but commercial VR headsets are more comfortable to wear for lengthy periods.
Similarly, simple AR and MR experiences run perfectly well on a smartphone with a reasonable camera; however, holding the phone up to look “through” it is an obvious problem in a work environment: it’s inefficient at best, and dangerous at worst.
AR headsets address this issue. Magic Leap and Hololens are well-known examples of affordable mixed-reality headsets with a growing range of applications. From training surgeons, to navigating and designing cars, companies are adopting mixed reality at a growing rate.
What are the barriers to adoption?
Motion sickness is a common problem with VR, but not for MR and AR. Because MR/AR users can see the real world, the signals from their eyes and inner ear match, thereby removing the primary cause of motion sickness.
Interacting with virtual objects in 3D is a technical challenge. The mouse, keyboard and touchscreen aren’t viable in XR environments. Current XR systems use eye, head or hand tracking, data gloves or game pads to interact with the system, but interfaces are still developing.
Lack of user technical literacy and training slow adoption of any technology; XR is no different. With increasing adoption of XR for entertainment and recreation, equipment and interfaces will become more familiar, developers will improve user onboarding processes, and resistance to XR will drop.
The maritime industry presents extra barriers to XR adoption. While the human factors are similar, other factors differ considerably.
Certain XR applications depend on 3D models of real-world installations. While generating a model is straightforward when building a ship or installation from scratch, scanning an existing installation to generate a model is more complicated. The size of ships and platforms, and the fact that we build to customised designs exacerbates the problem. Unlike cars or aircraft, the cost and effort of generating 3D ship models for XR isn’t subject to the economies of scale. However, companies are working to simplify the process of scanning and model generation to bring the time and cost down.
Until recently, the price and reliability of internet at sea limited adoption of many technologies on ships, including XR. Two changes in the last few years have addressed this problem: affordable, reliable internet is becoming more widespread; and XR startups are developing techniques to work around limited connectivity.
The marine environment itself is the next barrier. While it presents problems for all technology at sea, movement is a particular problem for XR. Many XR applications rely on gyro or GPS sensors to align a virtual model with the real world, and to track the user’s movement and gaze. A ship’s motion adds extra components to this calculation.
To make it worse, most ship structures are quite uniform. On a ship, the visual similarity of frame 30, frame 130 and frame 230 make it difficult to fall back on visual cues if GPS and motion-tracking fail.
Is XR just a fad?
It’s unlikely, given the accelerating growth and development over the last sixty years. In June 2020, Angel List listed 2,083 augmented reality startups and 2,398 virtual reality startups. According to Statista, Microsoft owns over 10,000 virtual reality/augmented reality patents, and companies expect to invest 8.5 billion US dollars in AR/VR technology just in training applications in 2023. In the third quarter of 2019, 46.3% of survey respondents said their business used XR for design and prototyping.
Giulia Carosella of IDC explained, “AR/VR commercial uptake will continue to expand as the cost of entry declines and the benefits from full deployment become more tangible. Focus is shifting from talking about technology benefits to showing real and measurable business outcomes, including productivity and efficiency gains, knowledge transfer, employees’ safety, and more engaging customer experiences.”
How do companies use XR?
Other industries use XR to simplify repairs and maintenance; improve effectiveness and reduce costs of training, simulation, design and prototyping; data visualisation, healthcare and surgery, and logistics. All of those would be just as useful in the maritime industry – and some of them already are.
In 2019, Wartsila successfully tested voice-controlled AR wearables and remote guidance software that allowed crew and technicians to communicate with shore-based experts in real-time. Wartsila’s even worked out how to use the system in areas of low connectivity.
In the oil and gas industry, Luminous Group generated and used 3D models of a vessel to design and build pipework systems that fit first time, then repurposed those models for VR training. This saved time and money, and improved efficiency.
The same technology is effective in logistics and training – it’s already being used to teach surgery, fire safety, driving cranes and industrial equipment, and even people skills. It’s easy to imagine using similar technology to teach navigation and shiphandling, or improve situational awareness in navigation or emergency situations.
Design and prototyping are the most common uses of AR and VR tech in industry today. Since 2007, Airbus has been using Realistic Anthropological Mathematical System (RAMSIS), a VR system, to simulate cabin design. Now XR is spreading across industries from architecture to manufacturing.
Are maritime companies already using XR?
All sectors of the industry, from navigation and cargo planning to management and logistics can benefit from XR technologies. A few of the companies and organisations working in the maritime MR space are:
Centre of Excellence in Maritime Safety (CEMS)
Singapore Maritime Institute and Singapore Polytechnic set up CEMS to research solutions to enhance navigational safety. Their two areas of research are novel innovations for technology transfer to the maritime industry, and development of next-generation training systems & solutions.
Fostech
Fostech develops industrialised MR solutions for the maritime and offshore oil and gas industries. They focus on sales & marketing, training & education, service & maintenance, and operational usage purposes.
Kongsberg Digital
Kongsberg’s K-SIM Safety platform uses MR, VR and AR for advanced firefighting, search and rescue training. The system lets students learn, experiment and make mistakes in a safe virtual environment.
Luminous Group
Focuses on custom MR solutions for remote assistance, guided learning, visualisation & collaboration, and contextual data delivery. They’ve delivered solutions in the shipping and oil and gas industries.
SEDNA
The Safe Arctic Bridge project received 6.5 million euros to develop technology to improve navigation and operations in Arctic environments. It uses an augmented reality display to give the bridge team key data when they need it.
Wilhelmsen
Wilhelmsen, the maritime logistics, ship agency and ship management company, has developed a smartphone AR app to let customers overlay, view and interact with holograms of Wilhelmsen’s products.
Conclusion
XR technologies have the potential to improve safety, efficiency and cost-effectiveness in almost every industry. As adoption of XR tech grows in related industries, it’s inevitable that it will spill over into the maritime industry.
Compared to other industries, many sectors of the maritime industry rely on dangerous, outdated or inefficient technology for training, planning, design, logistics, and emergency response. XR can help to address many of these issues, and XR training advantages will help bring the maritime industry into the modern world – and keep it there.
Source: Nic Gardner, Thetius
