Today, we’re very excited to finally make our Skylight for iOS and Android smartphone and tablet offering public. This has been a long time coming for us at Upskill, as we have been working over the last few months working with our private beta customers to make sure we shipped a product that fully met the needs of our target customer base.
This latest Skylight release is particularly significant for our customers and partners because with its new full-featured functionality on smartphones and tablets, our vision of building a multi-experience AR platform is at last complete. Skylight can now bring the digital enterprise to every hands-on worker with the right user experience on the right device best suited for the task.
What is the right user experience and the right device? In our opinion, it is driven by the nature of the tasks, defined by the need for the hands-on worker to be hands-free, the need to be mobile (physically moving around), and the information that is needed to successfully complete the tasks. Let’s unpack what multi-experience means for the hands-on workforce.
As Skylight has been growing within our customer base across many industries, it has become clear to us that in order to achieve true scale of meeting the needs of every hands-on worker working for a given customer, we need to enable a heterogenous environment of devices that are best suited for the work, and that doesn’t mean just offering support for the industry-leading smart glasses and Microsoft HoloLens.
Breaking this down by the tasks that the hands-on workers perform, we can draw a simple chart where one axis is the need to be hands-free and the other is the need to be mobile. Note that when we say mobile here, we’re asking whether the worker is stationary most of the work day or moving around constantly.
This chart is broken up into quadrants. Examples of job roles for each quadrant are:
- A wire harness assembler visualizes step-by-step work instructions on smart glasses to perform a routing task on a 10×50’ formboard
- A retail store worker on a smartphone verifies stock levels and proper layout of products on the shelf, documenting anomalies with photos and a short text description of the issue
- A maintenance technician performs a gearbox teardown and overhaul while viewing instructions on a mounted tablet (a fixed display) on his workbench
- An engineer is on a PC providing remote assistance to a field service technician working on a repair order at a switching station
We firmly believe that smart glasses and their unique ability to enable heads-up interaction with task-specific information while keeping the worker’s hands free delivers huge value, as seen by our public case studies. However, in order to bring the digital enterprise to every hands-on worker in the factory, warehouse, and the field, expanded choice of devices is critical to achieving this outcome because of the variety of job roles and tasks that demand it.
Choosing devices for a particular user doesn’t mean that there is one right device and only that device. One benefit of investing in a single unifying platform that supports these device types is the ability to seamlessly transition from one device to the next. For example, the aforementioned maintenance technician performing the teardown and overhaul could very well be helping install the fresh gearbox back into a helicopter, necessitating the transition into smart glasses or AR headsets to unlock optimal productivity and accuracy.
The full spectrum of devices Skylight now supports unlock unique user experiences that can meet the needs of every task in making, moving, and servicing products. Just like the devices, the right experience for a particular hands-on worker is dependent upon the nature of the task he or she will perform. Here, the critical variable is the information needed to successfully complete the task.
For the last 7 years, Upskill has pioneered the concept of assisted reality – we actually invented the term to describe a non-immersive 2D wearable AR experience. Typically delivered on a monocular smart glasses device such as the Glass Enterprise Edition, Vuzix M300XL, Realwear HMT-1 (as well as its Zone 1 intrinsically safe model, the HMT-1Z1), and lower-cost binocular devices such as the Epson Moverio BT-350, assisted reality is best suited for providing visual cues to the user in a heads-up and hands-free fashion. This is the most prevalent form of industrial AR experiences on wearables today, due in part to the fact that assisted reality devices have been on the market the longest (starting with the original Google Glass Explorer Edition in 2012), as well as striking the balance between all-day usability and cost.
Following the convention of holding a 7 to 10-inch tablet at an arm’s length, the optimal content to display in assisted reality is glanceable – something you can quickly look at to understand what to do with the task at hand. Easily legible text, tables, images, videos, status information coming from IoT-enabled sensors, tools, and machines, and snippets of engineering diagrams are good examples of the type of information commonly found in assisted reality deployments.
Taking a task-centric perspective, assisted reality is best suited for someone who requires low to moderate fidelity of information to complete the task. It therefore is best for routine work, which can include complex tasks that are well known to the user. One notable exception to this is when assisted reality experiences incorporate remote assistance, which aids the hands-on worker in completing an unfamiliar task. Because assisted reality devices feature extended battery life over other device types and integrate with personal protective equipment (PPE), they are particularly popular in all-day/most-of-the-day use cases.
Mixed reality, while newer to the AR industry, is quickly gaining steam. Defined as the blending of the digital and physical worlds on wearable headsets, information is presented to the user’s line of sight in a more immersive fashion. With mixed reality, there is no limit as to whether the experience is 2D, 3D, or some mix of both. Microsoft HoloLens is the premier device for mixed reality experiences and is the only device we find viable for production use today, which is why Skylight’s mixed reality experience launched on it.
Mixed reality headsets combine the in-your-perspective stereoscopic displays with a suite of sensors to build awareness of the environment, whether sensing the space around the user or to detect particular objects. With spatial awareness, the information delivered with mixed reality can be placed in the real world and stay there – locking content to specific spaces and objects.
Skylight’s mixed reality experience today is focused on empowering users to create, anchor, and sharing our UX concept called Workspaces. Skylight Workspaces allow users to work with multiple sources of information visualized at the same time, and use gestures to place, move, and resize them in order to deliver the most intuitive layout of content and minimizing disruptions to the task at hand. It differentiates from assisted reality with the type of information more easily delivered to the user – simultaneous views of 2D glanceable content, complex engineering drawings requiring a larger and higher resolution to understand, PDF-based work instructions, videos showing high degrees of detail, etc. – all spatially oriented depending on the proper context.
Since the information available to the user can be much higher fidelity, mixed reality in Skylight is best suited for complex tasks unfamiliar to the user (or too complicated to rely on memory), on-the-job training, and non-routine work such as repair and equipment overhaul. As mixed reality devices tend to be higher cost over assisted reality, it is better targeted towards the highest value applications for augmented reality that demand high fidelity information.
Skylight Workspaces enable the future for users to work with 3D content sources, integrate even more tightly with content and workflows defined in CAD and PLM systems, and use artificial intelligence driven features such as natural language processing and object recognition to further make the system more intelligent. We’re excited to continue our development in making that more accessible and scalable – stay tuned.
Skylight Experiences on Smartphones & Tablets
The latest experiences added to Skylight leverage conventional smartphones and tablets to complement assisted reality and mixed reality. Compared to wearable devices, mobile devices unlock the ability for users to easily enter freeform information (e.g. typing, annotating drawings, videos, and images) using touch-enabled input. The greater accessibility and availability of mobile devices today also means that more people can access the same connections to the digital enterprise that Skylight brings to smart glasses and Microsoft HoloLens – thereby empowering every hands-on worker.
The Skylight experience on smartphones and tablets are split into two – standard handheld UI and Gallery Mode. In handheld UI, Skylight will display existing applications and workflows with touch-enabled input and provide native support for keyboard entry, including speech-to-text capabilities (may require internet connectivity). Rather than the horizontal scrolling found on assisted reality devices, the workflow is vertical and multiple, high-level steps can be shown at the same time, with the details a touch away. This UX is best suited for users performing tasks that do not require their hands to be free most of the time and have a high need to document their work, such as quality assurance, inspection, and routine maintenance.
Skylight Gallery Mode is designed for the fixed display applications that allow hands-on workers to access information while performing stationary work. Similar to assisted reality, the type of information is optimized for glanceability, as the display is likely affixed at some distance away. This UX is best suited for the same demographic that would find assisted reality experiences optimal for their work, but do not require leaving their workstations to perform their tasks (e.g. those working at a workbench or a fixed spot on the assembly line).
Skylight Gallery Mode is also best suited for operational environments where device theft is a major concern, where multiple users are working from the same work instructions in the same work area, and where there is an elevated price sensitivity in rolling out the solution. Of course, there is a tradeoff with the lower cost of this solution – the two primary factors to consider are that mounted tablet solutions may not be best suited for see-what-I-see remote assistance features, due to the camera being in a fixed location, and that they prevent always-available access to information on the move.
Sidebar: What about AR features on mobile devices?
We’ve been exploring mobile-based AR technologies such as computer vision and object recognition algorithms since the beginning of Upskill. While we believe capabilities like ARKit and ARCore are becoming more mature and makes spatial awareness on smartphones and tablets more accessible, we are still working to find the silver bullet of repeatable, cross-platform use cases that delivers the best ROI for businesses deploying Skylight. Of course, this doesn’t mean that those capabilities aren’t useful today – it just requires custom development. We continue to stay in lockstep with the latest developments and learn/co-innovate with our customers and partners to deliver this exciting capability in Skylight.
We believe that the role that mobile devices have today for the hands-on workforce is to complement wearable AR solutions with easy access to the digital enterprise, extending the device options and enabling use cases where those devices make better sense. The journey to enterprise-wide deployment AR is just beginning for many – this expands where they can start from and makes the on-ramp even lower, with the benefit of having just one platform to build, deploy, maintain, secure, and upgrade applications across their entire operations.
Multi-experience in Skylight means being able to work with the various types of devices, information, and tasks to empower the hands-on workforce to perform at their best. This includes enabling users to seamlessly transition from one experience to the next (e.g. a handheld mobile user putting on a Microsoft HoloLens to perform a task, and seamlessly picking the work back up on their mobile device afterwards).
As Jason Welsh, a Managing Director in the Accenture Extended Reality group said, “as clients look to leverage their existing investments in mobile technology for XR deployments, Skylight provides a unifying platform that enables them to explore a variety of AR devices and then choose the one that best meet the needs of industrial workers in the factory, warehouse, or in the field.”
We couldn’t agree more, Jason. This is an exciting milestone for Upskill, our strategic partners like Accenture, and the hands-on workforce across the world.