InstructAR is a set of visualization tools to overlay information directly onto a real world workspace allowing us to guide workers through complex, sometimes non-intuitive tasks simply and accurately. We are currently experimenting with applications of this idea to pallet and truck loading since loading instructions can become complex and loading accuracy is important for optimized pallet builds and truck loading plans.

In our pallet loading application we project the location of the next box to load directly onto the pallet itself using the Hololens mixed reality headset. The user advances through the steps by using air taps or voice command. For demo purposes the app has three pre-built pallet configurations but we imagine in the future that it could obtain pallet instructions from an optimizer running on a server.

Our Tire loading application has a similar interface but takes its loading plan from a real optimization algorithm created by the data science team at JDA Labs.

We have also experimented with visualizing the instructions using spatial AR, i.e. by projecting the instructions onto a pallet using a digital projector. This has the advantage of not requiring a bulky headset but has other design considerations that need to be worked out.