Categories

Meta

Interactive Hologram

One of my hang ups with the Leap Motion Controller is that there is a shortage of options when it comes to effective 3D visual outputs which take full advantage of the added dimension of control. Traditional 2D screens, or even projections, don’t accurately convey the sense of control over depth that the Leap provides. This is something that I’ve alluded to in a previous post regarding some of the possible applications of the Leap Motion Controller.

Last summer, I created a basic interactive product demo at 9 Yards, which utilised the DreamOC hologram unit and a Leap motion controller, to demonstrate a possible concept which allows the Leap to be used to create an interactive product catalogue. The result of which can be seen in the video below.

After showing this demo to a few clients, one in particular, a lightbulb manufacturer, became really interested in the idea and we worked with them to develop the initial concept into a full application. The idea was to turn the application into a holographic catalogue which would allow users to browse, select and view different types of lamp.

We took the initial concept, and proposed an application which would span a shared desktop over two HD resolution screens, the first being a holographic display and the second being a touch small OEM screen, a Leap Motion Controller would be mounted above. The application functionality would provide the user with an interface with which to navigate through a catalogue of lamps, using a 2D GUI on a touch screen mounted below the hologram. When a lamp was selected it would appear in the hologram, allowing the user to rotate it using gestures.

Whilst the initial demo was created using Flash and the Papervision3D library, I developed the final application in Unity3D which is a far more suitable and capable interactive 3D development environment. This was my first major project using Unity and it proved to be really powerful, allowing for really easy integration with the Leap Motion Controller as an input device, SQLite database functionality to track metrics on interaction and customer details, and most important, excellent real-time rendering and display of 3D models with really effective lighting and texturing.

We hit a few challenges with development along the way, here are a few of the most notable, and worth bearing in mind:

– Models:

Modelling can be done with pretty much any modelling software, Cinema 4D, Maya, Blender and so on. However, the tricky and sometimes unpredictable aspect of the modelling for us was the point at which you import into Unity. Often I would have to reapply materials after import in order to ensure that the lamps appeared as intended. To some extent, this was also true of the geometries within models – occasionally, we found that models would import into Unity in an incomplete state, this required us to make a few touch ups once they were in Unity to correct and missing features. I expect this may also be remedied through further investigation in to the export settings in the modelling environment. It’s also worth noting that Marmoset’s skyshop shaders came in really handy for this project!

– Dividing the viewport:

In order to show the lamp model on the holographic display, we had to create three separate viewports and render a different camera angle into each. This is due to the way in which the DreamOC hologram unit creates the holographic effect. Three angled two-way mirrors reflect an image from a screen mounted above them, this meant that the setup for the hologram view of the app was effectively three masked viewports which displayed the front and side views of the lamp model – it took quite a bit of tweaking to get the right effect!

– Unity updates:

Perhaps one of the most irritating issues I faced whilst developing with Unity was that, during the development cycle, updates to the software were released. On a few occasions, I went ahead with updates, not expecting them to compromise the stability of the application in development. Unfortunately, one of the updates, the major version change from version 4 to 5 caused a lot of issues, due to updates to the API. With hindsight, it’s probably a good idea to proceed with caution if you’re considering major software updates whilst heavily involved with a project. Luckily, that’s one of the many things that version control is useful for!

Here’s a short clip of the completed hologram element of the application which demonstrates a bulb being selected for viewing.

It was a great project to develop and I think the end result was really effective, it represented a real deep dive into Unity for me and I really enjoyed it. I’ll definitely be working with Unity in future and I’m really interested in the possible applications of 3D game engines, particularly outside of the realm of gaming, I think there’s a lot that can be done, especially when combining digital visuals with physical inputs and outputs.