Meleap delivers augmented reality to the iPhone before Apple

Meleap is an interesting case study about how pragmatic product engineering can create an AR product ahead of Apple’s rumored AR product

Meleap delivers augmented reality to the iPhone before Apple
Credit: Meleap

Today, augmented reality (AR) could be translated to mean “prototype.” Waiting for improved software and lighter, lower-cost and faster headsets, developers build prototype applications. One example is Japanese company Meleap, which used used clever engineering to deliver light, fast and low-cost AR today—on an iPhone no less.

Hado, an active game that was demonstrated at Virtual Reality Silicon Valley Expo, solves a lot of AR problems still on the horizon. Designed for a class of applications using tried and true technologies, meleap’s engineering is simple—beautifully simple.

The inspiration came from 31-year-old CTO Hitoshi Araki’s desire to physically play inside his video games when he was young. Hado is played on a physical court—like a volleyball with virtual fireballs.

This 9-second video shows how the game is played from a player’s perspective using a simulation prepared by the company.

And this 7-second video shows a spectator’s perspective. 

How Hado works

The Hado headset uses the iPhone’s camera to capture the field of view in front of the player and is housed in a light frame without sides, giving the player peripheral vision during play. The player wears a wristband located in the below image to the left of the headset that tracks hand movement and gestures when the player throws or returns a fireball. The wristband has an inertial motion sensor (IMU) that connects to the headset-mounted iPhone with Bluetooth. Meleap is working on adding an optional iPod Touch in place of the wristband to give the players in-game adjustments, such as ball speed and interchangeable game objects.

meleap hado ar headset wristband Steven Max Patterson/IDG

Though Apple has not introduced its rumored AR-capable iPhone, Hado can track the player’s location on the court on an ordinary iPhone without a 3D camera and sensor-generated point map. Within the design of the walls made from lightweight material are AR codes, which are geometric patterns that orient the player's position.

meleap hado ar diagram Meleap

When the camera detects an AR code, it knows its preprogrammed location. The player position can be deduced from the size and skew of the AR code image, and the relative orientation to the fireball can be correctly overlaid onto the player’s field of view seen through the iPhone camera. Each user’s perspective is independently aligned in the virtual 3D geometry with the physical world. Play is synchronized between headsets with Wi-Fi.

Comparing Hado to state-of-the-art technology

The Hado headset does not have the sophisticated technology that the Microsoft HoloLensand Metavision Mets 2 headsets have. Both will orient the wearer’s field of view to the 3D geometry of the room without AR codes, using multiple cameras and sensors that detect walls, furniture and other real objects. The Hado headset, on the other hand, breaks down the camera image into two tiny monitors on the iPhone screen a few centimeters from the eyes and focuses the images with two lenses. While the HoloLens and Meta 2 split a projected image, which is diffracted and layered to produce images on the visor lens overlaying the holographic image into the 3D geometry of the physical world.

The resolution of both holographic headsets is higher than the Hado/iPhone. Compared to the Hado, the user can interact with objects in the field of view of the holographic headsets because the user interface includes finely detailed hand tracking.

The mobility of the Hado is much greater. The HoloLens really is not designed for running and jumping, and the Meta 2 is tethered to a powerful PC by a 9-ft. HDMI cable. One of the limitations of AR is the user tends to be isolated in a task, albeit import tasks like surgery or engineering design. Applications tend to be single user. The Hado shares a common perspective in a group activity and is designed for movement and aerobic exercise.

Meleap’s business model

AR does not have a distribution channel for apps yet. Meleap distributes the headsets to theme parks, shopping mall operators and sports clubs, which in turn sell game time to consumers. About 90 locations have Hado. According to Araki, the four headsets, iPhones, the walls with AR codes, and supports cost about $15,000 in Asia.

Hado will not capture the imaginations of enterprises prototyping AR applications in a wide range of fields, such as architecture, engineering and surgery. Nevertheless, it is a compelling case study about how pragmatic product engineering can produce an interesting product within the scope of an application and the performance of the hardware.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10