A group of roboticists from the University of Pennsylvania have developed a robot that incorporates high-level sensors and Oculus Rift virtual reality to give the user a fully immersive experience from a remote location.
According to an IEEE Spectrum report, the DORA system (which stands for Dexterous Observational Roving Automaton) streams video taken from the robot's cameras, which look a little bit like the face of WALL-E, to the Oculus Rift. Sensors on the VR headset monitor the user's head motion and send the data to the robot, which is programmed to replicate those movements in real time. So when the user turns his or her head to the left, the robot does too, and it streams the video immediately, giving off the feeling that the user is in the same environment as the robot.
Here's a pretty great video showing how well the robot responds to a user's quick and random head motions:
The IEEE Spectrum report explains exactly how difficult of a task this really is. Virtual reality is quite difficult to accomplish because it essentially requires technology to trick the human brain into thinking that it is somewhere where it is not. In VR games and applications, software creates the video footage during the experience and streams it as the user accesses it. The report pointed out that VR research says the lag between the video stream and the user's movements is imperceptible when the latency is under 20 milliseconds and estimates the limit at 60 milliseconds. DORA's latency has been measured at about 70 milliseconds, according to the report. So far, only three out of 50 volunteer testers have reported motion sickness, the researchers said.
DORA currently has a 7-kilometer line-of-sight range and currently operates over a radio link. One current obstacle is the latency problems that could occur when the robot has to transition to 4G or Wi-Fi, which will be necessary if it's going to be deployed in the real world. The researchers are hoping improvements to wireless infrastructure in coming years will help make DORA more accessible in the wild, according to the report.
You can find more information on the project at its website, and check out their plan in this video: