First Look

New Google Translate includes VR signage, photo and speech

... no more getting lost and not being able to ask for directions

Chicago Theater / Wikipedia
Credit: Remixed by Mark Gibbs

If you’ve ever travelled in a country where you don’t speak the language you’ll know how confusing and frustrating it can be to try to make sense of things like signs, particularly when you’re trying to find catch a plane or train. Google wants to make your travels easier with the latest version of Google Translate app (available for Android and iOS).

google translate Google

Main user interface of Google Translat

The previous version of Google Translate allowed you to translate text in photos, which was very useful, but the new version takes the whole process much further. Now the app can access your smartphone camera and replace the foreign text, in near real time (Internet connection permitting), with the text translated to a different language. The end result is pretty amazing (although occasionally bizarre).

So, for example, you might be in downtown Mexico City and wonder what’s on at the theater (I’m sure there’s a theater in Mexico City called the Chicago). Just whip out your smartphone, point it at the marquee and you would discover that the sign says “My hovercraft is full of eels”. 


theater 1 Mark Gibbs

Google Translate virtual reality signage

The beauty of the system, when it can identify the text to process, is that the translation is in the visual context, i.e. it's a virtual reality overlay. And it works for any printed text including newspapers, books, and menus.

theater 2 Mark Gibbs

Google Translate using scan mode

Of course, it doesn’t always work perfectly; when I tried the same thing with Hungarian text (“Sajat legpárnás tele van angolna” which would, of course, be the correct language for the quote) it seemed to be harder for Google to identify the text to handle so I had to click on the “Scan” button in the app and then, using my finger, highlight the text to be translated.

google translate speech Google

Google Translate in speech mode

Currently the app handles 36 languages for photos of text and can translate signs for English to and from French, German, Italian, Portuguese, Russian, and Spanish. It can also translate speech fast enough for effective conversations (again, Internet connection speed allowing) and once two languages are chosen it can figure out which language to translate from and to without prompting as each participant speaks.

This is a terrific application that combines a whole heap of technologies and one that I wish I’d had way back when I was traveling a lot in Europe. It would have made decoding menus, not getting lost, and catching planes  a piece of cake.

Join the Network World communities on Facebook and LinkedIn to comment on topics that are top of mind.
Must read: 10 new UI features coming to Windows 10