15 years ago my obsession with GPS technology sent me on a journey to make its data useful to physically active consumers. Today I find myself dreaming about the potentials of computer vision on mobile devices.
In the Beginning
In 2000 I started to play with a GPS device I received from a colleague and my obsession to make the data useful resulted in MotionBased and subsequently Garmin Connect. It was the richness of the GPS Trackpoint data (location, speed, heading, time), it's reliability since it used perfect time + good-enough location and that so much data could be captured without constant user interaction.
Now I find myself pondering about the applications that can be built when Google's Project Tango hits the market. Like GPS devices brought world-wide location to the masses for outdoor places, Tango is bringing indoor location awareness to mobile devices. Using 3D computer vision, similar to Microsoft's Kinect sensor for the X-Box, Tango senses objects around the device and places it into a 3D map that it creates on the fly.
So now an application developer can determine where the device is located within an indoor space.
Said in technical terms, latitude/longitude/altitude/time for planet Earth is analogous to x/y/z/time for an indoor location. Each location has it's own map, hence it's own coordinate system. Tango matches a device's location to an existing map so a building owner may choose to publish a map of the building that your device could use to find it's location.
Tango goes beyond location awareness allowing a device to understand the physical objects that are present. The technical term is Point Clouds and is the enabling technology that allows the Kinect's Skeletal Tracking to understand the location of a person's hand playing virtual tennis.
The technology allows us developers to interact with the real world like never before. Google's MeasureIt App provides a simple, but useful demonstration of measuring my office sofa.
The possibilities for use of this application are far reaching like:
- finding exactly where the Nutella is located in the grocery store
- leading the blind through an unfamiliar building
- placing a virtual couch into your living room before purchase (see tango-apartment)
- playing virtual reality games that interact with the real world
- mobile autonomous robots (my favorite)
Since Tango can detect who is where at a time we can use Tango for lifelogging applications like GPS watches and other wearables. Think of Tango as being part of a new category of lifelogging called "observables" that can understand life without having to wear it. Said another way...your Tango device won't end up in the wash like your Fitbit.
Google's Not Alone
It's unknown if Apple or Microsoft will launch a phone or tablet with similar sensing technology, but a few years back Apple bought PrimeSense, the makers of the 3D technology for Kinect. Early Project Tango devices apparently used PrimeSense technology too, but I care less about the inner details and more about what it can provide us Application Developers. Using the Tango Service drains the battery faster than any other sensor I've ever used so I'm thinking the spirit of Steve Jobs will not let this technology land into iOS until the power problem is resolved.
Mobile devices of the future will require this 3D sensing technology so you can be sure Apple and Microsoft will be launching their own version of Tango at some point. Unfortunately for us developers we will have to make applications work with three different 3D frameworks built on mobile devices...and it isn't easy!
Fear of Missing Out
Project Tango makes me excited...like the way the untapped potential of GPS once made me excited. My excitement for GPS was ahead of its time and it's likely the case with mobile computer vision. The technology hasn't hit consumers yet and when it does it will take a long time for it to proliferate onto devices similar to the way GPS trickled its way onto your phone. So if you are a consumer app developer depending on millions of people to download your product then don't worry about it yet.
If your product targets a smaller set of users ... and specifically one that uses dedicated tablets ... then perhaps it's time for you to think about integrating this technology.
- Consumer Retailers that provide a Tablet loaner so customers may find a product or experience a virtual world
- Field workers doing inspections
- A custom product that integrates the mobile device as its brains and sensors
The latter being my current experiment in the form of a mobile cloud robot for the home.
Perhaps I'm just a nerd that loves technology, but watching MV4D's promotional video might make you excited too.