Google's speech recognition initiative - an interesting journey.…

You’ll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can’t say for sure that it will live up to the potential, but if it can, it could be something pretty special in the new field of intelligent push. Of course, Google didn’t build this product alone, there was a long, strange journey to get here. Where Siri was a product that had an identity when Apple bought it and reimagined it for iOS, Google Now has been something of a Frankenstein monster pieced together from a number of products - some built in-house, some acquired, and some built by personnel hired specifically for the job.

There is a bit of an argument about this whole process, with some claiming that Google Now is better than Siri because it was built “from the ground up” by Google (which isn’t wholly true), whereas Apple just bought Siri (which also isn’t quite correct). It’s all a silly argument based on geek pride, but it did get us looking into the process by which these products came to be what they are. 

Of course, what these products are is something of a trouble spot as well, because people insist on pulling both under the heading of “personal assistant”, and trying to compare them as similar. Such comparisons fall flat for us because “personal assistant” is too general a heading. A bike and a jet are both “transportation devices”, but that doesn’t mean it’s a good idea to compare them. In this case, Siri is a tool for finding information and organizing your life, which is based on active participation. Google Now is a tool not only for finding info and organizing your life, but more importantly it is a platform which will passively learn your behavior and interests in order to have relevant information ready for you, without the need for actively searching. Sure, there is overlap in what these two products do, but that doesn’t mean comparisons are necessary, or even useful.  

We can’t really get behind the argument that one way is better than the other, because it quickly becomes a slippery slope. The problem is that at some point, an acquired product can’t be attributed to the original creators anymore, even when the original creator comes along in the acquisition. The prime example for us is Android itself. Android was created by the Android team headed by Andy Rubin, then acquired as a fully functioning product by Google. Andy and the team still work for Google, but Android has become a wholly Google product (ignoring forks and NGAs, of course). Similarly, Siri was bought by Apple as a fully functioning product, but to say that was the same app that now anthropomorphizes the iPhone just isn’t correct.

The bit about Siri

As a standalone app back in 2010, Siri could make restaurant reservations, book movie tickets, get a taxi, find info or weather. Some of the functionality had to be removed from the app when Apple bought it, because Apple wasn’t going to use services like Bing, Yahoo, or Google for search results, but in place of that functionality, Siri got deep integration into iOS, meaning options for scheduling, reminders, calling/messaging contacts, playing music, and dictation. As Siri matures, it is gaining back some of the partnerships that Apple ditched, but gaining more valuable partnerships along the way. Although key partnerships with services like OpenTable and MovieTickets are still missing, Yelp and Wikipedia are in the iOS version of Siri, and the biggest addition is in the partnership with Wolfram Alpha as a big knowledge base. The point is that for better or worse, Siri is nowhere near the same app that it was when Apple purchased it. Sure, the backbone of voice recognition powered by Nuance is still at the center of Siri, but what the app can do has changed dramatically. What we see in iOS is no longer the product of the Siri developers simply purchased by Apple and stuck into iOS, this is an Apple product with features and uses dictated (no pun intended) by Apple.

Now… Google Now On the other hand, Google didn’t buy one singular product and transform it for Google Now. As we said, this product/platform is more of a Frankenstein monster made up of a number of acquisitions, hires, and in-house development that can be traced back at least 5 years. In many ways, Google Now is an initiative similar to Google+, which is intended to be a platform to unify a number of existing Google products and services, including search, places, travel, suggestions, and of course speech recognition.

Mike Cohen

Where Apple has partnered with Nuance and its Dragon software for the speech recognition behind Siri, Google has taken the last 5 years to build its own speech database. That project began with the hiring of Mike Cohen in 2007. Cohen was actually a co-founder of Nuance. He spearheaded the Google initiative to build a speech recognition database, which began collecting data through GOOG-411 in 2007 - the free place information service Google once ran - before getting a ton of data through dictation and voice actions on Android over the past 2 years or so, not to mention contributions from Google Voice and its terrible voicemail transcriptions (which have of course gotten better as Google’s voice recognition database has grown). Cohen left Google earlier this year, but the speech recognition work will go on.

On the other side of the voice coin, Google also needed a nice voice to respond to users just like Siri does. Luckily, Google had purchased Phonetic Arts back in December of 2010. That purchase was originally made to make robo-voices sound better in Google Translate, as well as the accessibility text-to-speech option found in Android. Phonetic Arts did a lot of work making robo-voices sound better, and that technology has come in very handy with Google Now, because it does sound like the assistant voice is smoother than Siri. As the voice database was growing to a sufficient level, Google also needed to beef up search results.

If you hadn’t noticed, Google has been slowly transitioning from a search engine into more of a knowledge provider. More and more when you search, you’ll not only get the search results, but an actual answer to your query, such as word definitions, sports scores, movie showtimes, and flight information. While some of that knowledge has come through partnerships and searchable content, big pieces of the Google Now puzzle came through acquisitions

Flight information is gathered via searchable results from flightstats.com, but eventually, Google could move to its own information repo which it purchased in the form of ITA Software (acquired July 2010). ITA Software now powers Google’s Flight Search for tickets, and that could easily become part of Google Now’s results. Place results, which are a huge part of any mobile search product (because more and more “mobile” really just means “local”) have also gotten a boost from a couple choice acquisitions. Google hired the entire team behind Ruba.com (May 2010), an online travel guide, in an effort to give local results a boost. Then, more recently, moves to purchase Clever Sense (December 2011), makers of local recommendation app Alfred, and restaurant guide Zagat (September 2011). Google notably had a falling out with Yelp, and the purchase of Zagat was the answer to that loss of local data.

Possibly the biggest acquisition of all was Metaweb, which Google purchased in mid-2010. Apple of course has to partner with information services like Wikipedia, Wolfram Alpha, and Yelp, but it’s Google’s business to provide organized data and knowledge to users, and Metaweb is the future of that endeavor. Metaweb just recently made its debut within Google in the form of Google’s Knowledge Graph and Semantic Search. Now, rather than searching for keywords, we’re searching semantic objects, which means there should be better differentiation between homographs (so Google now knows whether you’re searching the meteorological “thunder” or the NBA Thunder.) These results have shown up in the Knowledge Graph block to the right of standard results.

These blocks of information have all made the pretty easy transitions into being “cards” in the Google Now UI. And, on that topic, we have to mention the hire of Matias Duarte in May 2010, who has been one of the strongest forces behind getting the traditionally engineer-oriented Google to make well-designed products. Matias has always loved the “card” metaphor, and so we feel pretty safe in assuming the cards in Google Now were his idea. For those of you that ever used webOS, you know that cards were a central metaphor of that platform (designed by Matias), then we saw the multitasking menu of Android change to cards when Matias arrived and now we get the cards of information as part of Google Now. 

Matias Duarte

The information side of Google Now is pretty strange to follow, but the learning side of the platform is actually pretty easy, because it uses two things that Google has been cultivating for a long time now: web history and location services. Web history encompasses both your search history, and any visited pages from those search results, where location services is obviously looking at where you spend your time and where you travel. If you have an Android phone with location services enabled, Google probably already knows where you live and where you work, and all of that data can easily be found on your Latitude dashboard (note: this is all private data that only you can see, Google doesn’t share it.) When you combine that location data with your personal calendar, and Google’s ever expanding navigation options for driving, public transport, walking, and biking, you get a pretty powerful data set that only needs a bit of a nudge to predict what information you may need at a given time and place. 

Conclusion

We can’t say that Google Now was completely built in-house by Google any more than Apple built Siri in-house. Both took very different development paths because of the nature of the product, the nature of the respective companies, and the resources available to each company. Google is a software company that has been working hard for years to learn who you are (as an abstract entity, not as an individual), while Apple doesn’t have that information to leverage and had to build a product that can essentially start to harvest that data. As we said before, we fully expect Siri to continue its transformation, and eventually become an intelligent push service like Google Now, but whatever the end point, the journey of Siri becoming a fully realized product is much cleaner than Google Now. Apple has always tended to work from a centralized format, creating one singular product designed for a purpose within the larger whole of the Apple ecosystem, while Google is a far more messy affair.

Google captures various services and products for a number of different reasons, and may not always have an overarching plan to bring everything together. Since Larry Page has stepped into the CEO office, that has begun to change, but it can still be a pretty long and windy road leading from products, acquisitions, and hires into a cohesive platform like Google Now. 

37 Comments

• 1
• 2
• Next
• »

1. hateftotti posted on yesterday, 12:55 17 5

Google Now»»»»»»»»>siri
Just wait to see how apple want to mislead its fans again.

2. MichaelHeller posted on yesterday, 13:03 19 0

That’s like saying TV is much better than radio. They are two different things that have different purposes, but have a small set of overlapping features. The comparison is silly.

6. Droid_X_Doug posted on yesterday, 13:39 0 0

I wonder if the evolutionary path doesn’t reflect some of the philosophy of the O/S environments in which the products function? Siri is running in an environment that if there is going to be deep integration, requires the development work to be done by the iOS team. Conversely, given the more open environment of Android, more things can be done by ‘outsiders’.

8. MichaelHeller posted on yesterday, 13:46 0 0

That’s probably part of it. Google has more leeway to fold in external products as needed, and has APIs available for the rest. Surprisingly, there’s no real Google Now API, although the other APIs like Directions, Places, and Calendar all feed into Google Now.

9. Aeires posted on yesterday, 13:48 1 3

Both are personal assistants, to remind you of events and pull information you require. How is that different? The only big difference I see is one is far more intuitive than the other, but at the end of the day, they’re both personal assistants.

11. MichaelHeller posted on yesterday, 13:59 3 0

There is huge variation in the space of “personal assistant”. A to-do notebook can be considered a “personal assistant”, so can an actual human helper. The term “personal assistant” is so general as to be useless. To go back to my original analogy, that’s like saying TV, radio, MP3 players, etc are the same because they’re all “media devices”. A bike and a jet are both “transportation devices”, but that doesn’t make it a good idea to compare them.

17. Aeires posted on yesterday, 14:37 2 2

Except both are apps on a cell phone. Comparing a TV to a radio is silly, but comparing one app on a cell phone to another app on a cell phone that essentially does the same thing isn’t silly.

18. MichaelHeller posted on yesterday, 15:06 3 1

GTAIII and Angry Birds are both cell phone apps that “essentially do the same thing” (entertain). Shall we compare them? How about Hookt and Skype (communication)? How good will those comparisons be? It all depends on how specific your comparison criteria are. If you want to compare “personal assistants”, you won’t get very good comparison points because of the huge variation in methods and features across the thousands of apps that fall in that category.

For example, you could compare WriteRoom and Office, because they are both “productivity tools” with an overlap on word processing, but the comparison is going to be pretty useless.

28. Aeires posted on yesterday, 17:06 3 1

How about just the two that the article is about. I’ll be blunt with this so there’s no misunderstanding the point. I don’t think Apple put near as much work into Siri as you make out. Apple purchased a finished product and tweaked it to fit their needs, hardly deserving the credit of making Siri. As for Google, a Frankenstein approach isn’t that too far off to imagine. Google is well known for trying crazy things and sometimes they nail it, like with Google Now. If you look at everything that went into Now, it’s remarkable, and that shouldn’t be downplayed because they purchased some portions and hired the talent needed to pull it off. All the resources still had to be put together, which was no small feet.

But at the end of the day, they’re both personal assistants. The graphics in this article says “Hello, I’m Siri, your new personal assistant.”

10. itiswhatitis posted on yesterday, 13:55 1 2

If comparison is silly then i guess 99% of the World’s population is silly!

12. MichaelHeller posted on yesterday, 13:59 11 0

This is why my job is difficult…

13. itiswhatitis posted on yesterday, 14:04 0 0

I guess its like you said from one of you articles you must be throwing ‘millions of pebbles’ whatever it is good luck with that,its worth the try….

22. remixfa posted on yesterday, 15:29 11 0

ok.. how about this for a quick metaphore.. if you think of Google’s dozens of services as “cities” unto themselves, they are much like the US on a map.. cities here and there and everywhere. What Google Now does is basically build information highways between all these “cities” so they can now work together. Google Now takes all the information from all these cities (Maps, Search, ect ect ect) and uses it to figure out what you want… and eventually it might know what you want before you do. It watches you while it watches the cities and learns over time. Its pretty much free to grab any information from nearly anywhere from the way its designed. Apple is the opposite. They have one massive city where everything is done. Siri was build IN to the city to work inside it. It does not go outside the city walls as designed. As Apple’s city grows, Siri will grow with it. Siri takes all the information within Apple’s city and keeps it ready for you when you need it. It however will never go beyond the scope of Apple’s city. Some of the base features of both may be the same, but they operate completely different and they are built to do different things from each other.

We have joked that Siri was the beginnings of Skynet. In all honesty, Google Now is way closer to Skynet that Siri will be.. maybe ever. Its always watching and learning what you do. Compared to that, Siri is a rockem-sockem robot.

23. good2great posted on yesterday, 15:40 4 0

beautiful explanation remixfa!!! this lets me know that you are a very unbiased person and at least understand how both OS’s work… this definitely doesn’t take away from Michael H. article you pretty much explained it to educate the trolls in here…lol good article Michael H.

there are a few readers in here that actually enjoy the content of this site than coming here to crack jokes and bash/harass each other about which OS is better…

36. remixfa posted on yesterday, 23:06 0 0

Thanks :)
Yea, I never meant to take away from michael. only help reduce his headache from trying to make people understand. If it helps someone, then it was worth typing it up.
I dont know that true trolls can be educated though. Isnt that against the trolling Alma mater? lol

20. MartyK posted on yesterday, 15:13 5 1

I disagree with ALL; the difference is this: Ifan feels everything Apple market dept says/ puts out is pure-first -time-ever-magic (from Touch screen to Siri). Whereas, the rest of the world KNOWS that every thing is ( in Mike words) peice together, heck the whole COMPUTER language is peice together so there is really nothing new.

Except when it comes to Ifan and Apple, then it’s brand new, SOMEBODY have to call them out on it.

3. tedkord posted on yesterday, 13:07 7 4

But…I thought Google didn’t even think of an intelligent talking assistant until Siri, then they just copied.

7. smartphonemad posted on yesterday, 13:40 2 0

That’s what Apple fans want people to believe :L I mean, with all the people that create ideas for google do you not think someone would have thought of it?

14. protozeloz posted on yesterday, 14:25 0 0

are sarcasms so hard to get these days?

16. MichaelHeller posted on yesterday, 14:31 7 0

This is why we need to standardize punctuation for sarcasm on the web. So many people miss it in the written word. Easiest is just bracketing punctuation to convey sarcasm, since it wouldn’t need new keystrokes to be learned.
ie
Fanboys are so rational[!]

• 1
• 2
• Next
• »

Want to comment? Please or .

Home  › News  ›

 

How Google Now was built (and a bit on Siri too)

0. phoneArena posted on yesterday, 12:47

You’ll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can’t say for sure that it will live up to the potential, but if it can, it could be something pretty special. Of course, Google didn’t build this product alone, there was a long, strange journey to get here…

This is a discussion for a news. To read the whole news, click here

Post Comment

• «
• Prev
• 1
• 2

19. protozeloz posted on yesterday, 15:08 2 0

sarcasm standard would be kinda hard since there are a few keys used for sarcasm but my favorite is “/s” ie

phonearena is full of rational fanboys /s

29. jroc74 posted on yesterday, 18:23 0 0

tedkord shoulda added 2 more “but”s…lol.

15. Sniggly posted on yesterday, 14:27 0 0

my apologies, man. I meant to thumb this comment up. It was a good snarky little comment. :)

27. tedkord posted on yesterday, 16:57 0 0

Yeah, it was snarkier than I intended. I’ve been working 13 days straight, 12 hours per day, 1 day off and start over, for several weeks now. I am exhausted, and maybe a little cranky.

21. appleDOESNT.com posted on yesterday, 15:17 3 1

Siri isn’t even the first virtual assistant. your point is pointless. Anyways, Apple bought Siri in 2010 as a response to Google voice dictation and voice actions that debuted in 2009, no apple voice competitor until late 2011, years later.. it took Apple two years to launch Siri… Google Now is an extension on every thing search and they didn’t just whip that pure awesomness in 6 months… If they did? that is a company I like to back!!!

Apple let Google run voice wild for years without a competing product and even longer without Navigation. I like my companies to react and innovate quickly not take years to offer MMS and copy n paste!

30. jroc74 posted on yesterday, 18:29 1 0

lol! +1 to ya… I REALLY, REALLY like this article. Apple and Google took different paths for each product. Whoever says Google is copying Apple/Siri….needs to just play in traffic….sorry. Ok…I only 1/2 mean it…kinda.

Articles like this ….Apple, iPhone fanboys need to really pay attention to. Apple isnt the only company that innovates. How hard is that to understand?

4. sid07desai posted on yesterday, 13:17 8 0

Google is more credible for Android than Apple for Siri. When Google bought Android, it was just in the beginning stages. And then they came up with improvements like Ice Cream Sandwich on their own. And then they created Google Now. Their own creation. Apple did little to make certain changes to Siri but overall it cannot be credited to Apple. As far as efforts are concerned, I think Google has done a better job. But as Michael mentioned in the above comment, the comparison is silly. We users can only wait and see how these evolve on their own.

31. jroc74 posted on yesterday, 18:34 0 0

Agree 100%. At its core…Siri was already developed before Apple bought it. Google Now wasnt. Thats one of the big things about Apple that Apple, iPhone fanboys dont wanna admit. If Apple is innovative for buying companies, technology, apps and adding their twist to it….how can anyone say Google isnt?

It boggles the mind.

5. maxican16 posted on yesterday, 13:21 8 0

The important thing is what works best for each user. For me there is no question, it’s Google Now. I don’t want to have a conversation with my phone, nor do I want to ask the same question time and time again (just have it ready for me already!), or be given incorrect answers (which happens a third of the time on siri). Google Now is already incredibly functional and it will only improve. And when voice search doesn’t have an answer, it gives me a list of Google results that only improve the more I use my phone. Take note Apple… THIS is true innovation.

24. good2great posted on yesterday, 15:44 3 1

what would be funny is if Google released an iOS version of Google Now…lol you guys would be pissed huh? lol

looking at how its broken down its not impossible to happen… lol

25. Sackboy117 posted on yesterday, 16:10 4 0

it would probably become #1 on the App Store and would most likely reach out to previous iPhone models.

33. SBrown posted on yesterday, 21:41 1 0

I usually don’t participate on these comment threads, but I must admit, that was an excellent civil comeback. +1

26. protozeloz posted on yesterday, 16:18 0 0

is quite posible at least from goggles side of things

34. MartyK posted on yesterday, 21:49 0 0

It wouldn’t surprise anyone if Google did place this on IOS or Window phones, in fact most Android Users wouldn’t be upset at all.

Until, IOS Market dept and some crazy Ifan, start screaming out, “Google copy” and then Apple trieds to patent Google now.

37. remixfa posted on yesterday, 23:10 0 0

there are not enough hooks in the “google” apps on iOS to make it work the same. With Android, it comes with all the apps built in to the system. On iOS, not only would it take major app upgrades (like Maps for example.. huge difference between goog maps on android and iOS), it would also require that all those different Google apps be downloaded or preinstalled in iOS with additional hooks built into the OS itself (like Apple would allow that).

And you think the jokes about iOS becoming more “android” with each update is bad now? Just imagine what kind of jokes would be slinging if iOS had a mandatory “google apps” suite on it. lol. those jokes would never end. :)

32. satydesh posted on yesterday, 19:17 0 0

android fans are really aggressive thats Power given by Open source to demand more!!!

38. dreign91 posted on 11 hours ago 0 0

Michael H I must say I feel a tad bit smarter every time I read your articles. I’m very excited for Google Now as well. Yes in the beginning there will be a learning curve, but with all the sources of info your phone can pull from (Gmail, currents, play store, search, places, etc) this has the potential to be legend….(wait for it)…. dary! What will be truly awesome is when Google now is put together with the Google Glass project

 

google.com

via http://www.speechtechnologygroup.com/speech-blog - Google's speech recognition initiative - an interesting journey.… You’ll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can’t say for sure that it will live up to the potential, but if it can, it could be something pretty special in the new field of i ...