How Does Google Maps Do It?

[Season 2 • Episode 22 • published 10/26/23]

Every month, over a billion people open their phones and fire up Google Maps. Its original function—offering driving directions, with real-time traffic tracking—was disruptive enough in 2008, when most people had to pay $10 a month for traffic data. But since that time, it’s become a global business directory, a transit timetable, crowdedness monitor, a Street View miracle—and now, in its newest release, an augmented-reality viewer of the cityscape around you. The question is: How is Google doing it, and why is it free? Meet the man who runs Google’s entire Geo division.

Description

Intro

Theme begins.

Google Maps offers driving directions, of course… but it also knows subway and bus schedules, the hours and phone numbers of businesses, photos of 98% of the world’s inhabited surface, and a lot more.

PHILLIPS: It’s a magical experience and it’s so valuable. It’s just a phenomenal advancement in technology and and a utility that, really, so many people just can’t imagine living without.

This is the guy in charge of Google Maps…and my ticket to finding out the answers to everything everyone’s always wanted to know! Which is better: Google Maps or Waze? How does Google Maps know the traffic speed of every road? How does it know how busy a store will be next Thursday?

Today, we’re going to find out. I’m David Pogue. And this is “Unsung Science.”

First Ads

Season two, episode 22: How Does Google Maps Do It?

At one point in season 1 of this podcast, I was interviewing Luis von Ahn, the creator of Captcha, that website-access test. And during the conversation, he volunteered this opinion:

Luis: By the way, you know, like Google Maps is an amazing piece of software, by the way. I think it’s one of the biggest, you know, GPS and something like Google Maps is one of the pinnacle of achievement of humankind.

David: We’re doing an episode of this on Google Maps!

Luis: Yeah. It’s amazing. The whole thing.

One of the pinnacle achievements of mankind, huh? A phone app?

And yes—I have wanted to go behind the scenes of Google Maps since the day I dreamed up this podcast. And today is the day.

Google Maps is one of the most widely installed apps ever made. As of 2021, this app was on ten billion phones. A billion people a month use it. A billion.

As you probably know, Google Maps is great for giving you driving, or walking, or biking directions. But from its first appearance on phones in 2007, it’s aspired to be a lot more than that. Here’s me in a New York Times video, gushing about the iPhone app when it debuted:

DP: It’s the new Google Maps app for iPhone. The design is clean and modern. Street View lets you see what any address looks like, and Compass view lets you look around someplace without actually being there. Everything is built in: Points of interest, reviews, photos, even making reservations, and the public transportation feature is just amazing.

Now, if Luis Von Ahn is right, that this app is one of the pinnacle achievements of mankind, I think this is why: Because the whole thing is an inconceivably huge exercise in data wrangling. A maps app requires a massive electronic database of road maps; another database of satellite photos of the whole world; a third database with details about businesses; and so on. If you’re Google, you have to stitch all of these data sets together seamlessly.

That’s hard, but not as hard as keeping it all updated. Forever. This is the part that makes me nearly pass out to contemplate. You can never rest. I just—I just don’t know how that could be possible. Thousands of businesses open and close every day. Roads are built. Roads are washed out or closed for construction. Buildings go up. Buildings come down. Trying to keep on top of all that sounds like my version of hell.

PHILLIPS: I love to talk about the map being alive, you know. So, David, think about —it’s changing all the time.  It’s—roads can change. I mean, think about that four-lane road that now has two lanes because now there are cycling lanes added to it. Even speed limits change.

Chris Phillips is the general manager of Geo, which is Google’s name for the division that develops geography products, like Google Maps, Google Earth, and Waze.

PHILLIPS: So how do we keep these things up to date and businesses change? What was a coffee shop is now a laundromat is now a flower shop.   What we do is, in order to build a map, we start with—think of it as a layer cake. You know, we are putting together all this imagery that we captured from satellites in our aerial imagery and Street View imagery, and then we start to overlay really important data. And that data we get from what we collect first party, we also work with, you know, thousands of local authorities and transit and mapping services to understand the information that’s out there about a place and the road network.

Quick glossary of terms here: “first party” refers to data that Google has collected itself.

And you also heard him mention Street View—I mean, what a mind-boggling project. It’s a feature of Google Maps that lets you look around at any address. It shows you 360-degree, interactive, panoramic photos, so you can see how fancy or how sketchy a place is before you go there—or without ever going there.

And where did these photos of every address on every road come from? They were captured by Google Street View cars, with 360-degree cameras on the roof, driving every road in every state in the US. And a lot of other countries, too. And where the Street View cars can’t go, they’ve used bikes, trikes, camels, boats, and snowmobiles. They’ve been driving around, photographing the world for years, and they’re still at it. At this point, Street View includes 280 billion photos from 100 countries—and over 10 million miles of roads.

We’ll come back to Street View. But Chris was describing how they handle the notion that the world’s geography is constantly changing…

PHILLIPS: We also bring people into the mix. So our community of users are actually constantly providing information that’s helping us keep, “hey, I think the hours of operation are wrong on that business. Let’s edit it.”

Yep. You, too, can make corrections to Google Maps. Look for a button called Contribute or Suggest an Edit. And people do this a lot. Like, 300 million people send in 50 million suggested edits a day. The superest of the superfans are known as Local Guides.

PHILLIPS: I get stopped at dinner parties, like, “I’m a local guide at this level. I’ve contributed this many place reviews. I am the expert in my neighborhood, at restaurants.”

POGUE: Yeah, you made me realize there must be, like, Google Maps annoying nerds who are very enthusiastic.

PHILLIPS: Never annoying, just passionate.

POGUE: I remember reading an Atlantic article maybe ten years ago, back when it wasn’t so much machine learning, but it was a room full of 300 people or something, manually entering all these changes about the streets and the shops into Google Maps just to keep it current.

PHILLIPS: We still very much have our operators that help us make sure the quality of the information is accurate. Think of the advanced AI and machine learning as getting so much more coverage in scale than you could get from a person. And the people are actually helping us tune it and make sure it’s right.

Google also uses an army of AI bots to call up businesses and interview them over the phone.

PHILLPS: It’s called Duplex. It’s part of our Assistant team, calling millions of businesses and checking in with them with an automated service that says, you know, “are you open and closed? Have your hours changed?”

Now, we start the conversation with getting permission, like, “Hey, we’re calling from Google. This is an automated service.” The idea is that we get permission and then you just have a quick conversation.

And it’s amazing that we can get the latest hours. And even if there were COVID restrictions or do you have accessible bathrooms. Like, things that are really important to keep in that place up to date?

So this is not just great for consumers, but it helps businesses. Over 250 million businesses around the world.

POGUE: If I report a change using that little Contribute button, what stops me from deliberately trying to mess you up by making it wrong?

PHILLIPS: We are using all these advanced techniques to really understand the trust level we have with the feedback. In some cases, depending on the types of information you’re contributing, we will let things through quickly, or we’ll double, triple check our work. And so the moderation technology can have trust built into letting things flow through.

So it’s a really, really great example of, where you have the machines that can help us receive these, you know, tens and tens of millions of edits, while also using the people to help check it.

Remember, I said that we’d be coming back to Street View? Turns out that photographic record of every street in the developed world has a big payoff to Google Maps in another way, too.

PHILLIPS: Street View is such a cool story. We are using it for so much more than just seeing what a place looks like. So when we are driving, we are also capturing, like, on that street corner, is that still a laundromat or is it now a coffee shop? We are using it to capture changes on street signs. Now it’s something that we use to actually help us update the map.

Are you starting to get a sense of just how much data is involved in maintaining a representation of the entire planet in an app on your phone? And then keeping it updated?

Those Street View cars will never be done. They’ll be driving forever, just to keep up with the changes.

So, we now know how they’re keeping the road maps up to date. We now know where they get the photos of the buildings. We know that they get the subway, bus, and train schedules directly from the transit authorities in 800 cities.

But how on earth does Google Maps know the current traffic speed on a given road? It’s not like they’ve implanted sensors into the pavement all over the planet. And don’t forget, the app also shows you how crowded a restaurant or shop or museum is right now, or will be at a certain time in the future. How does it know that?

Turns out the answer to those two questions is, in fact, the same: Google can see the positions of people’s phones on that road, or in that restaurant. It measures the movement of those phones, and presto—there’s your traffic speed. It tallies how many phones are in that restaurant, and voila—there’s your crowdedness data.

Now, Google is not tracking you. Your phone’s position is anonymous and aggregated, meaning it’s dumped into a pool of data with everyone else’s phones. And even so, you can opt out of contributing your phone’s position.

The point is that Google has gotten to the point where it doesn’t just know what the traffic or crowdedness is now; it can tell you how bad it’ll be at some time in the future.

PHILLIPS: We take a historical information around what was in, you know, the trends and the patterns that happened before on a Saturday morning versus a Monday morning. You know, that same– what were the historical patterns of traffic for the people who were driving at those times aggregated and then were able to overlay, okay, what’s happening right now?

I just gotta mention this one YouTube video I saw: It was a guy who wanted to demonstrate how Google calculates traffic speed. So he piled 99 smartphones into a wagon—like, a kid’s red wagon, with a handle—and he walked along an empty street in Berlin, pulling the wagon behind him. And sure enough: Google Maps reported the traffic as his walking speed. When he stopped in the road, the street turned red, indicating a traffic jam. He literally tricked Maps into rerouting cars to avoid his one-man traffic jam.

POGUE: But here’s—here’s magic. This is real magic. I and other people have noticed that when I’m entering an address, I have to put in like one digit, and Google Maps proposes the address I want. How does it know? It saves me so much typing. How is that possible?

PHILLIPS: So it could be defaulting to things that are nearby. You also could be pulling up recent searches that you may have done.

But we understand even popular addresses. So what could be really popular as a place that people might be going?

So there’s a number of different techniques, and it’s not always one thing, which is part of the magic.

And, by the way, magic that doesn’t cost you anything. I’ve always wondered about that.

POGUE: How is it that this could possibly be worth it to Google Inc to develop this amazing app and then give it away?

PHILLIPS: We’re able to monetize with advertising when appropriate. If you’re searching for coffee shops near me, you might get a few coffee shops that are trying to promote themselves in the list of ones that are available.

POGUE: Do you make any revenue from licensing the data to corporations?

PHILLIPS: We do offer it to businesses and enterprises. That’s the other part of my job in the GEO portfolio is, some of your favorite apps have a map inside. Most developers that use it don’t pay. So there’s a pricing approach that keeps it free for most. And, you know, depending on how big you get, you can start to pay.

So: At this point, Google has been perfecting and enhancing Maps for 18 years. Every year, it adds more features, and only rarely drops features.

You gotta wonder: At this point, how much more useful stuff is there to add? How is it possible to come up with any more features in an app that’s already packed?

After the ad break—you’ll find out!

Second ad break

Before the break, we were talking about what a massive job it is to gather and maintain the data that describes the roads, the signs, the speed limits, the traffic speeds, the photographs, the aerial views, the business hours, the accessilbility features, and the crowdedness of 250 million businesses. Spoiler alert: A big job. Very big.

Which leads me to one of my favorite stories in all of tech.

When the iPhone came out in 2007, there was no app store. You got 15 preinstalled apps, and you liked it. And one of them was Google Maps.

But a year later, Google introduced its own iPhone lookalike phones, Android phones, and Steve Jobs was furious. He insisted that Google had ripped off the iPhone’s design. And as tensions rose between the companies, Jobs couldn’t help noticing that Google had blessed the Android version of Google Maps with important features that it didn’t give to the iPhone version—like turn-by-turn driving directions and vector graphics—that is, sharp instead of pixelated. Google was giving preferential treatment to its own phones!

So Jobs decided to dump Google Maps from the iPhone. No way was he going to give such prime iPhone real estate to Apple’s arch rival!

He commanded his team to create its own app—to be called Apple Maps—to replace Google Maps. And sure enough: in 2012, Apple Maps appeared on every iPhone instead of Google’s app.

And how’d that go?

Here’s what I wrote in my New York Times column that year:

“Maps may be the most embarrassing, least usable piece of software Apple has ever unleashed. In this new app, the Washington Monument has been moved to a new spot across the street. Search for Cleveland, Georgia, and you’ll wind up right smack in Cleveland, Tennessee. Riverside Hospital in Jacksonville, Florida, is in the right place but the wrong decade; it became a Publix supermarket 11 years ago.

Entire lakes, train stations, bridges, and tourist attractions have been moved, mislabeled or simply erased. Satellite-photo views consist of stitched-together scenes from completely different seasons, weather conditions, and even years.

The Brooklyn Bridge has melted into the river, the road to the Hoover Dam plunges straight down into a canyon, and Auckland’s main train station is in the middle of the sea.”

The problem was that Apple bought a couple dozen off-the-shelf databases—map data from TomTom, restaurant and store listings from Yelp, traffic data from Waze, and so on—and then just tried to jam them together.

To be fair, that’s how Google Maps got started, too, way back in 2005; Google bought a bunch of similar databases. But Google spent years massaging, perfecting, and supplementing that data—years that Apple didn’t have to spend.

The Apple app was so bad that Tim Cook issued a public letter, apologizing for the disaster and suggesting, I kid you not, that people use Google Maps instead!

Anyway. That was 11 years ago. Water under the Brooklyn Bridge!

Both Apple and Google have vastly improved their apps since then. Over the years, Google has added mapping of the moon, some planets, the Space Station, trails, college campuses, the Great Barrier Reef and the underwater coasts of Hawaii, and even indoor locations like malls, airports, and train stations. In 2012, the app started letting you download maps to the phone ahead of your trip, so you could still use it without an internet connection. In 2017, it started remembering where you parked.

A couple years ago, Maps added an augmented-reality feature, where you come up out of the subway, all disoriented, and Maps displays these giant floating arrows on the phone’s screen as you hold the phone upright in front of you.

POGUE: When I come out of a subway, I always use that live view thing that shows me with augmented reality with arrows which way to start walking.

PHILLIPS: I love that feature. That is so cool. That’s so helpful to orient you when you walk up, which is an example of what a two-dimensional map really always struggled to do.

Today, in fact, the very day of this episode’s release, Google Maps has a major new release.

So, first of all, I’ve always told whoever would listen that Google Maps is actually a stealth weapon against the climate crisis. I mean, think about it: The app’s primary effect on drivers is to get them directly to their destinations as quickly as possible. People don’t drive around lost anymore. Fewer miles driven means fewer emissions.

But lately, Google has been expanding the app’s pollution-saving features.

PHILLIPS: We have a new feature that’s called Eco-friendly routing. So when you’re looking at, ”OK, I need to go from here to there,”  in the instance where there’s a different route, we will show you a route that could be two or 3 minutes longer to get there, but actually uses less energy. And this is using advanced AI techniques, partnering with energy providers to really build out models that say, “hey, how many stop/starts are there? What’s the elevation happening?” Different conditions that make a route actually use more fuel. You could save money on fuel. You can have less carbon emissions in the real world.

Maps could already show you where the charging stations are if you have an electric car. But now, those icons also indicate the last time somebody used a charging station.

POGUE: How does that help me, if it says the last time it was used was just three months ago?

PHILLIPS: Well, that could be an indication that it might not be functioning. If it was used recently, you have a higher confidence that it will be working.

But maybe the coolest new feature of all—is something called Immersive Routes.

PHILLIPS: This is this is one of my favorite new things that we’re doing. It’s bringing to life a photorealistic view of what that journey is going to be like, whether you’re walking or riding your bike.

it’s like you’re flying over. You’re flying over the route and you’re able to see based on the time of day, what the traffic might be like, what the air quality is, what the weather is. And that is powered by fusing together satellite, aeria,l Street View imagery, overlaying all of this real time and predicted data sets into the experience.

So my wife and I were recently traveling, and we wanted to go from our hotel to a shopping district, and we were going to walk. We were able to fast forward to 4:00 that same day and see what the weather conditions were going to be like. And actually it was going to be raining. So we decided to go on the walk earlier.

In other words, it shows you an animated, overhead, photorealistic view of your intended path. They’re starting with this feature in about 50 major cities. This new feature is yet another payoff of all those Street View panoramas that Google’s cars have been capturing all these years.

PHILLIPS: It’s so cool because it shows, like— Is it dark out? Is it sunset? Is it raining based on your positioning? And it stitches all that together so it simulates what it would be like to live later today versus right now.

And, you know, when you’re deciding to, like go on a bike ride, what the weather’s going to be like or how busy the traffic is on the road, what places are along the way. This is an amazing way to kind of check out before you go. And you can kind of fast-track through the route to check it out or —

POGUE: I was gonna say— if it was real time, that would not be a good feature.

PHILLIPS: That would not be a good feature.

I give a lot of talks. And about six years ago, I used to include, in my technology talks, a couple of demos of augmented reality. That’s where the phone superimposes graphics on its camera view of the world around you, providing more information about what you’re looking at.

And I’d always fantasize about where augmented reality could go someday. You could scan a bunch of apartment buildings, and little arrows would pop up to show you which apartments are currently available to rent, and for how much. Or you could point your phone at a restaurant, and see a list of its health violations.

Well, Google hasn’t quite done that. But it’s started down that path with something called Lens and Maps.

PHILLIPS: You can lift up the phone and hit the camera search icon in Google Maps, and it will take and we represent what you’re looking at.

We use that street view imagery and so forth, and we actually build a view of what you’re seeing in front of you. And we overlay all the places. So you see like, oh, ATM shop, barber shop, another restaurant, a bar: the ratings, are they open, are they busy?

So it’s this example of really immersive, photorealistic information coming to life right on front in front of you.

I have to say, it was cool having the head of Google Maps and Waze on the horn. I could ask anything I wanted. I could even settle the old Maps-vs-Waze debate. Google owns both apps, but they look and work differently, and still sometimes provide different directions.

POGUE: Everyone always says, “Oh, I use Waze for navigation.” “I use Google Maps. It’s better.” Is one of them better?

PHILLIPS: We love that people are so passionate.  You know, they offer two different driving experiences. One, you know, Waze is like, you’re getting maneuvers that might be a little more aggressive, and you’re you’re beating the traffic.

And Google Maps has a different personality. You know, it’s more quietly there navigating you along, giving you the right audio updates, routes that, you know,  Sometimes they’re a little different.

And you know what? There’s a lot of people that use both. David, I’m one of them. I’ll have different scenarios when if I get into a heavy traffic situation, I or I know it’s gonna be really busy. I might check out Waze, because I really want to, you know, get around this traffic jam in a creative way.

POGUE: Well, wait a minute. It sounds like you’re saying that Waze might feel better, at least in traffic situations.

PHILLIPS: Well, we both offer different options and alternatives. Our teams are really passionate about keeping both brands in the market and having loyal users have that choice.

POGUE: I’m just not going to get you to say which one is better at navigation.

PHILLIPS: I mean, we’re in the family together, you know.  And we we do we do care about both, both teams in the products we do.

He did say that Google is beginning to cross-pollinate the two apps. Not with features or design, but with data, like speed traps, speed cameras, road obstructions, construction, and road conditions. Stuff that used to be available only in Waze is now showing up in Maps.You are the guy ultimately in charge of this iconic piece of software that a billion people are using a day. / do you dominate cocktail parties? I mean, do people come up to you with questions, like, “how come this button…?” Is it a blessing or is it a curse?

PHILLIPS: David, It’s— I feel extremely blessed. And here’s the thing. I am representing so many great people that have done so much great work for so many years. And we have team members that have been working on this for so many years and have been here from the beginning, and then so many new leaders that have come in and added new perspective and helped us take it forward. So I see myself as just helping lead.

But yes, back to your dinner party. I get asked all the time about, “OK, here’s what I love, here’s what you got to fix, okay? I need you to integrate with this. Do that. Let me show you.”

And then depending on the audience, depending on how charged the feedback is, I might turn around, say, and, “Did you check out this feature and check out this?”

So my, my, my wife will often kind of have to pull me away. She’s like, “OK, that’s enough.” But yes, I get a lot of feedback. I welcome it, and I think my family probably wish that it wasn’t always the case.

POGUE: Well, I’ll tell you what. You and your wife are invited to our place for drinks and dinner next Saturday night at eight.

PHILLIPS: Sounds great, David.

POGUE: I could totally geek out the whole time. And my very last question is,  do we know how much of the world Google Maps knows about?

PHILLIPS: Well, we’re operating in 250 countries, but we feel like there’s still a lot left to map. We’re not done. We have a lot more to do.

POGUE: Really?

PHILLIPS: A lot more to do. Oh, yes. This mapping job is not close to being done.

POGUE: It’d have to be, like, two-lane driveways in Montana or Siberia at this point!

PHILLIPS: Well, we have a product called Plus Codes, where we help the unaddressed get an address. And so if you look at countries in the world, places in the world where people don’t even have a physical address and helping work with NGOs and officials to give them a real address so that they can register for health care or have a voice or having emergency services arrive to them if they have needs.

There’s still so much to do, David. And that’s what keeps us going.

Podcast: Play in new window | Download