Google Self-Driving Car Project | SXSW Interactive 2016

On a related note, here's Chris Urmson's long talk at SXSW on how Google does automatic driving.[1] This is worth watching. He shows what Google cars sense and gives an overview of how they analyze the data. This is the first time Google has released this level of detail. If you're going to comment on automatic driving, you need to see this. Urmson goes into great detail on Google's 2mph collision with a bus. The sensor data from the Google car is shown. The video from the bus (it had cameras, too) is shown. Exactly what the software was trying to do is discussed. The assumptions the software made about what the bus driver would do are discussed. What they did to prevent this from happening again is mentioned. Most of the talk is about the hard cases. In the beginning, Google developed a highway driving system, but that was years ago. Now they're working hard on dealing with everything that can happen on a road, including someone in an electric wheelchair chasing ducks with a broom. [1] [this video]

That's Chris Urmson's talk at SXSW in 2016 [1], which has been widely viewed. It's one of the best talks on the subject. He said "How quickly can we get this into people's hands? If you read the papers, you see maybe it's three years, maybe it's thirty years. And I am here to tell you that honestly, it's a bit of both." Then he showed video of Google cars dealing with various challenging conditions. This included the famous scene where a Google self driving car encountered a woman in a powered wheelchair using a broom to chase a turkey. The software recognized this as something to avoid, even though it didn't classify it further. [1] [this video]

Except that Google was able to handle construction sites, traffic cones, and people directing traffic last year. Watch Urmson's video from SXSW.[1] Google has put a lot of effort into handling unusual situations. Those cars driving on residential streets in Mountain View and Austin have accumulated data on the hard cases. The California DMV continues to publish autonomous vehicle crash reports. The last one was in October.[2] A Google vehicle was rear-ended at 3mph when it was entering an intersection with poor visibility, detected cross-traffic, and stopped. I'm looking forward to the 2016 self-driving disengagement reports appearing on on the California DMV site.[3] Those were due January 1, covering the year ending 1 December 2016. Then we'll have comparative info on how all the players are doing. [1] [this video] [2] [link] [3] [link]

Cool! Thank you for sharing this. Here's the part where he starts talking about the 2mph bus accident [1]. He says the self-driving car should not have moved. He does not say why. He says they ran a lot of tests to make sure it will not happen again. In plain English, I'd say, when the SDC is stopped on the side of the road, give sufficient time for traffic coming from behind to clear out before the SDC moves forward. Also, if an accident is imminent, being in a non-moving position is probably safer and less likely to cause liability for the SDC than moving. Plus, rolling over some sandbags is probably preferable to being hit by a bus. [1] [this video]

Google does a lot of driving in simulation, using data captured by real vehicles. They log all the raw sensor data on the real vehicles, so they can do a full playback. They do far more miles in simulation than they do in the real world. That's how they debug, and how they regression-test. For near-misses, disconnects, poor decisions, and accidents, they replay the event in the simulator using the live data, and work on the software until that won't happen again. Analyzing problems which didn't rise to the level of an accident provides far more situations to analyze. They're not limited to just accidents. See Chris Urmson's talk at SXSW, which has lots of playbacks of situations Google cars have encountered.[1] [1] [this video]

Yeah, me too; I was in the 2005 Grand Challenge. Vision processing has made enormous progress since then. Here's Mobileye's guy explaining what they do and what needs to be done.[1] Here's the NVidia guy.[2] Here's Chris Urmson from Google.[3] All three use their vision system to draw boxes around things that look like obstacles. Then the planner uses those boxes to construct a path. The vision system just gives you a rectangular box in image space; the LIDAR based systems turn point clouds into simple 3D solids. These vision systems do OK on things that look like typical road obstacles - cars and people. Other stuff, not so much. Only Urmson from Google talks about that much. The videos from the vision systems don't seem to be doing much with road edge clutter. They're not marking every post, fireplug, and low-hanging branch. Train a deep learning system on cars, trucks, pedestrians, and bicycles, and your system sees only cars, trucks, pedestrians, and bicycles. These vision systems do a lot less than many people think they do. Watch the videos. [1] [link] [2] [link] [3] [this video]

What this really reflects is that Tesla has painted itself into a corner. They've shipped vehicles with a weak sensor suite that's claimed to be sufficient to support self-driving, leaving the software for later. Tesla, unlike everybody else who's serious, doesn't have a LIDAR. Now, it's "later", their software demos are about where Google was in 2010, and Tesla has a big problem. This is a really hard problem to do with cameras alone. Deep learning is useful, but it's not magic, and it's not strong AI. No wonder their head of automatic driving quit. Karpathy may bail in a few months, once he realizes he's joined a death march. If anything, Tesla should have learned by now that you don't want to need to recognize objects to avoid them. The Mobileye system works that way, being very focused on identifying moving cars, pedestrians, and bicycles. It's led to at least four high speed crashes with stationary objects it didn't identify as obstacles. This is pathetic. We had avoidance of big stationary objects working in the DARPA Grand Challenge back in 2005. With a good LIDAR, you get a point cloud. This tells you where there's something. Maybe you can identify some of the "somethings", but if there's an unidentified object out there, you know it's there. The planner can plot a course that stays on the road surface and doesn't hit anything. Object recognition is mostly for identifying other road users and trying to predict their behavior. Compare Chris Urmson's talk and videos at

Link to actual report.[1] (When you're posting a link to something important, post the link to the real document, not some pundit blithering about it. Yes, you may have to do some digging to find it when the pundit outlet doesn't provide a link.) There's not much new here if you've been following what Waymo is doing. Chris Urmson's SXSW talk from 2016 [2] covered most of this material in more detail. This is more of a PR-level version of that content. The new material covers mainly the limitations Waymo is imposing on their system. It works only where they have very detailed maps of the road system, with more information than even StreetView. This is conservative, but not a bad decision for initial release. They have a button to call customer service. (That's so unlike Google.) [1] [link] [2] h [this video]

> Why not the other way around? You do the driving, and the computer is the backup in case you mess up? Unfortunately having the computer takeover has the same issues. If it thinks you're following the wrong lane it could choose to steer you into a barrier, for example. Chris Urmson's team at Google ran user tests at the start of their autonomous vehicle program and found that users misbehave [1] when behind the wheel of semi-autonomous vehicles, despite having been given instruction to pay attention. We're seeing that scenario play out in slow motion in Teslas and perhaps other driver-assistance programs that get less press. [1] [this video]