>> please join me in thanking our panel. [applause] [applause]

next on "the communicators" a conversation on self-driving cars. the candidates were back on the campaign trail following last night's presidential debate.

>> this week "the communicators" goes to pittsburgh to talk about self-driving cars. we take a ride in the autonomous car that the university is developing. >> host: we will talk to "the communicators" of carnegie mellon. what is this? >> guest: we are looking at a cadillac suv from 2011 which is capable of driving itself so it's a self-driving cadillac. >> host: if you look at the card looks pretty normal from here. >> guest: that is the car where

it that we literally bought new from the cadillac dealership and bo did a bunch of censors and computers and it drives itself. >> host: do you want to give us a ride? >> guest: i would be happy to give you a ride. hop in. >> host: what is the main reaction you get from people writing in the car with you? >> guest: the typical reaction is one of anxiety, angst, fear and occasionally even panic attacks. but then they basically watch the vehicle that is able to drive, stopping when it should stop and it's actually taking the curve comfortably. they build a degree of comfort.

>> we learn more about the move to self-driving cars and professor ragnunathan rajkumar at the lab where were his experimental cars been worked on. >> host: .or "the communicators" what is your job at carnegie mellon university? >> i'm a professor of engineering and robotics at carnegie mellon university. >> host: how did you get into that? >> guest: i did my post graduation studies and my masters in ph.d. at carnegie mellon and then i joined ibm research and i came back because i like it that much. >> host: what kinds of things do you work on? >> guest: i work on what are called systems. these are things that embeds computers that but use a different device.

for example your phone is basically a device with computers inside of them but the body of the computers inside him. let me show you an example. it's meant to be a television. think about a project or. protector has computers inside of them but you think of it as basically that device. that we use on a daily basis that causes -- cost 10 bucks as an embedded computer embedded inside of the toaster. it embeds computers but it's a dedicated device of some other kind. i've been working on embedded systems since my doctoral and it turns out intelligent vehicles themselves are classical example of these systems so a car is a

car that transports people and goods from point a 2. b. it just does that with computers embedded inside of them. >> host: how did you get into the business of autonomous vehicles? >> guest: a great question. i had been working with general motors, the carmaker since 2004. i have been working with researchers in our department since year 2000 so i started working with the gm group and started applying my embedded systems into automobiles and in 2006 the defense advance research agency and the u.s. military announced a competition called the urban challenge. vehicles that drive themselves. they need to drive for 60 miles in fewer than six hours driving

in an urban like setting interacting with other self-driving makers as well as human and following the same rules of a road that you and i have to follow on a daily basis. but that competition, gm became our biggest partner and sponsor. we had about 23 other sponsors as well but gm was the biggest of them. we had have a very strong working relationship with gm and became an integral part of the team that worked on a vehicle which ended up winning the competition as well. that's how i got to be a documented maker and when our theme from carnegie mellon won the competition gm which was the biggest sponsor the team said hey our team sponsored the winning team and they said because it's about urban driving it clearly has implications to

the consumer vehicle market segment and we started a second lab on campus focusing exclusively on automated driving and i've been running that lab as well since it was launched so that's how i got to work with gm. my relationship with gm became extremely strong and very loyal. >> host: so does gm on the technology that you developed? >> guest: the tech knowledge he that the sponsor, their support is owned by carnegie mellon university and we have licensing agreements with gm who have access to parts of the technology. >> we learned more about the experimental cadillac as we got ready to take it for a drive. >> host: i want to start by looking at this. what is this monitor that's up on the top? >> guest: that is one of six laser sensors called lidar's there is one on before head of

the car and there is one on the bumper here right there. that's a second lidar in the third lidar on the other side of the bumper as well is in the back window of the car and there's one on the other side exactly opposite of that. >> host: what i'd do a reading? >> guest: what they are doing is sending out multiple laser beams and when the laser beams hit an object they bounce back and come back to the transmitter and because we know the speed of light we can actually calculate how far and because of the multiple teams that create scanning as well you get a profile of the obstacle that the vehicle just encountered and because we have lidar all over the car the lidar is telling you what happens or a vehicle. >> host: dr. ragnunathan rajkumar and this is car

commuting -- communicating with anybody anybody but self? >> guest: is capable of actually communicating with a properly equipped traffic light, traffic sign and other objects with similar ready or was -- radios. >> host: i also see some cameras inside the car right appeared. what are these? >> guest: in addition to the six lasers sensors we talked about there are three cameras in six radars as well so they are two cameras one on either side of the rear view mirror. one is pointed downward looking for land markers on the roads and other cameras are pointed upwards looking at traffic lights. the third camera on the back is tracking focusing see kinsey was going on.

there are six radars. there is one behind this cadillac emblem. this is a medical local with a plastic logo so you can see through plastic but it turns out the radar behind this bumper as well on top of the lidar behind the bumper, the bumper is made of plastic so the radar can see through the plastic as well. there is radar on the other side of the bumper and to on this side right next to the lidar you don't see it from the outside and he don't see from inside either. there's another radar on the back. cisco is this parsing 360 degrees? >> guest: existing 360 degrees all the time as opposed to humans where our heads would have to turn all the way around 360. >> host: professor raj rajkumar what is different in the luck than a regular

cadillac? >> guest: we try to make this car look normal on the outside and the inside. on the inside i pretty much -- it pretty much operates and looks like a normal car. just like when you pick up a car at the airport but it's able to drive so it's basically the same thing. you start the car manually and start driving. meanwhile if you look at the dashboard to things have changed. that is a button on the dashboard. it's an emergency stop and a button by the stick shift. think of this as the autonomy equal of cruise control so you would think age this button to go to autonomous driving mode. the engine rotates the switch and pulls it upwards to get into

an autonomous driving mode and that's a very conscious action on the driver's part. >> host: is this car license to drive the streets of pennsylvania? >> guest: in pennsylvania the law allows a the vehicle to drive itself as long as -- number one a licensee among drivers in the driver seat and member two that acumen can take over control at any one time so under those two conditions they can drive themselves autonomously. >> host: this good drive of the normal car as well? >> guest: this goes from an autonomous vehicle but if one did not have the time to do that you could turn it press the brake pedals in the gas pedal and the vehicle still response. any number of things could be happening for humans to take

over. this button here is for emergency purposes. we added a bunch of sensors and computers and engaged for example something totally unexpected happens like you start smelling smoke and you have no idea what's going on. the vehicle is driving come he does push the button and ended electronically and mechanically becomes -- knock on wood. >> host: what is the cost of all the different system you have added to this car? >> guest: many of these sensors, radar and the cameras are one unit. they tend to be expensive so it's an expensive vehicle because of that but it's an idea to be thinking about. when the volume goes up the cost

will go down significantly. our thinking is that when the vehicles are mass produced it would add $5000 on top of the price. >> host: you are going to give us a driverless ride. is it truly driverless? is that a fair word to use? >> guest: this is an automated vehicle. he it can drive itself under many conditions but not under all conditions, at least not yet reflect start driving and i will explain a few more things. >> host: and you are driving manually? >> guest: i am driving manually. i'll point out a few things on the screen.

if i switch back and forth and stopped the vehicle, this is the screen and by flipping a switch you can go back and forth. >> host: what are these images we are seeing here? >> guest: what you will see on the screen, so this is the display that shows what the vehicle is doing at any one time so we as humans can be comfortable that the vehicle is indeed doing the right thing. what we see here are some icons here and the icons for example say that i can stop, i can make the vehicle go to the airport or go home and it basically lets you zoom in or zoom out. and what you see on the rest of the screen is you see two blue lines which are the lanes you

are driving. >> host: there are no lanes in this part do not. >> guest: you go to the main main -- and that sparks the gps device. the blue line is the map if you will and you see a green line there. we have a program that shows where we are and the gps device calculates it. the gpas isn't a map database. that is the green line that you see. you also see a sharp red line right there. that is basically the car knowing the green line route, knowing the blue line map, it basically uses that sensory data from the lidar camera and

over the next 15 or 20 meters this is where i'm going to drive >> host: dr. rajkumar do you have to program every route ahead of time or can it go to a street that has never been on before? >> guest: it needs to have a map of the roads. >> host: a gps map. >> guest: a gps map and then you need to basically tell the system where you want to go. it uses the map information and the current location to calculate the route using the gps device and google maps. the red line is deciding -- so i guess now you will see that i will zoom back a little bit. you see a bunch of dots on the screen. those dots are the laser points or the laser sensors so what you see here is basically a bunch of little dots.

>> host: is that a white car? >> guest: that's a weichert and these blue colors respond. >> host: and all these white lights. >> guest: basically these are her eyes and her ears. the camera and radar acts as the eyes and ears. >> host: how far can it see? >> guest: you can see 175 meters but the technology we discussed earlier the wireless communications. your -- radios doppler and so on can go as far as 600 meters. >> host: already. >> guest: let's engage the vehicle in autonomous mode. the vehicles in parking mode. i'm going to engage in autonomous mode.

>> host: while it's impart? >> guest: while it is in part. it has started driving. >> host: does this ever make you nervous? >> guest: you feel a lot of anxiety and angst. it's a very normal reaction. but let's see how it does. >> host: okay. dallas, are you doing okay back there? >> i think so. >> host: at turns on the turn signals. as you told the where you wanted it to go already, correct? >> guest: yes. >> host: all right. so you hit the brakes. >> host: no, i did not. it hit the brakes.

>> host: and it does the speed limit. >> guest: yes, it's exactly 25. pretty much everybody drives at 35. basically right now we have people behind us and of course they will say why are you driving so slow? musculus seems to be doing a little meandering in the lane. is that a fair assessment? >> guest: it could be better, yes. >> host: but it's reading constantly. >> guest: yes. this is a curving winding road. so i'm not controlling the steering well at the break pedal are the gas pedal. it is shifting the transmission by itself. >> host: i see that.

it's sensed that biker? >> guest: yes. >> host: all right. how far he had become in 30 years? >> guest: we have, very long way but there is still some ways to go to remove the human from the driving. >> host: is this vehicle constantly learning? >> guest: this vehicle is not learning. you can ex-pat and it uses the data to feed the software. it is not learning as we drive. learning after-the-fact. >> host: how did another was a stop sign there? >> guest: the map basically has locations on where the stop signs are.

it did that all by itself. >> host: that was the car that did that. it wasn't sherbet speed? >> guest: up when slightly to the right. i will take over manually. i've pushed this and in this case the vehicle has gone back into manual driving. >> host: can you do that on the fly? >> guest: yes. >> host: it's seeing all of these things. >> guest: these lanes have changed recently and it's not on the maps. >> host: it's not quite ready

to be sent on the road is never been on before. >> guest: highways, highways that it's has never been on before can do that. but roads with bicyclists and strollers,. [inaudible] >> host: a bike. >> guest: we are back on the curvy winding road. >> host: can it. signs? >> guest: it can. some signs, yes but not all

signs. it does not understand all of them. so whenever you see that red line, you see the green line and is the red line is exactly on top of that. >> host: how long have you driven this car autonomously? what is the longest trip you have ever taken? >> guest: we have done a couple hundred miles on the highways. the vehicle is used by us enough that i created, basically it drives a 3500-mile journey and

his highways about 90.6% of the time. the highway is not a problem. so you have taken your first ride in the car. >> host: when will we do this regularly as consumers? >> guest: simple question, basic question and i will give you long complicated answer. we can our day by vehicles for example tesla with an autopilot feature that can drive by itself but the human must be -- general motors next year will create a similar feature that they call super cruise that can steer

itself and apply brakes as well and that will be the cadillac cts sometime next year and many high-end -- drive themselves today. some of these features are regarding available and three to five years from now we expect the vehicles will be able to drive themselves but in well specified well-defined geographically constrained areas and it's called geo-fencing. for example bicyclists are not allowed and there is no heavy rain or heavy snow. they could deploy these technologies earlier but when can the human not drive at all implies that technology should be able to drive the vehicle

itself back any point in maclennon point b in the u.s. that capability will it take at least 10 years. we have come a long way over the past couple of decades or so but there's still quite a ways to go before the human can take himself or herself out of the drivers seat go in the backseat and take a nap. >> host: have you allowed your kids or your wife to ride with you in the autonomous car? >> guest: family members and many other people go in the car, yes. >> host: i was a little surprise that we didn't have to sign a release before we got in. >> guest: that's because we are researchers. >> host: why are we talking to about autonomous cars in pittsburgh rather than detroit or silicon valley? >> guest: carnegie mellon is a

globally well-known and has a strong reputation for computer science. engineering as well as robotics. there is a robotics institute on campus and it has more than 100 researchers in it excluding students if you will and they are all extremely knowledgeable about the area of robotics and intelligence in the field. in fact we have carnegie mellon believe that we are the -- place of a automatic technology. in 2014 we literally celebrated the 30th birthday of this technology on campus. >> host: you said before we started this interview that computers are simultaneously

very intelligent and very. >> guest: yes. computers are simultaneously very intelligent, they can do things that amaze us. they can react very quickly and they can make decisions that the normal person find extremely smart. how does it know that it should be driving at the speed? they look very intelligent and they are very intelligent because they process the view of the vehicle with the multiple sensory data streams from lasers and cameras. very intelligent. at the same time they are if you will because they don't really have common sense. for example we know that when we fall down or when we basically catch fire at hurts and we won't do it. computers cannot make a simple inference.

if you crash, next time don't do that. unless its program specifically by human being. >> host: what are the differences between the two vehicles that we see in the lap of a jeep that seems to have a lot of equipment on it. >> guest: we see two vehicles. one is the red jeep and one is the cadillac. the vehicle on the left, the red jeep was created by a research love laboratory and the 11th generation of autonomy. so they are 11 versions of the

vehicle so that he had loaned the right is the cadillac that we were able to drive today. that's vehicle was in the project that i lead from general motors as well as the national science foundation. because of our close working relationship with gm we are extremely sensitive to the aesthetics of the vehicle, the exterior aesthetics in the interior athletics as well. it looks very normal and is something that gm would be proud to design manufacture and sell. >> host: how far along are we with this technology? >> guest: the technology has come very rapidly over the past several years. it has come very far.

the 2007 challenge basic we the outcome, our team from carnegie mellon won the competition. the notion of a vehicle that can drive itself is no longer science fiction. it has demonstrated beyond a shadow of a doubt that it's a question of when, not if. since then google hired key people from carnegie mellon, key people from san bernardino city and that's how google launched itself driving vehicle project and then google basically enlisted several hundred and technology and car sharing companies like uber. we have a financial economic incentive to basic layout they have the vehicles drive themselves and meanwhile the carmakers, general motors, ford and the u.s., audi, bmw and

daimlerchrysler in europe, nissan honda and toyota in japan, a whole bunch of chinese carmakers who basically want to have an edge in the market and south korea all of them have been investing big-time into this space and meanwhile big automotive suppliers like continental and germany as well as the u.s. have been investing big-time as well so we have a lot of action in terms of investment, research and creating technology portfolio some karmic is, the supply chain and technology companies like google, opera and apple. technology is progressing very rapidly. >> host: you talk about it is action that are they developing their own systems just like you are here to ask. >> thanks to capitalistic forces

there is a lot competition. it's expected to be huge in 15 or 20 years or so so all these companies that we just discussed are pushing big pieces of the technology. it's about establishing it. >> host: what about the city of expert? has it been supported? >> guest: the city of pittsburgh has been a top-notch support of technology. we have been driving on public roads in pittsburgh since 2011. the state of pennsylvania has been using the technology as well. they allow us to basically drive vehicles with this technology as long as there's a human licensed driver in the drivers seat and a person can take over the control at any point in time.

besides that the vehicle can drive itself and if something goes wrong because there's a human and drivers seat the human would be liable. >> host: .or rajkumar would it surprise you since 2011 when he bought the cadillac? >> guest: so what has been satisfying if you will is that the vehicle for the most part looks normal on the outside and the technology for sensing and computing and steering the will and pressing the pedals. itec elegy can be made aesthetically pleasing as well. >> host: what if you talk to computers about the entire system since 2011? what additional information has gone into it? >> guest: the system, for the vehicle to drive itself that needs to have sensors like cameras, lasers and radar and

these basically have models of some kind of steer the wheel and can basically break but at the end of the day all the data collected by the censors have been processed by a bank of computers. all the data processing happens by software so literally hundreds of thousands of lines of software and data streams apply to the rules of the road and then sends proper commands to the steering wheel into the pedals. all the intelligence if you will inc. into the software so at the end of the day it's the software and artificial intelligence. >> host: 10 years from now are we going to look at this cadillac and go that was a nice black and white tv? >> guest: guests, 10 or 15

years enough people will talk about lookout point that little vehicle was. >> host: what is the difference between an autonomous car and they connected car? we were in m city in ann arbor recently as well. >> guest: an autonomous cars or car car that is capable of driving itself typically using sensors and computers that are built into the vehicle so it depends on the vehicle itself. it depends on what's happening in the environment and it has data coming in from the censors and local commands for steering and braking and so on. that's an autonomous vehicle. it connected vehicle is something that is capable of communicating with the cloud on the internet with traffic lights that are appropriately equipped with traffic signs as well as other vehicles. your smartphone, your desktop

can communicate wirelessly with each other. if they can why can't a vehicle computers to talk to the internet? it can talk to the environment so connected car is capable of communicating wirelessly with others in the environment on what it plans to do and similarly the empire meant sends the information wirelessly sync the traffic light just turned red and it can basically say it will start the vehicle and the lane ahead and all the information can be received wirelessly. it can generate to the human driver. the wireless communities and slept the human know if there's a problem to slow down. the roads are slick ahead, slow down.

it's a huge safety addition if you will. studies indicate that up to 80% of accidents can be prevented by using connected technology. connected technology can generate warnings and alerts so it's possible they can combine the two. it is connectivity and nomination. >> host: would that take a big investment in u.s. infrastructure to get connected cars on the road? >> guest: a great question. connectivity requires a small wi-fi like device that we all use on a regular basis. one of those devices actually costs only about $100 each. it's very very inexpensive.

anything that the laser the radar and camera can see so for $100. a radio wireless base can come to you as well so it comes to the superhuman mission if you will. the capability command and low prices compared to the rest of the technology. they can come up with their own local laws that operate within the state. you get a license from the state that they drive and you have a license from pennsylvania you can -- use the same license in ohio and california.

it would be a complete nightmare for car manufacturers. so the national highway safety administration is providing a set of guidance if you will so they can be homogenized or harmonized so if i can operate in one state it will be this same tech elegy in a neighboring state. >> host: professor rajkumar up maybe 20,000 miles on the cadillac, autonomously. he gotten to the point where you will look away from the windshield while it's driving? >> guest: rehab simulated 20,000 miles. we do demonstrations so we have sometimes public can sometimes

we have people in the car and cameramen watching of the time. i take my eyes off of the road and the ability to get an in highlight. that happens subconsciously, yes. >> host: what is the main reaction you get from people writing the car from you? >> guest: it's a remarkable sequence that happens. most if not all people after writing in a driverless car had written for the very first time. a typical reaction is one of anxiety, angst, fear and occasionally panic attacks. but then they basically watch the vehicle is able to drive stopping when it should stop. actually takes the curve fairly comfortably and recognizes and

builds a degree of comfort. we are watching the vehicle very intently. what is it doing? after a while the dealer says it's doing what it needs to be doing. a few years go by and they start conversing with other people in the car and after a while they start noticing what the car is doing. this acute anxiety to insurance up at the vehicle is doing takes five to 15 minutes depending on the person so the concern that we have is not that people are fearful. people become too comfortable, too quickly. so the steering technology has quite a ways to go in a situation that they encounter in a day-to-day basis of the human still needs to be --. >> host: seems all of a sudden

all of this technology is pushing towards us. is that just a perception because of the media? >> guest: because of the huge market potential that exists many companies are interested. number two, the advances made are challenged. people understand that the technology is viable and they benefit economically from that so that's a significant investment on the technology side. basically a lot of aggressive marketing if you will to get the word out that each company is working on technology so the combination of the advances, the potential for economic gains in the future and the marketing initiative is basically what you're seeing today. >> host: what about security and privacy because so much of this technology is out there on

the airways. >> guest: because these are driven by computers and hundreds of thousands of lines of software we have to be cognizant of and be cautious about threats to technology. these could include cybersecurity attacks. i can send information out. i can also receive information back in. those entry and exit points become potential portals for attack. we need to be careful that the attacks coming from across the state, the country the continent or globally. the cybersecurity threat one also has to be worried about what what can be done the physical context. for example it can jam a gps and

possibly spoof the gps and use for example even a simple laser device you can buy cheaply to confuse the lidar censors so the cybersecurity threats and physical attacks that are possible so we like to combine the two. >> host: finally, what is the next-generation? are you working on the next generation of technology for this vehicle right now? >> guest: we are focusing on the capabilities. next-generatis being the conductor and if you look at the -- they all had very similar censors if you will a streaming lidar on the roof of the car being example. the cadillac you see behind me think ofhe