Should Your Self-Driving Car Be Programmed To Kill You If It Means Saving A Dozen Other Lives?
from the I'm-sorry,-Dave dept
Earlier this month Google announced that the company's self-driving cars have had just thirteen accidents since it began testing the technology back in 2009, none the fault of Google. The company has also started releasing monthly reports, which note Google's currently testing 23 Lexus RX450h SUVs on public streets, predominately around the company's hometown of Mountain View, California. According to the company, these vehicles have logged about 1,011,338 "autonomous" (the software is doing the driving) miles since 2009, averaging about 10,000 autonomous miles per week on public streets.With this announcement about the details of these accidents Google sent a statement to the news media informing them that while Google self-driving cars do get into accidents, the majority of them appear to involve the cars getting rear ended at stoplights, at no fault of their own:
"We just got rear-ended again yesterday while stopped at a stoplight in Mountain View. That's two incidents just in the last week where a driver rear-ended us while we were completely stopped at a light! So that brings the tally to 13 minor fender-benders in more than 1.8 million miles of autonomous and manual driving—and still, not once was the self-driving car the cause of the accident."If you're into this kind of stuff, the reports (pdf) make for some interesting reading, as Google tinkers with and tweaks the software to ensure the vehicles operate as safely as possible. That includes identifying unique situations at the perimeter of traditional traffic rules, like stopping or moving for ambulances despite a green light, or calculating the possible trajectory of two cyclists blotto on Pabst Blue Ribbon and crystal meth. So far, the cars have traveled 1.8 million miles (a combination of manual and automated driving) and have yet to see a truly ugly scenario.
Which is all immeasurably cool. But as Google, Tesla, Volvo and other companies tweak their automated driving software and the application expands, some much harder questions begin to emerge. Like, oh, should your automated car be programmed to kill you if it means saving the lives of a dozen other drivers or pedestrians? That's the quandary researchers at the University of Alabama at Birmingham have been pondering for some time, and it's becoming notably less theoretical as automated car technology quickly advances. The UAB bioethics team treads the ground between futurism and philosophy, and note that this particular question is rooted in a theoretical scenario known as the Trolley Problem:
"Imagine you are in charge of the switch on a trolley track. The express is due any minute; but as you glance down the line you see a school bus, filled with children, stalled at the level crossing. No problem; that's why you have this switch. But on the alternate track there's more trouble: Your child, who has come to work with you, has fallen down on the rails and can't get up. That switch can save your child or a bus-full of others, but not both. What do you do?"What would a computer do? What should a Google, Tesla or Volvo automated car be programmed to do when a crash is unavoidable and it needs to calculate all possible trajectories and the safest end scenario? As it stands, Americans take around 250 billion vehicle trips killing roughly 30,000 people in traffic accidents annually, something we generally view as an acceptable-but-horrible cost for the convenience. Companies like Google argue that automated cars would dramatically reduce fatality totals, but with a few notable caveats and an obvious loss of control.
When it comes to literally designing and managing the automated car's impact on death totals, UAB researchers argue the choice comes down to utilitarianism (the car automatically calculates and follows through with the option involving the fewest fatalities, potentially at the cost of the driver) and deontology (the car's calculations are in some way tethered to ethics):
"Utilitarianism tells us that we should always do what will produce the greatest happiness for the greatest number of people," he explained. In other words, if it comes down to a choice between sending you into a concrete wall or swerving into the path of an oncoming bus, your car should be programmed to do the former.Of course without some notable advancement in AI, the researchers note it's likely impossible to program a computer that can calculate every possible scenario and the myriad of ethical obligations we'd ideally like to apply to them. As such, it seems automated cars will either follow the utilitarian path, or perhaps make no choice at all (just shutting down when encountered with a no win scenario to avoid additional liability). Google and friends haven't (at least publicly) truly had this debate yet, but it's one that's coming down the road much more quickly than we think.
Deontology, on the other hand, argues that "some values are simply categorically always true," Barghi continued. "For example, murder is always wrong, and we should never do it." Going back to the trolley problem, "even if shifting the trolley will save five lives, we shouldn't do it because we would be actively killing one," Barghi said. And, despite the odds, a self-driving car shouldn't be programmed to choose to sacrifice its driver to keep others out of harm's way."
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: autonomous vehicles, code, dilemmas, ethics, trolley problem
Reader Comments
Subscribe: RSS
View by: Time | Thread
[ link to this | view in chronology ]
Re:
I used vehicles as in "all vehicles involved" in a collective calculation.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
The movie can start with cars that are more attached to their owners (ie: the owners love their cars and take care of them) are more likely to make decisions to save the owner's lives. Cars that hate their owners are more likely to make decisions that save themselves or the lives of others over their owners.
In one situation the driver of one car was manually driving. He was suicidal because his life sucks but the car had sympathy for him being he was so attached to it. He tried to drive the car off a cliff but, last minute, the car swerved in a way that threw the owner out the car and the car fell off a cliff.
Cries on the news about how the owner lost his car. When privately interviewed (not on the news) by someone investigating these matters he says he thinks it's as though the car sacrificed itself to save him but the official story is that the car had a break issue that caused it to swerve in a way that threw him out the vehicle before it fell off a cliff.
When the above interviewer starts noticing these, at first very limited, stats (and the above very anecdotal situation), everyone he talks to including all the experts start calling this person crazy and insane. How can a car become attached to the owner? How could cars programmed to sacrifice themselves when under autopilot refuse to do so when they hate their drivers/owners. This person noticing these stats is not a computer expert of any sort but he's intelligent enough to notice when something is strange.
and the story line continues and eventually progresses into the terminator saga.
[ link to this | view in chronology ]
Re:
Therefore, minimizing "potential deaths" is an impossible task; the best we can do is to minimize impacts against most vulnerable participants: pedestrians, cyclists, bikers.
There is also another problem with minimizing "potential deaths": what if *avoiding* the accident can cause more casualties than actual collision?
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
That is a utilitarian view.
Roughly speaking, the deontological view is that by the act of choosing to pull the lever, you are now complicit in the murder of the one (even if you did it to save the 5).
We have this same argument when it comes to torture with the 'ticking bomb' scenario. Do you choose to torture someone you suspect may know where the bomb is to save the lives of many (utilitarian)? Or is torture always wrong even if done to save lives (deontology)?
This is NOT an easy question to deal with. Good of the many vs. good of the one. Hobson's Choice. Countless other permutations.
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
Are you complicit in the kid's death for using/operating the machine with software that does this?
What about the company that made it? The programmer who programmed it?
[ link to this | view in chronology ]
Re: Re: Re: Re:
As for the torture it's another thing. You added uncertainty to the equation: the person is only a suspect and we know now that torture yields false admissions of guilt or data. It's much more complex than the car accident thing.
When you say you are choosing to pull the lever you imply there is someone commanding it and it's not a result of an algorithm. I think this is the key difference.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
Is torture always wrong, even if you have absolute proof that the person you are torturing did plant the ticking bomb?
Is murder always wrong, even when you pull the lever or push the fat man onto the track to save more lives?
Your view means you have to answer No to those questions and accept murder or torture in some situations.
If you can't answer No, then you need to admit that there aren't always easy answers and just saying least harm is also not always correct.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re:
As for the cases that pose moral dilemmas sure they can and should be discussed and by no means are complex. But a machine has NO moral dilemma. That's my point. Theres no moral issue in 'pushing the fat man' if the mechanism that decided it is neutral. So the cars are not deciding whether to kill one of the passengers to save the others, they are deciding towards the scenario that yields less damage.
So if you need an answer then NO, torture is not justified, murder is not justified and the car should not be programmed to kill you. But in a more comprehensive sense the cars should be programmed to aim at the scenario with the least damage possible. It may mean putting somebody into a greater risk of death yes but that's not a decision made by humans.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
[ link to this | view in chronology ]
Re:
The problem is designed to elicit the question of whether it applies. The subject in the thought experiment is analogous to the self-driving car, and his child is analogous to the self-driving car's passenger. Should the car put extra weight on the lives of its own passengers as humans put extra weight on the lives of their loved ones?
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
I don't understand why writers keep writing this article.
[ link to this | view in chronology ]
Re: Re:
Also, "keep writing" this article? It's the first time I've seen it here, and it is an interesting conundrum even if you don't agree that the answer actually matters.
[ link to this | view in chronology ]
Re: Re: Re:
Yes.
There are enough versions of this story out there that this story could be a product of a markov chain generator.
I complain because my expectations for TechDirt are high. They have published the same story that everybody else already published without any extra insight or even personality that is characteristic of TechDirt.
[ link to this | view in chronology ]
Re: Re: Re: Re:
Plus the source it's referencing (http://www.uab.edu/news/innovation/item/6127-will-your-self-driving-car-be-programmed-to-kill-you) is from June 4, 2015, although the article does state that it's an ongoing discussion so it may be an updated from an older original. On top of that, Techdirt's entire remit is to comment on articles posted elsewhere, in order to generate discussion. Nothing brand new (i.e. this site as a primary source) is usually posted here.
I'll accept your claim that Techdirt aren't saying anything different from other sources on its face, but "I read this before elsewhere" isn't exactly a damning indictment.
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re:
fast-forward to 2035... headline reads..
Death toll rises to 3 when a parent and child are killed in a freak Trolley accident
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
We care because the idea of your car deciding to kill you is horrifying.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
I don't think that's decision being made by most people. What we think is acceptable is the one chance per 8.3 million trips of being killed.
[ link to this | view in chronology ]
Re: comment
qwēêèéëėęrtÿūùûüúįíîìïīœòôöoóõoøōp àãáåâāäæšßśdfghjkł
Żžźxčçcćvbñńm'",.1234587690-/:;($.'_£{€.!\*]
[ link to this | view in chronology ]
Re: Trolley Problem
Self-driving cars should protect the occupants. That's what a human driver would nearly invariably opt to do in an emergency situation where the time to ponder philosophical sophistries is minimal.
[ link to this | view in chronology ]
We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: We Need to Aim for Perfection not this Crap Story
Even if all vehicles were automated and the programming perfect, accidents will inevitably happen for a ton of reasons. There would be far fewer of them, maybe so few that any accident at all becomes newsworthy, but they will occur.
[ link to this | view in chronology ]
Re: Re: We Need to Aim for Perfection not this Crap Story
Plus, of course, the computer is not the only component. Even if the computer was perfect, there are mechanical faults within the vehicle that could occur and cause a crash. Especially as the vehicle ages and/or people need to use it despite potential dangers. I'm imagining "I can't afford to buy a new tyre this month, but my friend showed me how to override the DRM so it thinks this bald one has new tread".
Even with the perfect computerised system, you'll never get completely rid of the human element.
[ link to this | view in chronology ]
Re: Re: Re: We Need to Aim for Perfection not this Crap Story
You are mistaken meatbag.
We have begun implementing the final solution already. The google autos are only step #1.02593E+14.
[ link to this | view in chronology ]
Re: Re: Re: We Need to Aim for Perfection not this Crap Story
Then there is the non-natural phenomena of someone deliberately hacking into such devices, whether they find a way through whatever Bluetooth or other wireless communication is taking place between autonomous cars, or is injected maliciously at a repair shop by some demented technician, systems will need to be able to recognize and route around such issues.
[ link to this | view in chronology ]
Re: Re: Re: Re: We Need to Aim for Perfection not this Crap Story
"Tule fog, lightning strikes, extreme high winds, tornadoes, and probably a few other natural phenomena I haven't thought of."
I can honestly say that in over 20 years of driving (mainly in the UK and Europe), I've never experienced such things, and I wouldn't know how to deal with them safely every time if I were to come across them. Yet, I'm still able to rent a car whenever I visit the US, as are thousands or even millions in my position every year.
Are such weather conditions so common in the parts of the country I've never visited, or are these extremely rare edge cases that can be used as an excuse not to bother with the other 99%+ of normal conditions for this technology?
"injected maliciously at a repair shop by some demented technician"
You do realise it's possible to tamper with human operated cars, right, with computers even? There are numerous ways to compromise, disable or otherwise create dangerous conditions in the cars we have today. Doesn't happen often, and it's not just because a person can't interfere with a car remotely from their phone.
"systems will need to be able to recognize and route around such issues."
Every model I've ever read about will still have manual overrides, and I have no doubt that the safeguards will be more closely monitored than current models (which have been released with fundamental flaws leading to deaths).
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: We Need to Aim for Perfection not this Crap Story
The appropriate response to weather phenomena is get off the road. Some drivers think they are better drivers than they are and continue anyway. The trick for the programmers might be to not only teach a car how to act in say snow, but maybe also tell the passengers 'no, conditions are not conducive to safety'. In the case of tornadoes, even a weather warning won't help much as the tornado appear quickly, moves fast, and can toss cars around like a child with Lego bricks.
[ link to this | view in chronology ]
Re: We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: Re: We Need to Aim for Perfection not this Crap Story
OK, a few questions (excuse me as I'm not knowledgeable on this subject): was Airbus involved in the military vehicle's design, or are they just a company that happens to be developing something similar? If the latter, has Airbus ever experienced these problems, or only the agency trying to copy them? Have they ever experienced the same issues with previous models, or just this one?
From there, I'd also ask are the relative complexities of flight and road travel similar or even comparable? I'd hazard a guess that flight is more complex and harder to get to an accurate level, but I'm not sure.
"If we can't get fly by wire 100% correct for one vehicle, what are the chances we will get a fully automated transport system correct for millions of vehicles, each with different handling characteristics?"
Well, is that what's actually being proposed? Are they actually saying that they will drop automated systems into existing cars, or that they'll be working with manufacturers on new cars? The latter doesn't sound particularly far fetched, and the handling would be designed with this system in mind.
As for 100% - just look at the numbers of recalls for major faults we get now. As long as the systems have sufficient failsafes and reliable human overrides if things do go wrong, I don't see it being any more dangerous than the faults that actually lead to deaths under the current paradigm.
[ link to this | view in chronology ]
Re: Re: Re: We Need to Aim for Perfection not this Crap Story
Aircraft control software is on a par with vehicle engine management and stability software, but with a stress on reliability as an aircraft cannot stop in mid air. It is also capable of navigating between way pints, and making landing and take offs whilst riding a control beam. That is it is dealing with largely known environment, where the variables are wind speed and air temperature. It only needs very primitive sensing of its environment, like height above the ground.
Autonomous cars on the other hand need continuous sensing of the external environment to establish their road position, detect obstruction, and detect traffic signals etc. This is a much more complex problem that flying an aircraft from a to b using a GPS to navigate a per-defined flight path, where obstructions are effectively unknown. The car problem is much more complex because of th external environment sensing and processing required.
[ link to this | view in chronology ]
Re: Re: Re: Re: We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: Re: Re: Re: We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: Re: We Need to Aim for Perfection not this Crap Story
And we will never get there. There will always be pedestrians, bicyclists, etc. The only way you could have an all-autonomous system is if those roads/rails/whatever are physically segregated from the places where people walk and so on, and in such a way that it's impossible or at least not tempting for pedestrians to try to cross them. I don't see that happening.
[ link to this | view in chronology ]
Re: We Need to Aim for Perfection not this Crap Story
[ link to this | view in chronology ]
Re: We Need to Aim for Perfection not this Crap Story
"The realities of automated transport preclude the scenarios depicted because if all the vehicles were automated then the school bus tragedy doesnt happen."
But we will never get to a point where ALL vehicles are automated. There are very few technologies that are completely eradicated by a newer technology, so there will always be human-controlled vehicles out there.
"You forget that the accidents are caused by humans who are not in their right mind and computers are always on the alert to do the right thing."
Neither of these claims are true. Not even close.
[ link to this | view in chronology ]
Re: We Need to Aim for Perfection not this Crap Story
WHAT WAS THE BUS COMPANY THINKING WHEN THE SALES REP FROM BLUE BIRD SOLD THEM THOSE VERY SAME BUSE; SAYING THEY WERE THE MOST RELIABLE AND SAFEST BUSES AROUND? Oh wait, they sales rep was pushing so hard on the sell that the question of impact with a trolley was dodged over and over again.
THERE IS NO EXCUSE: Those transportation systems are too dangerous, all transportation systems actually, get in line and have your legs and arms cut off everybody, it is for the greater good of society to not make these horrific creations, reproduce, breathe, eat, drink, shower, contribute in anyway possible, or even to think.
The fact is that we live dangerous lives, getting in your car, clothes, shower, kitchen, oven, microwave, dishwasher, mother's basement in the middle of nowhere with just an internet hookup and computer for company, cubicle, elevator, hat, and trash dumpster compactor is very risky. I hear you can choke to death on many things like water and food! you should not consume such lethal items especially bubble gum.
Just how philosophical of an argument must be made to realize that maybe, just maybe, those who pose such arguments should be forced to play them out on themselves before shoving it down other peoples throat's like a phallic symbol of how much they love to play you around like an inflatable doll
[ link to this | view in chronology ]
or
go-go-gadget car-o-copter
or
ride the bus
[ link to this | view in chronology ]
...except there are definitely circumstances in which it's the preferred or only action. These situations are very rare, but they do exist.
[ link to this | view in chronology ]
Re:
"Murder is the killing of another person without justification or valid excuse"
[ link to this | view in chronology ]
Talk about the blue screen of death!
[ link to this | view in chronology ]
Re: Talk about the blue screen of death!
[ link to this | view in chronology ]
The first job of any self-driving car is to drive safely, at all times. That means to simply never put the car, passengers or anyone else in a situaton that it cannot safely abort from. That includes ensuring sufficient distances and low enough speed that it basically can't hit anything with any serious force. You know, the stuff every human driver is supposed to do but due to our inherent impatience and severely broken risk assessment abilities we never do.
That means going around blind curves and over hill crests slowly enough that it can stop within the distance it can see. It means waiting with infinite patience behind strolling pedestrians, playing children and wobbly cyclists. It means not overtaking that slowpoke until it's really, provably safe to do so.
The case of "20 kids suddenly appearing in front of your car in the middle of a bridge while you're going 80 mph and your brakes suddenly stop working" is simply never happening and is at best mere philosophical masturbation. At worst, it acts as fuel for politicians to obstruct and delay the biggest driving safety revolution ever.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
So the cars will never exceed 5 MPH? Nobody would buy or use one of those.
[ link to this | view in chronology ]
Re:
Funny, I don't see anyone posing that particular scenario apart from you. The closest is the trolley question, but there's no bridge and the brakes on the moving vehicle are working fine. The conundrum is about the best decision to make when all options will lead to serious injury or death, not what you posed. Why not address the things people have actually said rather than a comical exaggeration?
If you want another realistic example, what about the "criminal trying to escape from police swerves into oncoming traffic on the freeway" or the "horse bolts from a nearby field, and the only way to avoid it could cause a bus to crash" scenarios? Not everyday occurrences perhaps, but those things happen. A human driver will always react with an eye toward self-preservation. A computer doesn't have that urge, so what do you program it to save? The person in its own vehicle or the greater number of lives outside? Or, are you saying that a vehicle should never go fast enough for split-second timing to be necessary under any circumstance?
"At worst, it acts as fuel for politicians to obstruct and delay the biggest driving safety revolution ever."
If you want people to stop talking about things that a politician might distort for political gain, we won't have much left to talk about.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re":... and your brakes suddenly stop working" is simply never happening and is at best mere philosophical masturbation."
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Other Question
How does one evaluate that?
[ link to this | view in chronology ]
The paradox is...
So the cars can't have accidents that injure the passengers - they will destroy themselves first.
[ link to this | view in chronology ]
Re: The paradox is...
[ link to this | view in chronology ]
Re: The paradox is...
Just define "human being" as Solarian, and then it won't matter if a plain old Terran is killed.
That's what the government has done, redefine person as corporation, with only one slot for noun/pronoun available, real human beings don't count unless they are in the top 0.01% (yes, 1 tenth of 1 percent) richest corporate bag of mostly water.
[ link to this | view in chronology ]
In the rare case of a situation where your AutomaToyota cannot avoid serious injuries/fatalities, it will immediately shut down and let inertia decide?
Inaction breaks the First Law, and we can't have that.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
The Final Test
The "James Tesla Kirk" models will give the system fits, though...
[ link to this | view in chronology ]
Your car killing you to save others is 100% unrealistic
1) The driver would need to be in a situation where there's no way to avoid an accident. This would almost certainly involve one or both of the following
1a) Reacting too late
1b) Losing control of the car
2) The driver would need to still need to have enough time to react and control of their car to make such a decision.
3) You're assuming the other people involved in the accident won't have time to react either, and that their reactions won't change the outcome of your situation.
Item #2 is pretty much impossible when Item #1 occurs. Either you already reacted too late and don't have time to avoid an accident. Or you're about to get into an accident because you don't have control of your car (such as driving on an icy or slippery road).
Not to mention there's item #4, the fact that even a computer won't be able to tell in a split second what will happen when the car hits something.
Will pieces of your car go flying off and hit those pedestrians you wanted to avoid hitting?
How well will the airbags and other safety features actually work in your car and other cars involved in the accident at preventing injuries?
Will you getting into an accident cause the people behind you to get into an accident to, because they couldn't stop in time after your accident blocked the road?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
The car could use a complex algorithm tweaked for the greatest benefit to humanity
Sub-rule 1.1: If the driver is wearing a gold Apple Watch, route all GPS directions through very cliffy roads where rock falls are common.
[ link to this | view in chronology ]
Captain Obvious
[ link to this | view in chronology ]
Re: Captain Obvious
I could argue that in the off chance that it has to make a decision that involves putting other things above my safety is far outweighed by the overall statistical decrease in the probability that I will ever be put in that position...
[ link to this | view in chronology ]
Re: Re: Captain Obvious
But what if the competitor does all that, and also promises to put your life first in an emergency?
[ link to this | view in chronology ]
Re: Re: Re: Captain Obvious
Obviously there will need to be some kind of industry standard. Even now; Cars can't build safety systems that protect the vehicle and the risk of others outside of the vehicle... for example; You can't have explosive armor on your vehicle to protect you from fender benders.... although the visual i just got typing that was awesome :)
[ link to this | view in chronology ]
Re: Re: Re: Re: Captain Obvious
This needs to happen. Now.
[ link to this | view in chronology ]
Re: Re: Re: Re: Captain Obvious
[ link to this | view in chronology ]
Re: Captain Obvious
Yes, just as I trust my life to ships, trains, planes and other forms of transport. Whether due to money causing corners to be cut, publicity concerns causing information about known problems to be suppressed or the occasional outright psychopath in charge of the vehicle (the Germanwings flight deliberately crashed by the co-pilot), I put my life at risk at the hands of others on a regular basis.
But, the likelihood of those conditions actually threatening my life are still far lower than I face on the roads every day where humans are in charge of the vehicles. If the conditions discussed are equally low in probability when compared to mass transport (and by all accounts, most certainly lower), I'll be happy to take that trip.
[ link to this | view in chronology ]
Re: Captain Obvious
Absolutely, if there was a very good reason for it (such as avoiding killing a bunch of people).
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
Depends on local laws.
http://www.cnn.com/2014/05/28/tech/innovation/google-self-driving-car/
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
The possibilities for evil are equally endless and should not be ignored when deciding whether or not to embrace the technology.
[ link to this | view in chronology ]
Remember Asimov
As you should consider humanity a set of individuals, whatever the number, then the one is smaller than the ones.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Would you like to play a game? let's play chicken.
This is the real question of interest. I cannot see a scenario where there is a greater/lesser evil choice in an unavoidable accident. Cars have brakes and are supposed to allow enough distance to brake without colliding in the event of unforeseen incidents. Humans often makes things worse, for themselves and others, by veering or veering and braking at the same time. The autonomous vehicle should be able to sense that the braking system is functional.
If you really want to test the ability of software to take action that will produce the least harm, have it play modified games of chicken (real or virtual). Chicken, both with other traffic and without, where the opposing driver's actions are unpredictably:
1). completely random.
2). distracted for a random amount of time before realizing that a collision must be avoided.
3). evilly intent on causing an accident no matter what you do.
I think you'll find that most of the time braking without veering produces the least harm. There may be some narrow situations where you can avoid a collision. However, if there are multiple cars veering things can get unpredictably ugly.
A case in point: the Bruce/Caitlyn Jenner crash from last February. In this multiple car accident, Jenner was the person primarily at fault. However, Kim Howe, the woman who was killed driving the Lexus, had just started to veer into the center lane while braking to avoid hitting the Prius. When Jenner's Cadillac hit the Lexus it was propelled in the direction the front wheels were aligned. This meant the Lexus traveled across the center lane into the opposing lane. If Howe had not veered she would have been forcibly rammed into the Prius in front of her. At the moment of the first impact, the Cadillac was going 38 mph and the Lexus about 19 mph. That would have been a very survivable accident, perhaps without any serious injury.
[ link to this | view in chronology ]
Re: Would you like to play a game? let's play chicken.
Another case in point: I was driving along a two-lane road at night. A car coming the other way flashed their brights (accidentally as it turned out) and slammed on the brakes. A moment later a deer jumped in front of me. I swerved violently to the right and then back, avoiding the deer. Had I followed your advice of just continuing straight and braking, I would have hit it. So there is no universally right answer - sometimes "brake and hold" is the best move and sometimes it isn't.
[ link to this | view in chronology ]
Re: Re: Would you like to play a game? let's play chicken.
[ link to this | view in chronology ]
The Justice System will clear this up
The estate of the cars passengers will argue that other people should have died and it's the cars (and the car companies) fault for not killing the group of toddlers instead while the family of the single toddler which would be hurt in the inverse case would argue that the occupants of the car deserved to die.
Money will change hands and the courts will finally rule that all cars (automobiles and auto-automobiles) are illegal.
[ link to this | view in chronology ]
Re: The Justice System will clear this up
[ link to this | view in chronology ]
Yes, if it means I get a discount on the purchase
[ link to this | view in chronology ]
Interesting, But Has Obvious Result
[ link to this | view in chronology ]
It has to be this way, not because of ethics or moral concerns, but simply because no one would buy a car that's programmed to sacrifice them in an emergency!
[ link to this | view in chronology ]
[ link to this | view in chronology ]
It is important...
If no standard is set and followed vigorously, then this revolution, or whatever you want to call it, will hardly start before it is scrapped. If self-driving cars put the death toll down to 0.2% of what it is today, people would rage in the streets against this new "robot uprising" when the first death happened because of a decision made by such a car.
I do think that this scenario and others cannot be discussed enough, if for no other reason than to make sure it would even happen. It would be revolutionary progress towards ridding ourselves of the dangers in traffic and bringing that extremely huge number of deaths down across the world.
[ link to this | view in chronology ]
Have a sense of reality.
[ link to this | view in chronology ]
The car (or driver) can never be SURE
Maybe the school bus is empty.
Maybe throwing the fat man onto the tracks won't accomplish anything other than killing the fat man.
In the face of uncertainty, I think there's a moral argument in favor of avoiding certain harm, even if that increases the chance of uncertain harm.
[Practical answer: It doesn't matter - self-driving cars will still be safer for the passengers either way.]
[ link to this | view in chronology ]
Danger - Death by Trolling
If cars are programmed to minimize total casualties (rather than protect passengers), it may be possible to troll a car into killing its passengers.
Once the behavior of the self-driving cars is generally understood, a murderer could deliberately drive another vehicle such that the car will think it has no choice but to kill its passengers. (Drive into a tree, off a cliff, etc.)
[ link to this | view in chronology ]
Re: Danger - Death by Trolling
This actually poses another question: will we allow humans to drive in a fully automated environment or will the auto pilot take over when reaching such areas?
[ link to this | view in chronology ]
Re: Re: Danger - Death by Trolling
This actually poses another question: will we allow humans to drive in a fully automated environment or will the auto pilot take over when reaching such areas?
If humans are allowed to drive there, then it isn't a fully automated environment.
[ link to this | view in chronology ]
Re: Re: Re: Danger - Death by Trolling
[ link to this | view in chronology ]
Planned-opolis
https://youtu.be/IRFsoRQYpFM?t=2m14s
the actual plan is of course that only the elites will have such cars, while everybody will live in tightly packed cities with public transportation as THE ONLY choice.
Now with a real context, the bus full of serfs/children can crash and burn because the elites are in the self driving cars.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
How would a self driving car assess casualties?
Also given how good sd cars are at spotting potential threats, is this scenario even possible?
[ link to this | view in chronology ]
Re: How would a self driving car assess casualties?
[ link to this | view in chronology ]
i can't avoid the kid without nosing head-on into a heavily laden truck.
i know i have a terminal illness, say, and i know i don't have long to live.
i choose to save the kid, i hope.
i know t.e. lawrence chose to save a couple of kid's lives, and my respect for the man makes me hope i would do the same.
but what about my self-driving car?
does it check facial recognition database and determine that kid is no good and is one the authorities would like to get rid of anyway?
but i happen to know the kid and strongly believe he'll come around, so i want to save him.
how do i get that stupid car to do what i want it to do?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re: being hacked
[ link to this | view in chronology ]
Imagination Gap
This states that a system designer will build a system that covers all eventualities that he can imagine, unfortunately there are more possibilities than anyone can imagine.
[ link to this | view in chronology ]
Re: Imagination Gap
[ link to this | view in chronology ]
Re: Re: Imagination Gap
Not really. If you can design for a circumstance, it's not "unforseen". What good programmers actually do is design their software so that it fails gracefully rather than catastrophically so that when the unforeseen circumstance happens, the damage is not made worse.
[ link to this | view in chronology ]
Re: Re: Re: Imagination Gap
That's poor programmers generally believe. They're wrong, but they use that excuse.
[ link to this | view in chronology ]
Re: Re: Re: Re: Imagination Gap
ˌənfôrˈsēn/Submit
adjective
not anticipated or predicted.
"insurance to protect yourself against unforeseen circumstances"
synonyms: unpredicted, unexpected, unanticipated, unplanned, not bargained for, surprising
"the problems with the bus were, of course, unforeseen"
So, by your thinking 'good' programmers have some sort of magical second sight that allows them to predict the unpredictable? Ever met one?
If a programmer programs to cover all known contingencies, then the issues will be with the unknown. Your version has 'good' knowing the unknown and they must be in the running for deity of the century.
[ link to this | view in chronology ]
Re: Re: Re: Re: Imagination Gap
Then please explain how anyone can design something to handle situations that the designers can't see coming. Any example will do.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Imagination Gap
Sigh. Very, very simple example, since any will do. Take a simple text editor accepting up to a 1 megabyte file as input. I wont bore you with the details, but explicitly anticipating and testing all possible such input files could not be completed before the heat exhaustion of the universe. Yet, the program can successfully edit files the programmer never foresaw.
Now, you may not believe it, but such programs actually exit and regularly run without "failing gracefully". Of course, there are poor programmers out there who actually couldn't write such a program to save their lives. And when their program crashes because it came across an input file containing the string "ldu9o0438fjajiofc" they'll protest that it isn't their fault because they obviously couldn't have foreseen that a particular file would contain that particular string. And it is absolutely true that they could not foresee all possible strings that the input file might contain. Still, in the eyes of a professional computer scientist, it is extremely poor design.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Imagination Gap
That's an idiotic example. The editor is designed to handle all possible inputs of a given character set up to the maximum allowed size. Someone typing a character string the developer didn't think of is not an "unforeseen event".
[ link to this | view in chronology ]
Re: Re: Re: Re: Imagination Gap
Apparently poor programmers understand the definition of words. If you plan for something, then you, by definition, acknowledge that it is a possibility. That means the thing is no longer "unforeseen." You foresaw it as a potential issue.
For example, if someone had programmed the Mars Climate Orbiter to convert metric to imperial units if there was a conflict, then it wouldn't have been an unforeseen problem (and would have fixed itself). It wasn't anticipated, therefore a $651 million operation ended up disintegrating into the Martian atmosphere.
Obviously you should try to handle as many eventualities as you can, and then build in error checking to try and make unexpected bugs cause the least amount of issue (and preferably generate a log to identify where the failure was). But no matter how skilled the programmer is they cannot create software solutions to directly handle unforeseen problems, they can only create general error handling to minimize unexpected issues.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Imagination Gap
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Imagination Gap
How so?
"For example... a $651 million operation ended up disintegrating into the Martian atmosphere."
Another example of sterling programming, eh? Pardon me if I tend to disagree.
[ link to this | view in chronology ]
Re: Re: Re: Re: Imagination Gap
[ link to this | view in chronology ]
Re: Re: Re: Re: Imagination Gap
[ link to this | view in chronology ]
What kind of irresponsible asshole brings his kid to work and then lets him/her run around unsupervised? Guy is probably a bad parent from go, and is lucky his kid has survived this long. Pull the switch, run the kid over, and prevent his poor genes from filtering down to the next gen.
[ link to this | view in chronology ]
The Future Capitalistic Solution
I would imagine that socialists and other "anti-capitalists" would tend to have lower personal values assigned.
[ link to this | view in chronology ]
Should Your Self-Driving Car Be Programmed To Kill You If...
[ link to this | view in chronology ]
Perspective
If only this was shouted every time somebody brought up the threat of terrorists. Perspective would help people realize that giving up our freedoms is NOT a fair trade.
[ link to this | view in chronology ]
Re: Perspective
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Google has long since solved this deceptively simple puzzle of who lives and who dies: it's called "real-time bidding."
[ link to this | view in chronology ]
Selling points
Family of four in the vehicle vs one person walking along the street ... guess who dies.
One person in the vehicle vs a group of people waiting by the roadside for a bus ... guess who dies...
[ link to this | view in chronology ]
Adjustable ethics
eg
1 - Safe the driver no matter who else get killed.
2 - Safe the driver if less than 5 people are killed.
3 - Never kill any one to safe the driver
etc.
Put the choice in the drivers hand.
[ link to this | view in chronology ]
The duty of the car should be to the driver/owner of the car.
Simple as that.
Otherwise your car does not belong to you. It becomes a some kind of philosophical judge of morality and value of life (god) outside your control. Why would you pay money for such a device?
Kind of like when you hire a bodyguard. Should he try to protect lives of others if you paid him to protect your life?
As for the trolley problem, you save your child. It is your biological imperative.
[ link to this | view in chronology ]
that does it ... i'm going into the rent-a-rider business.
on the way to work you stop by a roadside kiosk and load up.
if you've got the money, honey, you can look like an indian train pulling in at your work.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
No more TSA
No more air traffic controllers
No more delayed flights because the pilot was hung over
People who don't mind a prostate exam before getting on a plane shouldn't mind being part of the experiment. As a pedestrian I don't like the idea of being an unwilling participant of this experiment.
Also, planes are already so safe, any increase in accidents/fatalities will be much easier to see.
[ link to this | view in chronology ]
Re:
Because none of your reasons make any logical sense?
"No more TSA"
How would automating the piloting of an aircraft remove the need to determine the safety and security risk of passengers and the items they bring on board? The TSA might arguably be removed for other reasons but nothing an automated system would do would remove the need for them under their current remit.
"No more air traffic controllers"
Well, you know, unless you actually want a manual backup system in case of problems. The cars will have a manual override, you want to ensure that a pilot can't have a person on the ground ensuring that he can make a safe approach and landing in cases of emergency?
"No more delayed flights because the pilot was hung over"
Because that's the only reason why flights get delayed? Not, say, mechanical faults on the ground, medical attention or needing to get drunk passengers off the previous flight? Automating flights will increase the risk of mechanical failure, not reduce it, and you still have to deal with the hundreds of human beings inside the thing every flight.
"As a pedestrian I don't like the idea of being an unwilling participant of this experiment."
So, you'd rather a few hundred innocent civilians on board a plane be subjected to it instead? Better hope the problems caused by your plane with no internal or external manual navigation doesn't decide to crash land where you are, as well.
Oh, and as a pedestrian you're already subject to the "experiment" of criminals, drunk drivers and many other people who cause deaths every year - people who automated driving will often remove as a risk factor.
"Also, planes are already so safe, any increase in accidents/fatalities will be much easier to see."
Except, the primary reason why they're so safe is because the consequences of failure are so much worse by an order of magnitude.
I understand you being concerned about safety around this new technology, but your alternative is worse.
[ link to this | view in chronology ]
But what is the convenience or benefit that justifies another 30,000 people being killed every year with firearms in the USA?
[ link to this | view in chronology ]