Google's Self-Driving Car Causes First Accident, As Programmers Try To Balance Human Simulacrum And Perfection

from the get-out-of-my-lane,-Dave dept

Google's self-driving cars have driven millions of miles with only a dozen or so accidents, all of them being the fault of human drivers rear-ending Google vehicles. In most of these cases, the drivers either weren't paying attention, or weren't prepared for a vehicle that was actually following traffic rules. But this week, an incident report by the California Department of Motor Vehicles (pdf) highlighted that a Google automated vehicle was at fault in an accident for what's believed to be the first time.

According to the report, Google's vehicle was in the right-hand turn lane in a busy thoroughfare in Google's hometown of Mountain View, California, last month, when it was blocked by some sand bags. It attempted to move left to get around the sand bags, but slowly struck a city bus that the car's human observer assumed would slow down, but didn't. All in all, it's the kind of accident that any human being might take part in any day of the week. But given the press and the public's tendency toward occasional hysteria when self-driving technology proves fallible, Google's busy trying to get out ahead of the report.

Google historically compiles monthly reports for its self-driving car project; and while its February report addressing this specific accident hasn't been made public yet, Google's been releasing an early look to the media. In the report, Google notes that just like humans, trying to predict another driver's behavior isn't always successful on the road:

"Our test driver, who had been watching the bus in the mirror, also expected the bus to slow or stop. And we can imagine the bus driver assumed we were going to stay put. Unfortunately, all these assumptions led us to the same spot in the lane at the same time. This type of misunderstanding happens between human drivers on the road every day.

This is a classic example of the negotiation that’s a normal part of driving – we’re all trying to predict each other’s movements. In this case, we clearly bear some responsibility, because if our car hadn’t moved there wouldn’t have been a collision. That said, our test driver believed the bus was going to slow or stop to allow us to merge into the traffic, and that there would be sufficient space to do that.

We’ve now reviewed this incident (and thousands of variations on it) in our simulator in detail and made refinements to our software. From now on, our cars will more deeply understand that buses (and other large vehicles) are less likely to yield to us than other types of vehicles, and we hope to handle situations like this more gracefully in the future."

Live and learn. Or compute and learn. Whatever. If automated vehicles were going to cause an accident, it's at least good that this appears to be an experience (don't give city bus drivers the benefit of the doubt) programmers will learn from. The problem historically is that like so many technologies, people are afraid of self-driving cars. As such, automated vehicles can't just be as good as human beings, they'll have to be better than human beings for people to become comfortable with the technology seeing widespread adoption.

By any measure self-driving cars have been notably safer than most people imagined, but obviously there's still work to do. Recent data from the General Motors-Carnegie Mellon Autonomous Driving Collaborative Research Lab suggests that self-driving cars have twice the accidents of human-driven vehicles -- but again largely because people aren't used to drivers that aren't willing to bend the rules a little bit. Striking an acceptable balance between having an automated driver be perfect -- and having an automated driver be more human like -- is going to be a work in progress for some time.

Filed Under: accidents, autonomous vehicles, blame, fault, self driving cars
Companies: google

If We're Not Careful, Self-Driving Cars Will Be The Cornerstone Of The DRM'd, Surveillance Dystopias Of Tomorrow

from the who-controls-the-code dept

We've talked a lot about the ethical and programming problems currently facing those designing self driving cars. Some are less complicated, such as how to program cars to bend the rules slightly and be more more human like. Others get more complex, including whether or not cars should be programmed to kill the occupant -- if it means saving a school bus full of children (aka the trolley problem). And once automated cars are commonplace, can law enforcement have access to the car's code to automatically pull a driver over? There's an ocean of questions we're not really ready to answer.

But as we accelerate down the evolutionary highway of self-driving technology, the biggest question of all becomes: who gets to control this code? Will the automotive update process be transparent? Will the driver retain the ability to modify their car's code? Will automakers adapt and stop implementing the kind of paper mache level security that has resulted in the endless parade of stories about hacked automobiles it takes five years for automakers to patch?

Trying to force the issue before there's a hacker-induced automotive mass fatality, Ford, GM and Toyota were hit by a class action lawsuit earlier this year claiming the car companies were failing to adequately disclose the problems caused my abysmal auto security:

"Among other things, the lawsuit alleges Toyota, Ford and GM concealed or suppressed material facts concerning the safety, quality and functionality of vehicles equipped with these systems. It charges the companies with fraud, false advertising and violation of consumer protections statutes. Stanley continued, "We shouldn't need to wait for a hacker or terrorist to prove exactly how dangerous this is before requiring car makers to fix the defect. Just as Honda has been forced to recall cars to repair potentially deadly airbags, Toyota, Ford and GM should be required to recall cars with these dangerous electronic systems."

This month a court ruled that yes, we will have to probably wait for someone to die before automakers are held liable for lagging automotive security. The case was ultimately dismissed (pdf), the court ruling that the plaintiffs have yet to prove sufficiently concrete harms, and that potential damage (to the driver and to others) remains speculative. At the pace self-driving and smart car technology is advancing, one gets the sneaking suspicion we won't have long to wait before harms become notably more concrete.

But however complicated these legal, ethical, and technical questions are, they become immeasurably more complicated once you realize that smart cars will ultimately form the backbone of the smart cities of tomorrow, working in concert with city infrastructure to build a living urban organism designed to be as efficient as mathematically possible. As Cory Doctorow noted last week, this makes ensuring code transparency and consumer power more important than ever:

"The major attraction of autonomous vehicles for city planners is the possibility that they’ll reduce the number of cars on the road, by changing the norm from private ownership to a kind of driverless Uber. Uber can even be seen as a dry-run for autonomous, ever-circling, point-to-point fleet vehicles in which humans stand in for the robots to come – just as globalism and competition paved the way for exploitative overseas labour arrangements that in turn led to greater automation and the elimination of workers from many industrial processes.

If Uber is a morally ambiguous proposition now that it’s in the business of exploiting its workforce, that ambiguity will not vanish when the workers go. Your relationship to the car you ride in, but do not own, makes all the problems mentioned even harder. You won’t have the right to change (or even monitor, or certify) the software in an Autonom-uber. It will be designed to let third parties (the fleet’s owner) override it. It may have a user override (Tube trains have passenger-operated emergency brakes), possibly mandated by the insurer, but you can just as easily see how an insurer would prohibit such a thing altogether."

You'd hate to wander too casually into the hyperbole territory traditionally reserved for hysterical Luddites, but there's a laundry list of reasons to be worried about the trajectory of the lowly automobile. If we don't demand code transparency and consumer empowerment in automotive standards now, your car may find itself the cornerstone of a future in which DRM, encryption backdoors, lax security standards, eroded consumer legal rights, insurance companies and government power combine to create a supernova of dystopian dysfunction.

Filed Under: autonomous vehicles, cars, drm, privacy, self-driving cars, surveillance

Self-Driving Cars Have Twice The Accidents, But Only Because Humans Aren't Used To Vehicles Following The Rules

from the I'm-sorry-you-hit-me,-Dave dept

When Google discusses its latest self-driving car statistics (provided monthly at the company's website), the company is quick to highlight that with two million miles of autonomous and manual driving combined, the company's self-driving cars have only been involved in 17 minor accidents, none of them technically the fault of Google. Or, more specifically, these accidents almost always involve Google's cars being rear ended by human drivers. But what Google's updates usually don't discuss is the fact that quite often, self-driving cars are being rear ended because they're being too cautious and not human enough.

And that's proven to be one of the key obstacles in programming self-driving cars: getting them to drive more like flawed humans. That is, occasionally aggressive when necessary, and sometimes flexible when it comes to the rules. That's at least been the finding of the General Motors-Carnegie Mellon Autonomous Driving Collaborative Research Lab, which says getting self-driving cars onto the highway can still be a challenge:

"Last year, Rajkumar offered test drives to members of Congress in his lab’s self-driving Cadillac SRX sport utility vehicle. The Caddy performed perfectly, except when it had to merge onto I-395 South and swing across three lanes of traffic in 150 yards (137 meters) to head toward the Pentagon. The car’s cameras and laser sensors detected traffic in a 360-degree view but didn’t know how to trust that drivers would make room in the ceaseless flow, so the human minder had to take control to complete the maneuver."

And while Google may crow that none of the accidents their cars get into are technically Google's fault, accident rates for self-driving cars are still twice that of traditional vehicles, thanks in part to humans not being used to a vehicle that fully adheres to the rules:

"Turns out, though, their accident rates are twice as high as for regular cars, according to a study by the University of Michigan’s Transportation Research Institute in Ann Arbor, Michigan. Driverless vehicles have never been at fault, the study found: They’re usually hit from behind in slow-speed crashes by inattentive or aggressive humans unaccustomed to machine motorists that always follow the rules and proceed with caution."

But with a sometimes-technophpobic public quick to cry foul over the slightest self-driving car mishap, car programmers are proceeding cautiously when it comes to programming in an extra dose of rush-hour aggression. And regulators are being even more cautious still. California last week proposed new regulations that would require that all self-driving cars have full working human controls and a driver in the driver's seat at all times, ready to take control (which should ultimately do a wonderful job of -- pushing the self-driving car industry to other states like Texas).

The self-driving car future is coming up quickly whether car AI or self-driving auto philosophical dilemmas (should a car be programmed to kill the driver if it will save a dozen school children?) are settled or not. Google and Ford will announce a new joint venture at CES that may accelerate self-driving vehicle construction. And with 33,000 annual fatalities caused by highway-bound humans each year, it still seems likely that, overly-cautious rear enders aside, an automated auto industry will still likely save significant lives over the long haul.

Filed Under: accidents, autonomous vehicles, cars, driving, rules, self driving cars
Companies: google

Should Police Have The Right To Take Control Of Self-Driving Cars?

from the kill-switch dept

As Google, Tesla, Volvo, and other companies make great strides with their self-driving car technology, we've started moving past questions about whether the technology will work, and started digging into the ethics of how it should work. For example, we recently discussed whether or not cars should be programmed to sacrifice their own driver if it means saving the lives of countless others (like a number of children on a school bus). Programmers are also battling with how to program vehicles to obey all rules -- yet still account for highway safety's biggest threat: shitty human drivers.

But another key question recently raised its head in discussing what this brave new self-driving world will look like. Just how much power should law enforcement have over your self-driving vehicle? Should law enforcement be able to stop a self-driving vehicle if you refuse to? That was a question buried recently in this otherwise routine RAND report (pdf) which posits a number of theoretical situations in which law enforcement might find the need for some kind of automobile kill switch:

"The police officer directing traffic in the intersection could see the car barreling toward him and the occupant looking down at his smartphone. Officer Rodriguez gestured for the car to stop, and the self-driving vehicle rolled to a halt behind the crosswalk.

Commissioned by the National Institute of Justice, the RAND report is filled with benign theoreticals like this, and while it briefly discusses some of the obvious problems created by giving law enforcement (and by proxy intelligence agencies) this type of power over vehicle systems and data, it doesn't offer many solutions. As parts of the report make clear, having immediate access to driver and vehicle history and data is an incredibly sexy concept for law enforcement:

"Imagine a law enforcement officer interacting with a vehicle that has sensors connected to the Internet. With the appropriate judicial clearances, an officer could ask the vehicle to identify its occupants and location histories. … Or, if the vehicle is unmanned but capable of autonomous movement and in an undesirable location (for example, parked illegally or in the immediate vicinity of an emergency), an officer could direct the vehicle to move to a new location (with the vehicle’s intelligent agents recognizing “officer” and “directions to move”) and automatically notify its owner and occupants."

Yes, because if the history of intelligence and law enforcement is any indication, the "appropriate judicial clearances" are of the utmost importance. Thanks to what will inevitably be a push for backdoors to this data, we'll obviously be creating entirely new delicious targets for hackers -- who've already been poking holes in the paper mache grade security currently "protecting" current vehicle electronics. The report does briefly acknowledge this "risk to the public’s civil rights, privacy rights, and security," but as we've seen time and time again, these concerns are a footnote in the expansion of surveillance authority.

We already live in an age where the consumer doesn't have the ability to control or modify their own vehicle's electronics courtesy of DRM and copyright, and self-driving cars are already going to be a tough sell for many people from a liberty and personal freedom perspective. Adding the ability for law enforcement to not only snoop on vehicle data but take direct control of your vehicle -- is a conversation we should start having sooner rather than later.

Filed Under: autonomous vehicles, lawa enforcement, police, remote control, self-driving cars, surveillance

DailyDirt: Lethal Machines

from the urls-we-dig-up dept

Artificial intelligence is obviously pretty far from gaining sentience or even any kind of disturbingly smart general intelligence, but some of its advances are nonetheless pretty impressive (eg. beating human chess grandmasters, playing poker, driving cars, etc). Software controls more and more stuff that come in contact with people, so more people are starting to wonder when all of this smart technology might turn on us humans. It's not a completely idle line of thinking. Self-driving cars/trucks are legitimate safety hazards. Autonomous drones might prevent firefighters from doing their job. There are plenty of situations that are not entirely theoretical in which robots could potentially harm large numbers of people unintentionally (and possibly in a preventable fashion). Where should we draw the line? Asimov's 3 laws of robotics may be insufficient, so what kind of ethical coding should we adopt instead?

• An open letter from the Future of Life Institute (FLI) is warning against the possibility of an artificial intelligence (AI) arms race that could threaten humanity. Autonomous weapons are a reality that could hinder beneficial AI research -- as well as systematically kill people without "meaningful human control" behind the algorithms. [url]
• Autonomous cars with an ethical code in addition to just software code... are getting increasing attention as the odds of self-driving vehicles on public roads grows ever more likely. If a child runs in front of an autonomous car, should the car swerve to avoid the kid? There is an ethical dilemma inherent in making vehicles that are smart enough to know the difference between a kid and some other moving object, but these questions might be avoided entirely by making smart systems only so smart and no smarter -- minimizing liability for the companies making the machines. [url]
• A precursor to an artificial intelligence race might be a supercomputer hardware arms race, and we're already ordering up a National Strategic Computing Initiative (NSCI) to build an exaflop computer to rival China's Tianhe-2. Sure, artificial intelligence doesn't need to be developed on super fast computers, but if fast computers are considered potential weapons, it's not a huge leap of logic to see a supercomputer arms race as a military threat. [url]

After you've finished checking out those links, take a look at our Daily Deals for cool gadgets and other awesome stuff.

Filed Under: ai, algorithms, artificial intelligence, asimov, autonomous vehicles, drones, ethical code, fli, military, national strategic computing initiative, nsci, robotics, supercomputers, tianhe-2, war, weapons
Companies: future of life institute

Driverless Cars: Disrupting Government Reliance On Petty Traffic Enforcement

from the uber-but-for-killing-your-small-town-speed-trap dept

Self-driving cars are on the way, and in their wake, they'll leave a variety of entities slightly less better off. Insurance companies may be the first to feel the pinch, as less-than-risk-averse drivers are replaced with Electric Grandmothers more than willing to maintain safe speed limits and the proper distance between vehicles. And as goes the car accident, so go other areas of the private sector: personal injury/DUI lawyers, hospitals, body shops, red light camera manufacturers, towing companies, etc.

But the public sector will take the hit as well. "Flow my tears," said the policeman.

Consider the following. This past year, the City of Los Angeles generated $161 million from parking violations. Red light violations have a fee of $490. Californians caught driving under the influence are fined up to $15,649 for a first-offense misdemeanor DUI conviction and up to $22,492 for an under-21 equivalent. Cities in California collect, on average, $40 million annually in towing fees that they divide with towing firms. Simply put, the hundreds of millions of dollars generated from poor driving-related behaviors provide significant funding for transportation infrastructure and maintenance, public schools, judicial salaries, domestic violence advocacy, conservation, and many other public services.

Since California legalized driverless vehicles, Google has logged more than 1.7 million miles during the testing phase and been involved in 11 accidents, none of which were the fault of the driverless vehicle. Tesla, Mercedes, and others are not far behind. It turns out that automated vehicle technology—unlike humans—abides by the law. And that’s bad news for local government revenues. In other words, once driverless cars become mainstream, deep revenue sources acquired from driving-related violations such as speeding tickets and DUIs will decrease greatly.

Someone has to pay for the roads and other government activities, but it won't be drivers. So, as the Brookings Institution report points out, new revenue streams will have to be sought. The obvious suggestion is tax-per-mile billing, but that puts the government right in your vehicle -- an idea that's not going to gain in popularity any time soon.

While the loss of revenue will have an impact, the picture painted here is skewed. For many years, communities have treated police departments as revenue generators, rather than crime fighters. This has skewed incentives so badly that some small towns have become nothing more than profitable speed traps. That's one end of the issue: the pressure (or the willingness) to overpolice minor traffic violations to keep city governments (and the police departments themselves) funded.

But that's only part of it. The situation looks rather dire, especially if one doesn't examine what's not being said in these paragraphs. As Scott Shackford at Reason points out, the Brookings Institution report does some mighty fine cherry-picking for its list of potentially-affected government services. Without a doubt, a downturn in revenue will affect good government programs like public schools and domestic violence programs. But it will also cut back funding for far more dubious government spending.

What an interesting list of government-financed uses they've chosen. Notice they left off "Poorly made third-party database software that will stop working properly in less than three years and that was purchased from somebody belonging to the same frat as the assistant city manager," "police abuse settlements," and "blatant pension spiking."

These "losses" will also be somewhat offset by less tax revenue being spent on traffic enforcement, accident response units and other related law enforcement activities. This will also mean fewer law enforcement officers will need to be employed, which should further reduce government expeditures.

The problem is that most governments aren't capable of heading off this sort of "threat" to their livelihoods, even with years of advance notice. Trimming back unneeded public sector employees won't happen until years after it's obvious they're no longer needed and will often come accompanied with expensive severance packages. New tax revenue streams won't be explored until they can be put off no longer, and often will just be added on top of existing taxes, rather than replacing those that have slowed to a trickle.

Worse, those most affected by this sort of shift will be the same people most affected by most government tax increases: the poor. The lowest income brackets will be the last to adopt driverless vehicles, leaving them the most exposed to fines for traffic violations (fines that will likely increase as revenue dwindles), as well as new costs like per-mile taxation. They're also most likely to see support programs they rely on suffer cuts as traffic enforcement money dries up.

The report somewhat addresses this outcome with a discussion of income inequality and the "disappearance of the middle class." While some of it is accurate and some of it is mostly buzzwords in search of a point, there's no doubt that traffic enforcement revenue will mostly be collected from those who can least afford it. After all, governments have done this for years -- something that helped fuel the outrage and backlash in Ferguson after the shooting of Michael Brown.

Is Brookings actually trying to blame the gap between billionaires and the poor for the racial tension in Ferguson? Which venture capitalist was it who told the Ferguson police to step up fine collection to rake in more money for the city's coffers? Which hedge fund manager invented the bureaucratic court system in Ferguson and other St. Louis County cities designed to wring every last cent from any indigent minority who couldn't afford an attorney? Which Wall Street "fat cat" is adding additional fees to every little fine so that getting pulled over for something as simple as not signaling a turn could end up costing hundreds of dollars for somebody who could end up losing his license and his ability to even work?

While driverless cars hold a great deal of disruption potential, when it's all said and done, governments will remain largely undisrupted. Whatever changes are made in response will arrive well after they're needed and be badly implemented. The same people who suffered in the previous system will find no improvement in the next one. While one would hope the drastic reduction in traffic enforcement would result in better, smarter policing more focused on serious criminal activity, old habits die hard. Cops will just go where the driverless car ain't, rather than trim that area of law enforcement to the minimum required. And cities will cut programs deemed expendable, rather than subject their own spending habits to greater scrutiny.

Filed Under: autonomous vehicles, driverless cars, taxes, traffic enforcement, traffic fines

Breaking: Self-Driving Cars Avoid Accident, Do Exactly What They Were Programmed To Do

from the I-can-and-will-do-that,-Dave dept

We just got done talking about how, after logging 1,011,338 autonomous miles since 2009, Google's automated cars have had just thirteen accidents -- none of which were the fault of the Google vehicles. By and large the technology appears to be working incredibly well, with most of the accidents the fault of inattentive human drivers rear-ending Google's specially-equipped Lexus SUVs at stop lights. But apparently, the fact that this technology is working well isn't quite interesting enough for the nation's technology press.

A Reuters report making the rounds earlier today proclaimed that two self-driving cars from Google and Delphi Automotive almost got into an accident this week in California. According to the Reuters report, Google's self-driving Lexus "cut off" Delphi's self-driving Audi, forcing the Audi to take "appropriate action." This apparently got the nation's technology media in a bit of a heated lather, with countless headlines detailing the "almost crash." The Washington Post was even quick to inform readers that the almost-crash "is now raising concerns over the technology."

Except it's not. Because not only did the cars not crash, it apparently wasn't even a close call. Both Delphi and Google spokespeople told Ars Technica that both cars did exactly what they were programmed to do and Reuters apparently made an automated mountain out of a molehill:

"I was there for the discussion with Reuters about automated vehicles," she told Ars by e-mail. "The story was taken completely out of context when describing a type of complex driving scenario that can occur in the real world. Our expert provided an example of a lane change scenario that our car recently experienced which, coincidentally, was with one of the Google cars also on the road at that time. It wasn’t a 'near miss' as described in the Reuters story."

Instead, she explained how this was a normal scenario, and the Delphi car performed admirably.

"Our car did exactly what it was supposed to," she wrote. "Our car saw the Google car move into the same lane as our car was planning to move into, but upon detecting that the lane was no longer open it decided to terminate the move and wait until it was clear again."

In other words, As Twitter's Nu Wexler observed, the two cars did exactly what they were programmed to do, though that's obviously a notably less sexy story than Reuters' apparently hallucinated tale of automated automotive incompetence.

Breaking: Self-driving cars avoid accident, doing exactly what they are programmed to do

— Nu Wexler (@wexler) June 26, 2015

Filed Under: accidents, autonomous vehicles, cars, driving, near miss, self-driving
Companies: delphi, google

Should Your Self-Driving Car Be Programmed To Kill You If It Means Saving A Dozen Other Lives?

from the I'm-sorry,-Dave dept

Earlier this month Google announced that the company's self-driving cars have had just thirteen accidents since it began testing the technology back in 2009, none the fault of Google. The company has also started releasing monthly reports, which note Google's currently testing 23 Lexus RX450h SUVs on public streets, predominately around the company's hometown of Mountain View, California. According to the company, these vehicles have logged about 1,011,338 "autonomous" (the software is doing the driving) miles since 2009, averaging about 10,000 autonomous miles per week on public streets.

With this announcement about the details of these accidents Google sent a statement to the news media informing them that while Google self-driving cars do get into accidents, the majority of them appear to involve the cars getting rear ended at stoplights, at no fault of their own:

"We just got rear-ended again yesterday while stopped at a stoplight in Mountain View. That's two incidents just in the last week where a driver rear-ended us while we were completely stopped at a light! So that brings the tally to 13 minor fender-benders in more than 1.8 million miles of autonomous and manual driving—and still, not once was the self-driving car the cause of the accident."

If you're into this kind of stuff, the reports (pdf) make for some interesting reading, as Google tinkers with and tweaks the software to ensure the vehicles operate as safely as possible. That includes identifying unique situations at the perimeter of traditional traffic rules, like stopping or moving for ambulances despite a green light, or calculating the possible trajectory of two cyclists blotto on Pabst Blue Ribbon and crystal meth. So far, the cars have traveled 1.8 million miles (a combination of manual and automated driving) and have yet to see a truly ugly scenario.

Which is all immeasurably cool. But as Google, Tesla, Volvo and other companies tweak their automated driving software and the application expands, some much harder questions begin to emerge. Like, oh, should your automated car be programmed to kill you if it means saving the lives of a dozen other drivers or pedestrians? That's the quandary researchers at the University of Alabama at Birmingham have been pondering for some time, and it's becoming notably less theoretical as automated car technology quickly advances. The UAB bioethics team treads the ground between futurism and philosophy, and note that this particular question is rooted in a theoretical scenario known as the Trolley Problem:

"Imagine you are in charge of the switch on a trolley track. The express is due any minute; but as you glance down the line you see a school bus, filled with children, stalled at the level crossing. No problem; that's why you have this switch. But on the alternate track there's more trouble: Your child, who has come to work with you, has fallen down on the rails and can't get up. That switch can save your child or a bus-full of others, but not both. What do you do?"

What would a computer do? What should a Google, Tesla or Volvo automated car be programmed to do when a crash is unavoidable and it needs to calculate all possible trajectories and the safest end scenario? As it stands, Americans take around 250 billion vehicle trips killing roughly 30,000 people in traffic accidents annually, something we generally view as an acceptable-but-horrible cost for the convenience. Companies like Google argue that automated cars would dramatically reduce fatality totals, but with a few notable caveats and an obvious loss of control.

When it comes to literally designing and managing the automated car's impact on death totals, UAB researchers argue the choice comes down to utilitarianism (the car automatically calculates and follows through with the option involving the fewest fatalities, potentially at the cost of the driver) and deontology (the car's calculations are in some way tethered to ethics):

"Utilitarianism tells us that we should always do what will produce the greatest happiness for the greatest number of people," he explained. In other words, if it comes down to a choice between sending you into a concrete wall or swerving into the path of an oncoming bus, your car should be programmed to do the former.

Deontology, on the other hand, argues that "some values are simply categorically always true," Barghi continued. "For example, murder is always wrong, and we should never do it." Going back to the trolley problem, "even if shifting the trolley will save five lives, we shouldn't do it because we would be actively killing one," Barghi said. And, despite the odds, a self-driving car shouldn't be programmed to choose to sacrifice its driver to keep others out of harm's way."

Of course without some notable advancement in AI, the researchers note it's likely impossible to program a computer that can calculate every possible scenario and the myriad of ethical obligations we'd ideally like to apply to them. As such, it seems automated cars will either follow the utilitarian path, or perhaps make no choice at all (just shutting down when encountered with a no win scenario to avoid additional liability). Google and friends haven't (at least publicly) truly had this debate yet, but it's one that's coming down the road much more quickly than we think.

Filed Under: autonomous vehicles, code, dilemmas, ethics, trolley problem

FBI Thinks Driverless Cars Could Be Criminals' New Best Friends

from the I,-for-one,-welcome-out-new-Robot-Death-Car-overlords dept

There have been dozens of techno-panics over the past several years, but one usually expects cooler heads to prevail in government agencies where the word "investigations" is in the title and the employees have access to plethora of cutting-edge equipment. (Note: said "cutting-edge equipment" for surveillance only. Agency computer systems remain an outdated mess and the idea of recording in-custody interviews has finally arrived nearly four decades too late.)

We are, of course, talking about the FBI. The FBI's relationship with technology is painfully one-sided. On one hand, it's pushing to gets its biometric database online and fully operational, hopefully years before its report on this database's privacy implications finally arrives. On the other hand, it has argued in court (via the DOJ) that smartphone technology vastly outpaces law enforcement's tools to access possible evidence and, therefore, should be obtainable without a warrant.

But it is an investigative agency, which would seem to indicate it has the ability to gather facts and come to informed decisions. But it seems to prefer worried conjecture to actual data. (See also: Insane Clown Posse fans are an organized criminal entity.) But here it goes again, seeing another technological development as another way for criminals to gain the upper hand.

In an unclassified but restricted report obtained by the Guardian under a public records request, the FBI predicts that autonomous cars “will have a high impact on transforming what both law enforcement and its adversaries can operationally do with a car.”

In a section called Multitasking, the report notes that “bad actors will be able to conduct tasks that require use of both hands or taking one’s eyes off the road which would be impossible today.”

The FBI looks at something that has the potential to make roads much safer and sees… autonomous vehicles loaded with gunmen, all of whom can use both hands to fire at pursuing law enforcement… or something.

Autonomy … will make mobility more efficient, but will also open up greater possibilities for dual-use applications and ways for a car to be more of a potential lethal weapon that it is today.”

Sure, the driverless vehicles could be loaded up with explosives and "told" to drive itself to its destination, but that seems like an incredibly expensive way to deliver a payload. And sure, vehicles might be hacked to ignore everything about them that makes driving safer, but that last part is nothing a human operator can't do in a normal, cheaper vehicle. And any vehicle with a driver can still carry armed criminals/explosives.

Even the FBI admits that autonomous cars present the agency with certain advantages, including the fact that the first few iterations of car AI will be able to do little more than recreate OJ Simpson's "getaway."

[A]utonomous cars would likely face many hardships with evasive driving or car chases...

But using this AI for good (and hacking it to serve its purposes) may also revolutionize the FBI and law enforcement's pursuit techniques.

"[A]lgorithms can control the distance that the patrol car is behind the target to avoid detection or intentionally have a patrol car make opposite turns at intersections, yet successfully meet up at later points with the target."

While some of this report is undoubtedly dedicated to "what if" scenarios not unlike the risk disclosures included in IPOs, there's still something ridiculous about an investigative agency being so tuned into the terror frequency that it sees criminal intent in every technological advancement. If we wanted fear-based speculation about potential havoc-wreaking by new inventions, we'd ask the MPAA.

Filed Under: autonomous vehicles, driverless cars, fbi, threats