Autonomous Bot Seized For Illegal Purchases: Who's Liable When A Bot Breaks The Law?
from the get-those-lawyers-ready dept
If you program a bot to autonomously buy things online, and some of those things turn out to be illegal, who's liable? We may be about to have the first such test case in Switzerland, after an autonomous buying bot was "seized" by law enforcement.Two years ago, we wrote about the coming legal questions concerning liability and autonomous vehicles. Those vehicles are going to have some accidents (though, likely fewer than human driven cars) and then there are all sorts of questions about who is liable. Or what if they speed? Who gets the ticket? There are a lot of legal questions raised by autonomous vehicles. But, of course, it's not just autonomous vehicles raising these questions. With high-frequency trading taking over Wall Street, who is responsible if an algorithm goes haywire?
This question was raised in a slightly different context last month when some London-based Swiss artists, !Mediengruppe Bitnik, presented an exhibition in Zurich of The Darknet: From Memes to Onionland. Specifically, they had programmed a bot with some Bitcoin to randomly buy $100 worth of things each week via a darknet market, like Silk Road (in this case, it was actually Agora). The artists' focus was more about the nature of dark markets, and whether or not it makes sense to make them illegal:
The pair see parallels between copyright law and drug laws: “You can enforce laws, but what does that mean for society? Trading is something people have always done without regulation, but today it is regulated,” says ays Weiskopff.But the effort also had some interesting findings, including that the dark markets were fairly reliable:
“There have always been darkmarkets in cities, online or offline. These questions need to be explored. But what systems do we have to explore them in? Post Snowden, space for free-thinking online has become limited, and offline is not a lot better.”
“The markets copied procedures from Amazon and eBay – their rating and feedback system is so interesting,” adds Smojlo. “With such simple tools you can gain trust. The service level was impressive – we had 12 items and everything arrived.”But, still, the much more interesting question is about liability in this situation. The Guardian reporter who wrote about this in December spoke to Swiss law enforcement, who noted that the situation was "unusual":
“There has been no scam, no rip-off, nothing,” says Weiskopff. “One guy could not deliver a handbag the bot ordered, but he then returned the bitcoins to us.”
A spokesman for the National Crime Agency, which incorporates the National Cyber Crime Unit, was less philosophical, acknowledging that the question of criminal culpability in the case of a randomised software agent making a purchase of an illegal drug was “very unusual”.Apparently, that assessment has concluded in this case, because right after the exhibit closed in Switzerland, law enforcement showed up to seize stuff:
“If the purchase is made in Switzerland, then it’s of course potentially subject to Swiss law, on which we couldn’t comment,” said the NCA. “In the UK, it’s obviously illegal to purchase a prohibited drug (such as ecstasy), but any criminal liability would need to assessed on a case-by-case basis.”
On the morning of January 12, the day after the three-month exhibition was closed, the public prosecutor's office of St. Gallen seized and sealed our work. It seems, the purpose of the confiscation is to impede an endangerment of third parties through the drugs exhibited by destroying them. This is what we know at present. We believe that the confiscation is an unjustified intervention into freedom of art. We'd also like to thank Kunst Halle St. Gallen for their ongoing support and the wonderful collaboration. Furthermore, we are convinced, that it is an objective of art to shed light on the fringes of society and to pose fundamental contemporary questions.It appears possible that, in this case, law enforcement was just looking to seize and destroy the contraband products that were purchased by the bot, and may not then seek further prosecution, but it still does raise some interesting questions. I'm not sure I buy the "unjustified intervention in the freedom of art" argument (though that reminds me of another, unrelated story, of former MIT lecturer Joseph Gibbons, who was recently arrested for robbing banks, but who is arguing that it was all part of an "art project").
Still, these legal questions are not going away and are only going to become more and more pressing as more and more autonomous systems start popping up in different areas of our lives. The number of different court battles, jurisdictional arguments and fights over who's really liable are likely to be very, very messy -- but absolutely fascinating.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Filed Under: art project, autonomous bot, autonomous cars, bots, buying, drugs, liability, markets, mediengruppe bitnik
Reader Comments
Subscribe: RSS
View by: Time | Thread
As for the bot, clearly nobody is at fault. The acquisitions were clearly random even if it ended up getting something illegal AND it was an experiment (nobody used the illegal stuff). So I'd say the seizure was disproportionate even if I agree that law enforcement should have checked what was going on.
[ link to this | view in chronology ]
Response to: Ninja on Jan 23rd, 2015 @ 8:13am
You'd like to think that, wouldn't you! Clearly this can only be solved by the "War on Bots". Uncle Sam will require billions of dollars worth of military hardware bought from crony capitalists. Every computer will need to be fitted with a keylogger and uniquely identifiable microchip. Encryption will need to be banned. Warrantless no - knock raids on any place suspected to harbor a bot. Millions will need to be jailed in for -profit prisons. You may be ready to surrender to the autonomous horror, Mr. Ninja, but some of us are willing to do what it takes to protect our profits. .. peace. I meant peace.
[ link to this | view in chronology ]
Re:
One could argue that the *sellers* were the ones at fault, for putting the stuff up where the robot would stumble upon it.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
[ link to this | view in chronology ]
They waited until the end of the exhibition
There's also a matter of context: until the exhibition ended, it was a work of art. After it ended, it's just a bunch of drugs and other stuff stored somewhere.
[ link to this | view in chronology ]
Some of This Happens All the Time
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
Yes, there is. Quantum decay, avalanche diode noise, etc. All modern operating systems mix together several sources of random numbers, and make the result available to applications.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re: Re: Re:
A lot of recent Intel processors have that special equipment directly on the CPU (it's used by the RDRAND and RDSEED instructions). If you have a TPM (and it's enabled), it also has that special equipment. So, not that uncommon.
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re:
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re:
First, using thermal noise as the entropy source is extremely good. However, the thermal noise present in the system is affected by the whatever operations the computer is performing as well as the physical environment it is operating in. A true source of entropy is not affected by any environmental or operating conditions. For most purposes, it's certainly close enough -- but not for all, particularly when it comes to cryptography. There are a number of instances where mathematically strong crypto (or even unbreakable crypto such as OTPs) have been compromised because the random numbers were only nearly random.
Second, the entropy source is not directly the source of the random numbers used. It is used to seed a random number generator -- which, again, does not actually generate random numbers. Even if the entropy source were perfectly random, using it to seed a computational random number generator means that you're not actually getting random numbers in the end.
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re:
Consider a circuit which takes the output of a Bernoulli process and chains it into an inverter.
Is the output of that circuit, from the inverter, a Bernoulli process?
[ link to this | view in chronology ]
Re: Re: Re: Re: Re: Re: Re: Re:
Do there exist stochastic processes which are non-stationary in nature?
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
TRNGs [was Re: ]
Random.org: True Random Number Service
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Let's take firearms as a good example. Non-autonomous, traditional firearms, require a human to pull the trigger. That's why gun manufactures can't be held responsible for what the owner of a firearm does with it.
Now let's say the firearms is fully autonomous. Seeking out and finding targets on it's own. I would still argue the owner of the firearm, not the manufacturer, is responsible for what that autonomous system does. The owner is the one who sets up the autonomous system in an area and programs the targeting parameters for the system to seek out.
The only way a manufacturer could be held responsible for an autonomous system's use, is if there's a verifiable software bug in the systems they're selling. Take the Chevy ignition switch defect as an example. Cars were randomly shutting down on drivers due to a design defect in Chevy's ignition system. Hence, in this case the manufacturer is responsible.
So coming back to the autonomous Black market bot. In my opinion, the artists are responsible for the autonomous system because they programmed in the parameters for the autonomous system to execute. They could, for example, have programmed in for the bot to avoid keywords such as 'drug' and 'ecstasy', but they chose not to as both the owner and manufacturer of the autonomous system.
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Re: Re:
But even if the programmers did put in an exclude list, they could have missed something that was illegal someplace and the bot bought it.
Intent is why it is so important in the US legal system....Sorry WAS so important. Doesn't seem that way anymore.
[ link to this | view in chronology ]
Re: Re: Re:
And once they received the contraband, they intentionally kept it instead of destroying it or turning it over to police - so they'd still be guilty of possession even if they weren't guilty of purchasing.
[ link to this | view in chronology ]
Re:
If you set up a shotgun boo trap on the door of your business and it kills a fireman trying to do his job, negligent homicide is probably the least you'd becharged with. Making it a smarter system changes nothing.
[ link to this | view in chronology ]
Re: Re:
[ link to this | view in chronology ]
Re:
Owner of autonomous killing machine knowingly set loose this device on unsuspecting people. The owner is responsible for causing murder.
Programming a bot to anonymously purchase random things is not harming anyone. The people doing wrong are the ones offering to sell (Intent to commit illegal act) and then actually selling and shipping the item. (Committed illegal act)
[ link to this | view in chronology ]
Re: Re:
Funding a criminal enterprise could be considered harm.
[ link to this | view in chronology ]
Re: Re:
But how do you know the seller doesn't have their own bot to purchase random items and resell them for a profit?
[ link to this | view in chronology ]
One missing detail
I think he had an excellent point there.
[ link to this | view in chronology ]
Re: One missing detail
Tax payers win too, no government monies were used to purchase the evidence to crack down on these sort of illegal sales.
[ link to this | view in chronology ]
Same Question with Markets Coming About too
I'm sure you may have seen the video but Kevin Slavin made a presentation on this topic about the book on Amazon with a rogue algorithm that with no interaction at all kept pricing itself higher and higher to where a boring little book on Flies got up to a million dollars to purchase..oh those rogues.
http://ducknetweb.blogspot.com/2011/07/how-algorithms-shape-our-worldted-talks.html
Why do you think we have circuit breakers on the stock exchanges? It's the same reason to stop all trading when the algos and their little bot brains get scrambled and nothing is making sense or trades are done with rogue activity and numbers that are not true. This is a big deal as we are talking big money now.
[ link to this | view in chronology ]
Classic
[ link to this | view in chronology ]
It should be illegal !
(Only the rich and powerful should be allowed to break the law.)
[ link to this | view in chronology ]
So, just how do you go about seizing a 'bot'?
Since this _bot_ wasn't actually a physical _ro_bot (like say R2D2), but a bit of software, what _can_ the cops actually _seize_?
The computer(s) that it's currently running in?
An SD Card (or USB stick) that holds a copy of the software)?
A hard copy print out of the source code?
What if it's running on many computers? Perhaps there are many copies, or maybe it's some kind of hive mind.
If you _turn_off_ the computer (or all of them) that it's running on, wouldn't that be the equivalent of capital punishment? Sounds kind of extreme for making a few drug purchases.
Perhaps you turn off all the computers except one and then take _that_ one into 'custody'?
Do we take into account the intellectual sophistication of the _bot_? We don't prosecute a mentally challenged criminal (or a small child) as harshly as a competent adult criminal do we? Will we now need 'competency hearings' when we arrest a _bot_?
If your adult child commits a crime, we don't arrest their parents. So why would it be O.K. to arrest the programmers of a criminal _bot_?
There's along way to go from there to HAL (or SkyNet).
[ link to this | view in chronology ]
Re: So, just how do you go about seizing a 'bot'?
But if you release your dog into your neighbor's henhouse, you would likely be criminally responsible if he kills the chickens.
[ link to this | view in chronology ]
My point of view
It would be a different situation if the bot were limited to legal marketplaces. In that case, if the bot ended up buying something illegal, the person who deployed it should not be liable, as that outcome would not be one that would be so easily predicted.
[ link to this | view in chronology ]
http://xkcd.com/576/
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Bots are still in their infancy
[ link to this | view in chronology ]
WHY IS THIS A QUESTION?!
Why is this a question?!!!
If we replace people with automation, the person who decided to automate that process or task is still responsible.
Are we so ignorant that we think that just because this new thing exists, that all logic about responsibility VANISHES?
To be clear, there are NO CASES where we should question who is responsible.
NO CASES where a person shouldn't not be held accountable for their actions.
The owner of the self driving car is responsible for it's actions!
The owner of a bot is responsible for it's actions
This question has been answered many times before.
This is not an "interesting" question...
It's SHOCKING that we need to ASK!
[ link to this | view in chronology ]
Re: WHY IS THIS A QUESTION?!
For example, let's say you have a robot that malfunctions and commits a crime without the knowledge, consent, or positive action on the part of the owner. It doesn't seem obvious to me that the owner of the robot should be convicted of the crime that the robot did. The owner did not commit, cause, or condone the commission of the crime, after all.
Perhaps you could argue that owner committed some other offense such as inadequately supervising the actions of the robot, but not he crime the robot itself committed.
"This question has been answered many times before."
No, it really hasn't.
[ link to this | view in chronology ]
Re: Re: WHY IS THIS A QUESTION?!
For instance, if you hire a taxi there, and the driver kills a pedestrian, it is YOU who will go to jail and face charges. In their way of thinking, the accident would never have happened if you had not hired and given instructions to the taxi driver (which is certainly true) and so therfore the driver is basically "just following orders."
[ link to this | view in chronology ]
Re: Re: Re: WHY IS THIS A QUESTION?!
[ link to this | view in chronology ]
Re: Re: Re: Re: WHY IS THIS A QUESTION?!
[ link to this | view in chronology ]
Re: Re: Re: WHY IS THIS A QUESTION?!
Just out of curiosity, what if your instructions included "obey all traffic laws and don't hit anybody"?
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re:
No they're not, as long as they're operating within the established rules of engagement.
[ link to this | view in chronology ]
Ghost in the Shell
[ link to this | view in chronology ]