We'll Have To Wait For The Next Lawsuit To Find Out If A Web Crawler Can Enter Into A Contract

from the settled-away dept

Mon, Apr 30th 2007 6:42am — Mike Masnick

Last month, news spread concerning a somewhat odd lawsuit involving the Internet Archive and the question as to whether or not a computer spider can enter into a contract just by indexing a website. The case involved a woman who ran a website and had put some text at the bottom claiming that just visiting the website was entering into a contract, and part of that contract included not copying or distributing the content. The Internet Archive's spider did what it does and archived the page, leading to the threat of a lawsuit. The Internet Archive preemptively went to court to have a judge say they were in the clear, at which point the woman countersued. Of course, she didn't just countersue for copyright infringement, but a range of charges including racketeering. Most of the discussion focused on whether or not a spider could enter into a contract, though an equally compelling question is whether or not you can automatically force someone to give up their fair use rights. Unfortunately, neither question is going to be decided in this case. WebProNews reports that the woman and the Internet Archive have settled the case out of court with both sides putting happy faces on the story. At the same time, however, WebProNews also reports that the woman in question is still going after some of her critics, including publishing all sorts of personal information about at least two of them, potentially violating some privacy laws (at least one of the critics she's revealing info on is a minor). So perhaps there will still be a lawsuit stemming from this situation after all.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

24 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

TheDock22, 30 Apr 2007 @ 7:06am

Unfortunate
I can not believe they settled since this crazy woman was completely in the wrong. One good argument for bots to not enter into legal agreements is that the entity must understand that they are entering into an agreement and a programming script never could. We do not hold children accountable for entering into legal contracts under the age of 18, so why should we a bot who is not even alive?

I say next time she should directly sue the bot, I would like to see a judge laugh her out of court.
[ link to this | view in thread ]
Casper, 30 Apr 2007 @ 7:25am

Re: Unfortunate
It's all a balancing game. While they know she would lose in court, it does not mean they want to go down that path. If she asks for $50,000 to settle and you can predict that the legal fees will be many times that amount, it just makes business sense to settle (although I have no idea what the actual settlement values were and I would have preferred to see her end up living in a cardboard box for being stupid).

This woman is an example of everything I hate. Not only is she an idiot, but he has a website to spread her idiocy and the audacity to abuse the legal system (as if it hasn't been abused enough). All they need to do to top this off, for me anyway, is find out she is on welfare or something. That would really ruin my day.
[ link to this | view in thread ]
Starky, 30 Apr 2007 @ 7:40am

Re: Re: Unfortunate
I am not a lawyer, but I believe that the Internet Archive could have asked that she be forced to pay their legal fees if they win.
[ link to this | view in thread ]
Brad, 30 Apr 2007 @ 7:45am

Re: Unfortunate
Thats what the robots.txt file is for. if she didn't want it to be archived all she had to do was take five mins and set up that file properly.
[ link to this | view in thread ]
Brian, 30 Apr 2007 @ 7:52am

I want to make a contract!
If that worked, I want to make a page that says by reading this you are now in a contract with me and must pay $100 a month for reading it.
[ link to this | view in thread ]
TheDock22, 30 Apr 2007 @ 8:08am

Re: Re: Re: Unfortunate
Actually, it is pretty rare that a court will force one party to pay the legal fees of the winning party.

In most states, there are laws that only allow payment of legal fees from the losing party if their lawsuit is completely unfounded or basically just wasting the courts time. Since there is no precedent for this sort of lawsuit, it would not be considered frivolous, even if morally it is.

Of course, Internet Archive could always counter-sue her for the legal fees, but that would just mean more legal fees and they couldn't counter-sue her for the counter-suits legal fees.

Silly, isn't it?
[ link to this | view in thread ]
Charles Griswold, 30 Apr 2007 @ 8:13am

Re: I want to make a contract!
I'm going to insert a clause into my web page saying that if you index my page, you agree to give me your first-born child as payment.
[insert evil laugh here]
[ link to this | view in thread ]
charlie potatoes, 30 Apr 2007 @ 8:52am

contracts
If i check the box that says i accept the terms of service on a website, it seems i have entered into a contract. if i program my bot to do the same isn't my bot then entering into a contract in my name?
[ link to this | view in thread ]
thecolor, 30 Apr 2007 @ 8:56am

Re: Unfortunate
In addition to "understanding that they are entering into an agreement", does this woman have this contract on her index (or main home page (1st point of entry)) and if so, should she not have a page prior to the "contract page" indicating that you will be entering a contract if you continue (allowing you to avoid entering)? Seems like entrapment if you tell someone you are "now" (after you've done it) in a contract since you entered my page, without knowing ahead of time that you would be in a contract? :(
[ link to this | view in thread ]
TheDock22, 30 Apr 2007 @ 8:57am

Re: contracts
Not really. You can not have someone else sign your name for you and enter into a legal binding contract without your knowledge/consent.
[ link to this | view in thread ]
TheDock22, 30 Apr 2007 @ 9:03am

Re: Re: Unfortunate
In addition to "understanding that they are entering into an agreement", does this woman have this contract on her index (or main home page (1st point of entry)) and if so, should she not have a page prior to the "contract page" indicating that you will be entering a contract if you continue (allowing you to avoid entering)?

Not really, she can have it on the first webpage. The contract simply states you can not use her material or link to it. The disclaimer seemed to be on the first page you visited so people know what the contract is. They can either a) view the information, b) leave the site, or c) use the content and be sued. Since most people will not use the content, it is a pretty solid disclaimer.

The issue is whether a bot, hence the bot making company, can be sued for "agreeing" to the terms, but still linking to the content. I say no since a bot is not human, but then again I am not a judge. Beside, she did not reasonable try to avoid the bot linking to her content using robot.txt.

I think we should call this woman "black widow" since that is obviously what she is doing. Setting a trap and then making the kill.
[ link to this | view in thread ]
Robert Johnson, 30 Apr 2007 @ 9:06am

Brad is right
that is EXACTLY what the robots.txt is for. Anyone using the web to post material SHOULD know that. SHE should be paying the court costs, unless it is found the spider was ignoring robot.txt files.

When we drive a car, it is asumed we know certain things about it. We even must test our knowledge on it before being allowed to drive.

Perhaps this is were computer use is headed. Before being allowed to access the information highway, you need a liscence. I know a LOT of people who really have no clue about using a computer, and accessing the Internet? I get calls on a regular basis from friends to fix very basic Internet releated PC problems. Problems that should never occur, if the had more than "a little knowledge."
[ link to this | view in thread ]
Casper, 30 Apr 2007 @ 9:11am

Re: Re: Re: Unfortunate
The other portion of her problem is her site design. You can not have a disclaimer on the "first page" of a site if someone can reference another page directly. What if someone hits your site through a search and bypasses the main page directly to content that has not dislaimers or provisions? This one page disclaimer works for a flash or other programmed site that has a single control that changes the content, but it really won't hold up if the pages are individuals.

Every page should be copyrighted or have a disclaimer if you want it noted.
[ link to this | view in thread ]
TheDock22, 30 Apr 2007 @ 9:25am

Re: Re: Re: Re: Unfortunate
I was under the impression every website and webpage did have the disclaimer.

She is an ex-lawyer, so chances of her overlooking disclaimers are slim to none. Does anyone actually know where this website is? I want to see it now.
[ link to this | view in thread ]
Anonymous Coward, 30 Apr 2007 @ 10:09am

Re: Re: Re: Re: Re: Unfortunate
The woman's website is: http://www.profane-justice.org/

I guess she now does NOT allow any hot-linking to the entire website - not just individual pages. The only way to get into the pages is to manually copy the link and paste it into a new window.... though, that seem like much more work than it's worth.
[ link to this | view in thread ]
lizard, 30 Apr 2007 @ 10:59am

oh my god. i just followed that link and got a 403 forbidden. i then removed the 'www' from the link and got an ugly page that **wanted access to my clipboard**. wtf did it want with the information in my clipboard?

site looked like it was all text, but last time i saw that site it had a 'design' -- and both times it made me want to scoop out my eyejuice with rusty fishhooks rather than gaze upon its wretchedness.

i exaggerate, but only slightly.
[ link to this | view in thread ]
Steven, 30 Apr 2007 @ 11:38am

HA!
If you go to http://www.profane-justice.com there is a disclaimer and it asks you to accept an agreement by clicking a link, but if you go to http://www.profane-justice.com/profanejustice/index.html and you don't run JavaScript (thank you NoScript), there is no such acceptance of agreement.

The site looks pretty crappy without JavaScript, but the JavaScript doesn't help much :)
[ link to this | view in thread ]
Ryan, 30 Apr 2007 @ 11:48am

no-follow/no-index anyone?
Does the Internet Archive not adhere to the no-index meta tag that you can put in your documents?

The statement is all fine and dandy for a person, but these types of meta-tags are there specifically for this kind of thing. If she had a no-index there and the Internet Archives ignored it, then she has a point. Otherwise, she's completely in the wrong IMO.
[ link to this | view in thread ]
JJ, 30 Apr 2007 @ 12:00pm

robots.txt
A web crawler can not enter a contract under contract law. There are a few things to consider when agreeing to a contract, one is capacity. Robots do not have the capacity to read and analyze the contract, therefore voiding the contract.

This is why web crawlers should be granted permission in advance to browse a site through the robots.txt file. "Every" website that wants to be indexed should contain a robots.txt file. (this is not currently the case)

If a site does not have a robots.txt file, then it should illegal for anyone to index it. Everyone who wants to be indexed will create a robots.txt file. Everyone who is stupid will not have one.
[ link to this | view in thread ]
Anonymous Coward, 30 Apr 2007 @ 4:51pm

People need to stop thinking about a suit against a "robot." Of a person gets into a car accident, they do not sue the other car, they sue the cars owner, driver or manufacturer (all all three). The issue is whether the person or company who put the robot out there is entering into the contract but entering, through the use of the robot.
[ link to this | view in thread ]
Charles Griswold, 30 Apr 2007 @ 5:16pm

Re: robots.txt
If a site does not have a robots.txt file, then it should illegal for anyone to index it. Everyone who wants to be indexed will create a robots.txt file. Everyone who is stupid will not have one.

Bad idea. Most people either want their site indexed or just don't care. Why should those people be inconvenienced for the sake of a stupid, whiny minority?
[ link to this | view in thread ]
Anonymous Coward, 30 Apr 2007 @ 7:02pm

Re: Re: robots.txt
Bad idea. Most people either want their site indexed or just don't care. Why should those people be inconvenienced for the sake of a stupid, whiny minority?

It is actually a good idea. It does not matter at all that most people want their site indexed. If they want their site indexed, they should create a robots.txt file. The ignorant probably do not even have quality content to be listed, anyways.

Search engines should be required to ask permission before copying content. The robots.txt file is perfect for this request.

Can I copy other peoples websites without asking? No. Why should robots be allowed.

This idea would prevent ANY future court cases on this topic.
[ link to this | view in thread ]
Anonymous Coward, 2 May 2007 @ 12:31am

There still is no robots.txt at that site.

Are we surprised?
[ link to this | view in thread ]
|333173|3|_||3, 2 May 2007 @ 6:57am

Re: Re: Re: robots.txt
The web operates on the principle that everything is open to use unless you specify otherwise, and enforce that. Indeed, the internet as a whole works the same way: I have, through my ISP, acces to everything that no-one has secured.
Maybe all websites should have a set of standard, W3C mandated, Terms of Use, which apply to robots, which simply states that the robots.txt file must be obeyed by robots, provided it meets W3C standards for the file. Naturally, this can only apply to W3C-compilant sites, but provided each country's implementation of the mandate states that if there is no valid robots.txt file the site is considered to be open to robots to acces however the programmer desires.
[ link to this | view in thread ]