B2B Corpse Exhumed For Yet Another Autopsy

Is It Your Fault If Someone Else's Spam Filter Catches Your Important Email?

from the playing-the-blame-game dept

Wed, May 24th 2006 2:29am — Mike Masnick

We've all heard the stories about overeager spam filters catching messages they shouldn't -- but now there's a case on hand that ended up costing a school district a quarter of a million dollars by not awarding a phone contract to the cheapest bidder, because a spam filter ate an important email. However, where the story gets even more interesting is in the details, as people start figuring out who to blame. Normally, you'd say that anyone making a bid on such a big contract should know not to trust that an email had arrived safely. However, this email wasn't the original bid (which was delivered in a hard copy), but a followup question from the district which clearly stated: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further." After all that, though, the school district is still blaming the vendor, saying a good bidder knows it needs to outwit spam filters.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

48 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

Brad, 24 May 2006 @ 2:39am

Hah!
As long as we can blame anyone but ourselves, we're safe.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 3:23am

why is it so hard for people to hold up there hands and admit that they were wrong or made a mistake? things would go a lot smoother if they did

or am i living in a fantasy world there?
[ link to this | view in thread ]
?, 24 May 2006 @ 3:45am

Sounds like the people in charge of education
Have some learning to do themselves.
[ link to this | view in thread ]
Peet McKimmie (profile), 24 May 2006 @ 4:43am

Can someone clarify, please...?
Was it the school district's follow-up question that was eaten by the vendor's SPAM filter, or was it the vendor's reply that was eaten by the school district's SPAM filter? I can't work it out from the article above...
[ link to this | view in thread ]
William Lane, 24 May 2006 @ 4:46am

Spam Filter
Why would ANYONE send an important email and not have a confirmation sent back to let the sender know his email has been opened. If the confirmation was not returned in a timely manner or received, then the sender could take action to find out if they received it.
[ link to this | view in thread ]
lar3ry, 24 May 2006 @ 5:21am

No such thing as a confirmation
The Entire World Doesn't Use Outlook.
Just because Outlook and a few other mailers support delivery status notification (DSN), there is no guarantee that anybody that you send email will support it. There is no guarantee of delivery of email--it can be rejected (you'll get notified), or it will be silently discarded like the SPAM filters of some ISPs.
The relevant email protocols allow for delivery failure notification (in other words, "don't bother us, spammer!") to inform the sender that the email wasn't received. However, it's very simple to configure most mail transfer agents to silently drop messages. I think the reasoning is that SPAM affects network traffic (there's so much of it) that sending out failure notices increases the network traffic, so you combat SPAM by simply ignoring it.
I had an interesting thing happen about 11 years ago. Somebody tried to send an email with the name of a wine in it. Parental filters (not mine, but the sender's) kept on refusing to deliver the email, saying the message was inappropriate. A LOT of digging turned out that the word "Chardonnay" has a "bad word" in character positions 2-7.
Back to the main point, how can the vendor know if his message was tagged as SPAM if he didn't get a message indicating failure of delivery (again, this is not the same as "confirmation of delivery")?
The onus should be on ISPs to simply route email properly--or send back a failure stating why the message wasn't delivered. However, the ISPs are acting much like the old Bell System of the 1970s, which Lily Tomlin lampooned as Ernestine the Telephone Operator: "We're the phone company. We don't care. We don't have to."
Since this cost the vendor money, I think the company that lost out has a good case against the ISP or maintainer of the SPAM filter. (Oh, my... am I advocating another lawsuit???)
[ link to this | view in thread ]
Starseed, 24 May 2006 @ 5:26am

Here's the solution... Check your junk mail every once in a while! Not that difficult. The school district is clearly to blame here. They made a mistake. Woop! It happens, it just happened to be a pricey one.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 5:45am

Blame whatever management insisted on a spam filter. Personally, I find not giving my email address out to everyone in the world, not using it on sites, and just using the delete button seems to work best, at least I don't miss anything.
[ link to this | view in thread ]
bmee, 24 May 2006 @ 5:50am

Ask for an answer
School district should have checked their spam messages.

Another mistake: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further."
I think the vendor should have required a response to both cases: want or don't want to be considered further. If they don't get a response they may insist, maybe a phone call, just to hear "oops! it was on the spam folder"
[ link to this | view in thread ]
Dave, 24 May 2006 @ 5:53am

spam filter catches important email
Dear teach, the spam filter ate my homework ..
[ link to this | view in thread ]
ITBubba, 24 May 2006 @ 5:57am

Response to "No such thing as a confirmation by la
If the vendor was blocked by the schools spam filter then all it takes is a phone call. If you pick up the phone and make a call you will be able to determine if your email was received. If you are a good salesperson then you follow up on ALL correspondence. The blame is on the vendor for not following up with the school to verify that the email was received.

If the school was blocked by the vendors spam filter then the blame still lies with the vendor. Don't block potential clients emails.

The customer is NEVER to blame when YOU are trying to bid on work. It comes down to how bad the vendor really wanted the contract. It looks like they didn't want it enough.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 6:07am

Re: Response to "No such thing as a confirmation b
"The customer is NEVER to blame when YOU are trying to bid on work. It comes down to how bad the vendor really wanted the contract. It looks like they didn't want it enough."

Sure they are. If you follow their instrustions on how to submit the bid, or in this case an update, and the customer drops the ball how is it the vendor's fault?
[ link to this | view in thread ]
Skippyboy, 24 May 2006 @ 6:15am

Re: Anonymous Coward Fantasy World
Boss! Boss! de Plane! de Plane!

Does that answer your question? :P
[ link to this | view in thread ]
I, for one, 24 May 2006 @ 6:27am

Mail systems
Look at old legal case histories and you will see thousands of "I sent it. No you didn't. Yes I did..." yarns. At one time in England the Royal Mail was considered infallible. A mail coach could be impeded by no one, they carried armed guards, on recipt it became the property of the crown and obstructing the Kings Mail was a crime that carried the death penalty. Consequently an entire branch of law was built on the idea that once you placed a letter in the box it was tantamount to having been delivered to the recipient. While this was mostly for the benefit of those seeking money, it did set a useful standard in law, even if it didn't work in practice and was open to abuse. Many contract law disputes have been settled on the basis of when an invitation or response was _sent_, not when it was recieved. The evidence in those cases would have been the franking mark of the post office closest to the sender.

To my knowledge as it stands, the law, both in the UK and the USA, is in complete disarray regarding email. Lawyers and judges have been quietly ignoring the issue hoping it will go away for a long time. I'm not even sure there is any legal basis to an electronic communication, since it is not possible to confirm or deny it, it carries no signature and has no certainty of delivery. Real, serious companies still do their business on paper, CEOs carry a pen in their jacket pocket.

Furthermore, this state of affairs benefits many companies because of the manner in which they conduct business and their attitude to customers. They hide behind a wall of unaccountability, isolated as far as they can be from the rest of the world. Just try finding a phone number for any company these days. As such they can deny the receipt of any inconvenient communication.

Good advice: If you ever want to really achieve something, get people jumping and asking "How high Sir?!" send a registered letter (I don't know what you call this in the US)
[ link to this | view in thread ]
me, 24 May 2006 @ 6:29am

Re: Can someone clarify, please...?
um... how could the vendor reply to an email that they never received? It's really not that hard to deduce on your own.
[ link to this | view in thread ]
Mikhail, 24 May 2006 @ 6:47am

Re: No such thing as a confirmation
Correct, the entire world does not use outlook. However, the entire world of email users is capable of typing "Please send me a reply to this email confirming that you have received it." in their email's text.

If you don't receive the reply, you can simply send another email - if it was the case where the School District was unwilling to take phone calls and would get irritated by receiving multiple emails, then they weren't looking for the lowest bidder to begin with (and why should they?).
[ link to this | view in thread ]
SPR, 24 May 2006 @ 6:49am

Important Email
It is foolish to depend exclusively on email for anything important. While email is relatively reliable it can fail, and usually does, at the worst possible moment.
[ link to this | view in thread ]
Mark, 24 May 2006 @ 7:00am

Another typical problem is lack of technical knowl
We see more and more companies putting in spam attacking solutions and nobody knowing how it works. Even if we know a message fails - how do you get it through if there's nobody technical at the filtering side who knows how the thing works.

And worse than just keyword filters, is those which require DNS to be a certain way - for message headers to match machine names - and things like that.

We've had to resort to private yahoo accounts to transfer files - assuming you don't have time to snail-mail them.
[ link to this | view in thread ]
Nilt, 24 May 2006 @ 7:27am

Ever hear of a Read Receipt?
Yeah, sure not everyone uses Outlook. I'm pretty sure, though, anyone bidding on a wireless contract would have a mail client capable of handling read receipts.

For crying out loud, if those things weren't intended or just this sort of thing, I don't know what they were intended for.
[ link to this | view in thread ]
Michael, 24 May 2006 @ 7:27am

But How?
"If the vendor was blocked by the schools spam filter then all it takes is a phone call. "

HAHAHAHAHA!!! How could they call? The school was trying to get a contract for their phones!

Michael
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 7:38am

Courier Fallacy
There's an old story about Napolean and the courier fallacy...or something like that. Goes something like...

N sends a message to a subcommander to attack at a very specific time, in order to coordinate an attack. It is imparative that the subcommander replies that he received the message so that N doesn't attack without support. The Subcommander receives the order, confirms, and sends a reply. Now, the subcommander needs to know that N received his reply so that he knows to attack at the coordinated time. If N doesn't receive the confirmation, then the subcommander may be left to attack without support, so he needs confirmation that N received confirmation.

And so it continues back and forth.

The moral....if it's important (and I consider anything worth > $1000 important) talk to someone in person or over the phone. It's a bad policy of the school to rely on email. Change the policy and move on. Is the vendor entitled to any compensation...nope. It was a mistake. Not every mistake requires a lawsuit and compensation. That's how we've become the litigious society that we are.
[ link to this | view in thread ]
Charlie, 24 May 2006 @ 7:44am

Spam Filters
I have my MTA configured to generate a permanent delivery error when it recieves spam, and then deliver the message anyways. If the spam is coming from a custom SPAM MTA, the error is ignored. If it is a real mail server, it should generate a bounce to allow the sender to correct the error.

It isn't rocket science.
[ link to this | view in thread ]
Tom, 24 May 2006 @ 7:47am

The "blame game"...
... and of course, nobody is accountable here. Least of all the school district who is obviously held hostage by the technology that they serve.
I am in the middle of trying to deal with a startup who wants my help, but keeps losing my emails because of yahoos spam filters. IMO, anyone who doesn't understand the infrastructure they are beholden to deserves whatever mediocrity life brings them- try running a business from a phone booth instead.
[ link to this | view in thread ]
Dan, 24 May 2006 @ 7:48am

Re: Can someone clarify, please...?
It was the Vendor's reply that was eaten by the school district's spam filter.

And yet, it's not their fault... ;)
[ link to this | view in thread ]
JerseyRich, 24 May 2006 @ 7:49am

Typical...
Typical bureaucratic "it ain't my fault" BS.

If you REQUIRE responses via email then you bear the burden of making sure the email gets received.

Sure, it wasn't the district's fault, but they should acknowledge that it wasn't the businessman's either.

The district finance director's comment about"a good businessman should follow up to make sure it was received" is a cop-out, plain and simple.
[ link to this | view in thread ]
Tom, 24 May 2006 @ 8:03am

Re: Because admitting fault is akin to a guilty pl
If you hold up your hand and say "My fault, sorry" you have just admitted liability for which you will pay. Who wants to volunteeer if you can wait for the courts to decide that you really are at fault; they may decide you aren't after all. . It is a fantasy to believe that as an adult you can just admit you're wrong and say you're sorry and then nothing will happen. that may work with friends, but not most everywhere else.
[ link to this | view in thread ]
Uncommon Sense, 24 May 2006 @ 8:30am

What the??
First, if you send a life-and-death email, forward a copy to self as a proof (it's kinda like certified mail), at the very least it gives you something to show as evidence.

Second, the bidder states that he sent multiple emails yet only the most critical failed. I believe there may be more to this story than at first glance...but that's another story about unfair bidding practices.
[ link to this | view in thread ]
Bryan Price, 24 May 2006 @ 8:32am

Bad spam filters
I've run into this with my personal website's ISP. Their spam filter doesn't filter, exactly, it throws it away, so I don't even get a chance. He claims that there's nothing to be done about it, although I tend to doubt it. Since the ISP belongs to a personal friend, that's also a contention.
[ link to this | view in thread ]
davidlow, 24 May 2006 @ 8:50am

What ever happened to Zoemail??
I'm surprised I never see any mention of Zoemail.com as a
complete and absolute solution to spam problems. It's been doing
the job for me for over two years. There is no filtering, so there's
no danger of blocking any friendlies. None. I thought for sure it
would catch on. Am I alone?

I just posted the same comment over at CNet's version of the same story.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 8:54am

Re: by Starseed
I worked at a school division as a central administrator. The SPAM filter caught about 16,000 emails a day. If you want to go through all of them, go right ahead, but it is far too time consuming to go through the JUNK MAIL folder every once in a while, as you suggest.

Often, if someone was expecting an email from someone, and it didn't come through, it was a fairly simple job to go through the filter and find the email (because we knew who it was from), and add that sender to the safe list.

However, if the school was not expecting the email, they would never have known to go in and check for it.
[ link to this | view in thread ]
joe stacey, 24 May 2006 @ 9:08am

The vendor should have been aware of the unreliability of email. If they don't know this, they shouldn't have a business. It's basic stuff.

Anyone business owner with a brain should know to follow up on an email of this level of importance.

derhh...
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 10:10am

Re: No such thing as a confirmation
I worked on a spam filter system for businesss where the message was silently delete, AFTER being stored in web friendly interface for retrieving spam.

I think ISP should be required to store spam for at least 30 days in a location its users can access.

However, just like regular mail, you get no guarantee of the mail being recieved unless you require delivery notification or a signature. Similar services exist for email and should be used for important emails like this. Even without spam filters, email delivery is not guaranteed. Servers, proxies, mail programs, etc can all fail along the way, losing the mail in the process.

PS - The spam filter I worked on knew the difference between "hardon" and " hardon ".
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 10:11am

Re: Re: by Starseed
Good spam filters are able to give a degree of certainty that a mail is spam. High certainty can be ignored, lowe certainty should be investigated. The problem is most spam filters at the ISP level just drop the message, never alerting anyone that it happened.
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 10:24am

Re: What ever happened to Zoemail??
Once spammers catch on to this service they will simply send to "Sally@example.com" and "Sally.realword@example.com". What will you do then?
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 10:27am

Whitelist words
Good spam filters support whitelist words. If the word is in the subject or body of the email, the mail proceeds, NEVER being filtered. The word could be your business name, and address, a phone number, or a nonsense word. You would put this keyword in your outbound message so that the reply would have it and go through.

Blocking real spam is easy, spammers only use a few tricks to get around spam filters, these tricks are so obvious that they give it away as being spam. Of course you need a qualified human at the filtering end to pick up on new trends and key these in. This is where spam filters fail, no one wants to pay someone to maintain the spam filter.
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 10:29am

Re: What ever happened to Zoemail??
The spam filter I worked on had a similar service. It allowed you to make throwaway emails, a random string in front of your domain. Once the email was used for a limited number of times, or a limited amount of time (your preference), it was deleted. You would use these trash emails at retailers and such that are guaranteed to spam you, but you need to get a confirmation email.
[ link to this | view in thread ]
ZA, 24 May 2006 @ 11:27am

Re: Re: Can someone clarify, please...?
I sure hope someone lost their job over losing, what must have been a multi-million dollar contract. How difficult is it for a sales person to follow up with a phone call? "Hey, did you receive the email information that you requested?"

"No? Really? I know it was sent. Let me try to find out why you didn't receive it."

Thank you for playing; have a nice life!
[ link to this | view in thread ]
Another District, 24 May 2006 @ 12:30pm

I Feel Their Pain
I am the IT Director of a very large school district. Most people posting comments here have obviously never managed a very large e-mail system.

We use a spam filter for our district's e-mail system. The filter rules, white lists, black lists, etc. are downloaded daily from the manufacturer of the filter (very much like antivirus software). We estimate the filter to be about 90% effective removing junk e-mail. We block approximately 60,000 messages per day. It is not the same thing as desktop junk e-mail software that you all are saying takes a few minutes to check each day. The mail is filtered before it reaches the mailbox. There's simply no practical way to check it for "good" e-mail every day. And for those of you that are not aware, most school districts have an IT staff level about 80% smaller than corporations of similar size. So before you all condemn the school district for missing the e-mail, you should realize we aren't talking about desktop filters and staff members that have time to review 60,000 messages per day.

Filtering systems are a fact of life, and like web filters, junk mail filters are not perfect. The more effective they are at blocking junk e-mail, the more likely they are to accidentally block good e-mail. Security comes at a cost of convenience and usability. And this is one of the disadvantages of spam filters.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 1:36pm

Re: I Feel Their Pain
severely understaffed, but as head of the department you seem to have time to surf the net for worthwhile tripe like this.

And yet the schools demand milliions more dollars because they are sooooo overworked.

Pathetic.
[ link to this | view in thread ]
mthorn, 24 May 2006 @ 1:37pm

Re: I Feel Their Pain
Each recipient doesn't get 60,000 messages a day. The recipient should be responsible for determining if a message is good or not. This drastically reduces the amount of messages that need to be checked per user. It's not the IT departments job to determine the validity of an email, they don't know what the message is about, nor should they.

Spam filters can also determine if it is most likely spam, or definitly spam. The definites (the majority of spam) do not need to be searched by a user unless they can't find the message they are expecting in their inbox.

Reguardless of the amount of spam or how it is determined to be spam, the recipient should have a way to find it, most conveintly on a web interface. It should not be up to the IT department to trash messages they think is spam.

I worked on a system that had hundreds of thousands of junk email a day, each one archived. This archived came in handy many times when messages were inappropriatly blocked. This became and invaluable tool for use to determine why it was falsely blocked and how to prevent it again.

If your answer is "we have so much spam our only choice is to delete it", then you are using the wrong software, or you don't have the staff qualified to answer the question correctly.
[ link to this | view in thread ]
Anonymous Coward, 24 May 2006 @ 1:40pm

Re: I Feel Their Pain
Did you read that description of the spam filter off the back of the box you got at Best Buy?

Maybe if you employed an internet filter as well, teachers and administrators wouldn't be surfing the net all day eliciting spam from retailers.
[ link to this | view in thread ]
Anonymous Howard, 24 May 2006 @ 3:17pm

Re: Re: I Feel Their Pain
Maybe before judging him and assuming the worst you might look at the time stamp. 12:30 PM. MAYBE he was on lunch break. Maybe not, I don't know what time zone he posted from. Maybe he uses Techdirt to keep on top of the situation so his school distrcit is aware of these problems. Maybe (more likely Probably) you're just an arse!
[ link to this | view in thread ]
*Rolling eyes*, 24 May 2006 @ 8:44pm

Re: But How?
There's always someone to call. If the schools actually had no phones (not just looking for a new contract when their current one expired) then there would have been a point of contact at a district office or somewhere else.

Think before you open your dumb mouth.
[ link to this | view in thread ]
Anonymous Coward, 25 May 2006 @ 10:27am

Re: Spam Filter
How do you "have" a confirmation sent back when you don't run the receiving system? You can "request" confirmation, but that is far different from commanding it. And, by the way, most people know that receipt confirmation requests are almost always universally disregarded anyway.
[ link to this | view in thread ]
Anonymous Coward, 25 May 2006 @ 10:36am

What if someone wanted to rig the bidding process? Requiring responses by email and then using filters to selectively delete certain replies would be an effective way of doing so and then denying culpability.
[ link to this | view in thread ]
Anonymous Coward, 25 May 2006 @ 10:39am

Re:
Then they would be violating the bidding instructions which might itself be a basis for disqualification.
[ link to this | view in thread ]
Peet McKimmie (profile), 1 Jun 2006 @ 7:48am

Re: Re: Can someone clarify, please...?
"It's really not that hard to deduce on your own."

From what it says up above, the school sent a message saying that the vendor had to reply by email. It also says that an unspecified message was eaten by a SPAM filter. It doesn't say whether the message that was eaten was the school's request for a reply or the vendor's reply, and it doesn't actually say that the vendor replied.

In summary, yes, it really is that hard to deduce on your own without any further information, which your "helpful" reply doesn't contain.
[ link to this | view in thread ]
Anonymous Coward, 8 Jun 2006 @ 5:31am

Yet another example of the judicial system gone awry.

The public and the law is so highly against SPAM and malicious activity that if we do not implement protection we could be held very accountable if who knows what could happen, especially in an educational institution, yet, when these same tools block a single email that institution is fined for doing what it has to do.

No one can be held accountable for using SPAM filters. Period. End of story. If you are relying so heavily on e-mail then you have absolutely no grasp of reality and definately don't deserve your job.
[ link to this | view in thread ]