Is It Your Fault If Someone Else's Spam Filter Catches Your Important Email?
from the playing-the-blame-game dept
We've all heard the stories about overeager spam filters catching messages they shouldn't -- but now there's a case on hand that ended up costing a school district a quarter of a million dollars by not awarding a phone contract to the cheapest bidder, because a spam filter ate an important email. However, where the story gets even more interesting is in the details, as people start figuring out who to blame. Normally, you'd say that anyone making a bid on such a big contract should know not to trust that an email had arrived safely. However, this email wasn't the original bid (which was delivered in a hard copy), but a followup question from the district which clearly stated: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further." After all that, though, the school district is still blaming the vendor, saying a good bidder knows it needs to outwit spam filters.
Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.
Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.
While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.
–The Techdirt Team
Reader Comments
Subscribe: RSS
View by: Time | Thread
Hah!
[ link to this | view in chronology ]
or am i living in a fantasy world there?
[ link to this | view in chronology ]
Re: Anonymous Coward Fantasy World
Does that answer your question? :P
[ link to this | view in chronology ]
Re: Because admitting fault is akin to a guilty pl
[ link to this | view in chronology ]
Sounds like the people in charge of education
[ link to this | view in chronology ]
Can someone clarify, please...?
[ link to this | view in chronology ]
Re: Can someone clarify, please...?
[ link to this | view in chronology ]
Re: Re: Can someone clarify, please...?
From what it says up above, the school sent a message saying that the vendor had to reply by email. It also says that an unspecified message was eaten by a SPAM filter. It doesn't say whether the message that was eaten was the school's request for a reply or the vendor's reply, and it doesn't actually say that the vendor replied.
In summary, yes, it really is that hard to deduce on your own without any further information, which your "helpful" reply doesn't contain.
[ link to this | view in chronology ]
Re: Can someone clarify, please...?
And yet, it's not their fault... ;)
[ link to this | view in chronology ]
Re: Re: Can someone clarify, please...?
"No? Really? I know it was sent. Let me try to find out why you didn't receive it."
Thank you for playing; have a nice life!
[ link to this | view in chronology ]
Spam Filter
[ link to this | view in chronology ]
Re: Spam Filter
[ link to this | view in chronology ]
No such thing as a confirmation
Just because Outlook and a few other mailers support delivery status notification (DSN), there is no guarantee that anybody that you send email will support it. There is no guarantee of delivery of email--it can be rejected (you'll get notified), or it will be silently discarded like the SPAM filters of some ISPs.
The relevant email protocols allow for delivery failure notification (in other words, "don't bother us, spammer!") to inform the sender that the email wasn't received. However, it's very simple to configure most mail transfer agents to silently drop messages. I think the reasoning is that SPAM affects network traffic (there's so much of it) that sending out failure notices increases the network traffic, so you combat SPAM by simply ignoring it.
I had an interesting thing happen about 11 years ago. Somebody tried to send an email with the name of a wine in it. Parental filters (not mine, but the sender's) kept on refusing to deliver the email, saying the message was inappropriate. A LOT of digging turned out that the word "Chardonnay" has a "bad word" in character positions 2-7.
Back to the main point, how can the vendor know if his message was tagged as SPAM if he didn't get a message indicating failure of delivery (again, this is not the same as "confirmation of delivery")?
The onus should be on ISPs to simply route email properly--or send back a failure stating why the message wasn't delivered. However, the ISPs are acting much like the old Bell System of the 1970s, which Lily Tomlin lampooned as Ernestine the Telephone Operator: "We're the phone company. We don't care. We don't have to."
Since this cost the vendor money, I think the company that lost out has a good case against the ISP or maintainer of the SPAM filter. (Oh, my... am I advocating another lawsuit???)
[ link to this | view in chronology ]
Re: No such thing as a confirmation
If you don't receive the reply, you can simply send another email - if it was the case where the School District was unwilling to take phone calls and would get irritated by receiving multiple emails, then they weren't looking for the lowest bidder to begin with (and why should they?).
[ link to this | view in chronology ]
Re: No such thing as a confirmation
I think ISP should be required to store spam for at least 30 days in a location its users can access.
However, just like regular mail, you get no guarantee of the mail being recieved unless you require delivery notification or a signature. Similar services exist for email and should be used for important emails like this. Even without spam filters, email delivery is not guaranteed. Servers, proxies, mail programs, etc can all fail along the way, losing the mail in the process.
PS - The spam filter I worked on knew the difference between "hardon" and " hardon ".
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Re: by Starseed
Often, if someone was expecting an email from someone, and it didn't come through, it was a fairly simple job to go through the filter and find the email (because we knew who it was from), and add that sender to the safe list.
However, if the school was not expecting the email, they would never have known to go in and check for it.
[ link to this | view in chronology ]
Re: Re: by Starseed
[ link to this | view in chronology ]
[ link to this | view in chronology ]
Ask for an answer
Another mistake: "We must have our response to this request returned via e-mail ... in order for your company's response to be considered further."
I think the vendor should have required a response to both cases: want or don't want to be considered further. If they don't get a response they may insist, maybe a phone call, just to hear "oops! it was on the spam folder"
[ link to this | view in chronology ]
spam filter catches important email
[ link to this | view in chronology ]
Response to "No such thing as a confirmation by la
If the school was blocked by the vendors spam filter then the blame still lies with the vendor. Don't block potential clients emails.
The customer is NEVER to blame when YOU are trying to bid on work. It comes down to how bad the vendor really wanted the contract. It looks like they didn't want it enough.
[ link to this | view in chronology ]
Re: Response to "No such thing as a confirmation b
Sure they are. If you follow their instrustions on how to submit the bid, or in this case an update, and the customer drops the ball how is it the vendor's fault?
[ link to this | view in chronology ]
But How?
HAHAHAHAHA!!! How could they call? The school was trying to get a contract for their phones!
Michael
[ link to this | view in chronology ]
Re: But How?
Think before you open your dumb mouth.
[ link to this | view in chronology ]
Mail systems
To my knowledge as it stands, the law, both in the UK and the USA, is in complete disarray regarding email. Lawyers and judges have been quietly ignoring the issue hoping it will go away for a long time. I'm not even sure there is any legal basis to an electronic communication, since it is not possible to confirm or deny it, it carries no signature and has no certainty of delivery. Real, serious companies still do their business on paper, CEOs carry a pen in their jacket pocket.
Furthermore, this state of affairs benefits many companies because of the manner in which they conduct business and their attitude to customers. They hide behind a wall of unaccountability, isolated as far as they can be from the rest of the world. Just try finding a phone number for any company these days. As such they can deny the receipt of any inconvenient communication.
Good advice: If you ever want to really achieve something, get people jumping and asking "How high Sir?!" send a registered letter (I don't know what you call this in the US)
[ link to this | view in chronology ]
Important Email
[ link to this | view in chronology ]
Another typical problem is lack of technical knowl
And worse than just keyword filters, is those which require DNS to be a certain way - for message headers to match machine names - and things like that.
We've had to resort to private yahoo accounts to transfer files - assuming you don't have time to snail-mail them.
[ link to this | view in chronology ]
Ever hear of a Read Receipt?
For crying out loud, if those things weren't intended or just this sort of thing, I don't know what they were intended for.
[ link to this | view in chronology ]
Courier Fallacy
N sends a message to a subcommander to attack at a very specific time, in order to coordinate an attack. It is imparative that the subcommander replies that he received the message so that N doesn't attack without support. The Subcommander receives the order, confirms, and sends a reply. Now, the subcommander needs to know that N received his reply so that he knows to attack at the coordinated time. If N doesn't receive the confirmation, then the subcommander may be left to attack without support, so he needs confirmation that N received confirmation.
And so it continues back and forth.
The moral....if it's important (and I consider anything worth > $1000 important) talk to someone in person or over the phone. It's a bad policy of the school to rely on email. Change the policy and move on. Is the vendor entitled to any compensation...nope. It was a mistake. Not every mistake requires a lawsuit and compensation. That's how we've become the litigious society that we are.
[ link to this | view in chronology ]
Spam Filters
It isn't rocket science.
[ link to this | view in chronology ]
The "blame game"...
I am in the middle of trying to deal with a startup who wants my help, but keeps losing my emails because of yahoos spam filters. IMO, anyone who doesn't understand the infrastructure they are beholden to deserves whatever mediocrity life brings them- try running a business from a phone booth instead.
[ link to this | view in chronology ]
Typical...
If you REQUIRE responses via email then you bear the burden of making sure the email gets received.
Sure, it wasn't the district's fault, but they should acknowledge that it wasn't the businessman's either.
The district finance director's comment about"a good businessman should follow up to make sure it was received" is a cop-out, plain and simple.
[ link to this | view in chronology ]
What the??
Second, the bidder states that he sent multiple emails yet only the most critical failed. I believe there may be more to this story than at first glance...but that's another story about unfair bidding practices.
[ link to this | view in chronology ]
Bad spam filters
[ link to this | view in chronology ]
What ever happened to Zoemail??
complete and absolute solution to spam problems. It's been doing
the job for me for over two years. There is no filtering, so there's
no danger of blocking any friendlies. None. I thought for sure it
would catch on. Am I alone?
I just posted the same comment over at CNet's version of the same story.
[ link to this | view in chronology ]
Re: What ever happened to Zoemail??
[ link to this | view in chronology ]
Re: What ever happened to Zoemail??
[ link to this | view in chronology ]
Anyone business owner with a brain should know to follow up on an email of this level of importance.
derhh...
[ link to this | view in chronology ]
Re:
[ link to this | view in chronology ]
Whitelist words
Blocking real spam is easy, spammers only use a few tricks to get around spam filters, these tricks are so obvious that they give it away as being spam. Of course you need a qualified human at the filtering end to pick up on new trends and key these in. This is where spam filters fail, no one wants to pay someone to maintain the spam filter.
[ link to this | view in chronology ]
I Feel Their Pain
We use a spam filter for our district's e-mail system. The filter rules, white lists, black lists, etc. are downloaded daily from the manufacturer of the filter (very much like antivirus software). We estimate the filter to be about 90% effective removing junk e-mail. We block approximately 60,000 messages per day. It is not the same thing as desktop junk e-mail software that you all are saying takes a few minutes to check each day. The mail is filtered before it reaches the mailbox. There's simply no practical way to check it for "good" e-mail every day. And for those of you that are not aware, most school districts have an IT staff level about 80% smaller than corporations of similar size. So before you all condemn the school district for missing the e-mail, you should realize we aren't talking about desktop filters and staff members that have time to review 60,000 messages per day.
Filtering systems are a fact of life, and like web filters, junk mail filters are not perfect. The more effective they are at blocking junk e-mail, the more likely they are to accidentally block good e-mail. Security comes at a cost of convenience and usability. And this is one of the disadvantages of spam filters.
[ link to this | view in chronology ]
Re: I Feel Their Pain
And yet the schools demand milliions more dollars because they are sooooo overworked.
Pathetic.
[ link to this | view in chronology ]
Re: Re: I Feel Their Pain
[ link to this | view in chronology ]
Re: I Feel Their Pain
Spam filters can also determine if it is most likely spam, or definitly spam. The definites (the majority of spam) do not need to be searched by a user unless they can't find the message they are expecting in their inbox.
Reguardless of the amount of spam or how it is determined to be spam, the recipient should have a way to find it, most conveintly on a web interface. It should not be up to the IT department to trash messages they think is spam.
I worked on a system that had hundreds of thousands of junk email a day, each one archived. This archived came in handy many times when messages were inappropriatly blocked. This became and invaluable tool for use to determine why it was falsely blocked and how to prevent it again.
If your answer is "we have so much spam our only choice is to delete it", then you are using the wrong software, or you don't have the staff qualified to answer the question correctly.
[ link to this | view in chronology ]
Re: I Feel Their Pain
Maybe if you employed an internet filter as well, teachers and administrators wouldn't be surfing the net all day eliciting spam from retailers.
[ link to this | view in chronology ]
[ link to this | view in chronology ]
The public and the law is so highly against SPAM and malicious activity that if we do not implement protection we could be held very accountable if who knows what could happen, especially in an educational institution, yet, when these same tools block a single email that institution is fined for doing what it has to do.
No one can be held accountable for using SPAM filters. Period. End of story. If you are relying so heavily on e-mail then you have absolutely no grasp of reality and definately don't deserve your job.
[ link to this | view in chronology ]