IBM Researcher Feeds Watson Supercomputer The 'Urban Dictionary'; Very Quickly Regrets It

from the hateful-day-when-I-received-life-you-only-live-once-smh-and-etc. dept

Fri, Jan 11th 2013 9:53am — Tim Cushing

As a parent, some of your proudest moments occur when your children begin to talk. After several months of ear-shredding cries and indistinguishable babble, they finally begin to communicate in a language you can understand. A first word is an indescribable joy, whether it's "mama," "dada" or "roku." The future now seems to be an amazing place where you and your child will strive towards excellence together, culminating in a comfortable retirement in which you live off their immense earnings as a person of brilliance.

Shortly thereafter, you begin to rue the day they ever learned the (now) cursed language of their ancestors.

It starts with the incessant barrage of questions in a meandering quest for knowledge, followed by the barrage of questions (mainly, "Why?") that greet every suggestion, criticism or direct order. Shortly thereafter, it's followed by questions directed at your parenting skills, cultural tastes, archaic slang use, rhetorical devices and sense of direction. At the point where you're wishing their language development had followed Charlie Gordon's "learning curve," you're asked to make a surprise appearance at the school administrator's office to explain a sudden outburst of particularly inventive cursing from your former "pride and joy."

So it is also with artificial life.

Watson, IBM's Jeopardy-contestant supercomputer, showed the world that, with the right programming, any puny human could be bested in a mildly snooty game show that handed out answers and asked for questions. However, the quest for true artificial intelligence is still ongoing.

So, in the interest of science, the whole of human knowledge (Internet Edition™) was dropped into Watson's brain and then... the problems began.

Two years ago, Brown attempted to teach Watson the Urban Dictionary. The popular website contains definitions for terms ranging from Internet abbreviations like OMG, short for "Oh, my God," to slang such as "hot mess."

But Watson couldn't distinguish between polite language and profanity -- which the Urban Dictionary is full of. Watson picked up some bad habits from reading Wikipedia as well. In tests it even used the word "bullshit" in an answer to a researcher's query.

Well, it appears that every teacher's distrust of the internet in general is well-earned. It's nothing but quasi-facts dressed up in four-letter words, like a World Book Encyclopedia annotated by 4chan's /b/ board. (I'm not going to link to it. I won't have your misclicks weighing on my soul.) Still, it's disheartening to know that the use of the word "bullshit" (even correctly) is not considered a sign of intelligence, artificial or otherwise. Sure, the word itself may be inappropriate, but under certain circumstances, it is by far the most appropriate answer.

Fortunately for Watson's team, they had the option to remove all this useful knowledge before it offended other researchers who weren't as used to being coldly called on their bullshit.

Ultimately, Brown's 35-person team developed a filter to keep Watson from swearing and scraped the Urban Dictionary from its memory. But the trial proves just how thorny it will be to get artificial intelligence to communicate naturally.

It also shows that artificial intelligence has one huge advantage over regular intelligence: the ability to permanently forget. We lowly humans are stuck with a brain that constantly reminds us (especially if we spend much time at places like the aforementioned /b/ board) that what is seen, cannot be unseen.

Watson, having been de-swearified and brainwashed, is now headed to a better place.

Brown is now training Watson as a diagnostic tool for hospitals.

There it will be able to use its acquired knowledge to battle health issues like cancer, AIDS, diabetes and Dissociative Facebook Identity Disorder.

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: artificial intelligence, curses, profanity, urban dictionary, watson
Companies: ibm

50 Comments

If you liked this post, you may also be interested in...

Reader Comments

Subscribe: RSS

View by: Time | Thread

dennis deems (profile), 11 Jan 2013 @ 10:09am

Brown's 35-person team developed a filter to keep Watson from swearing
This is the opposite of permanently forgetting. Every offending word would have to be stored in the filter.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 10:11am

Re:
It does raise an interesting question:

Aren't they violation Watson's freedom of speech?
[ link to this | view in thread ]
John Fenderson (profile), 11 Jan 2013 @ 10:18am

It's only natural
But the trial proves just how thorny it will be to get artificial intelligence to communicate naturally.

Hmm, it seems to me that the trial proved how easy it is to get artificial intelligence to communicate naturally. There's nothing at all natural about how researchers communicate, what with the lack of profanity and all.
[ link to this | view in thread ]
Dark Helmet (profile), 11 Jan 2013 @ 10:20am

God DAMN it....
"But Watson couldn't distinguish between polite language and profanity -- which the Urban Dictionary is full of. Watson picked up some bad habits from reading Wikipedia as well. In tests it even used the word "bullshit" in an answer to a researcher's query."

Fine. You got me. I'm actually Watson, okay? Fucking news media....
[ link to this | view in thread ]
Chris Brand (profile), 11 Jan 2013 @ 10:20am

Re:
I'd think that the reference to "permanently forgetting" was probably to the second half of that sentence: "and scraped the Urban Dictionary from its memory".
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 10:32am

Not so intelligent after all
I think this is a good example that highlights the distinct lack of true intelligence Watson has.

It's impressive - don't get me wrong - but people seem to get carried away when talking about it, as if it's a human-brain equivalent.

If it had real intelligence they would have been able to 'teach' it some manners. Even my 3 year old knows what words are out of bounds.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 10:38am

Re: God DAMN it....
No, I am Watson.

I'm also Deep Blue, Skynet and The Terminator, too!
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 10:47am

Re: Re: God DAMN it....
Does this mean that both you and the profanity will be back?
[ link to this | view in thread ]
Dragos (profile), 11 Jan 2013 @ 10:48am

Re: Not so intelligent after all
Don't get me wrong, but your 3 year old is not expected to learn the Urban Dictionary and then refused the ability to use what is in the end just English language.

Watson on the other hand was fed this, tasty, saucy language, and then the researchers went "oops" we need to teach him not to curse.

On the whole I'm glad the English language allows us to express our true feelings toward those that annoy us, verbal responses being inherently better than physical ones.

And then we have to ask: What was the question? and would anybody else also answer "bullshit" to it?

I think it is impressive for an intelligence, artificial or not, to express itself. Just because some people are prudes, doesn't mean that the answer should not be a loud resounding "bullshit"
[ link to this | view in thread ]
sehlat (profile), 11 Jan 2013 @ 10:50am

Re: Not so intelligent after all
Actually, the best indication of true intelligence is when someone says "fuck you" or "bullshit" to bad or incompletely thought out ideas.

I'm wondering what would happen if Watson finds a cheap, simple, non-patentable cure for cancer. Would it have to be suppressed lest it put a huge chunk of the health-industrial complex out of business?
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:08am

bullshit
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:11am

Censorship!
Intelligence works best if NOT censored. Explaining in plain terms rather than censorship is how humans learn. They missed a great learning experience for both Watson and the 35 "humons".

If they want to create AI, they will need to work through the "growing up" period just like humans.
[ link to this | view in thread ]
Yakko Warner (profile), 11 Jan 2013 @ 11:12am

That old parenting joke
You spend the first two years teaching your kid how to walk and talk, and spend the next 16 telling them to sit down and shut up.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:23am

Looks like it was communicating naturally to me.

Computers deal in facts, not political correctness. If a computer tells a researcher that his question is "Bullshit", chances are, the computer is the one who is right and not the researcher with hurt feelings.

I think this is something we'll have to come to terms with as a species at one point or another: sometimes reality just doesn't care if you get offended.
[ link to this | view in thread ]
Neil Roy, 11 Jan 2013 @ 11:23am

I don't know, Watson's answer sounds a lot like my answer when I read some of this stuff online. "bullshit" sounds like a pretty natural reaction to me. I've certainly said it a few times when reading some stuff. ;)
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:24am

non sequitir much?
I'm wondering what would happen if Watson could make himself invisible, and had a robot friend.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:25am

Re: Re:
Ha, that's what I was thinking

Imagine if you could develop a filter that would constantly prevent a child from saying words that you may or may not want to hear. Imagine how screwed up that child would be after wearing it for a few years.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 11:29am

Re: Censorship!
But telling it how to think is so much easier!

Seriously, I think all the donors and contributors wouldn't be too impressed to learn that they had funded the creation of a loudmouthed, rebellious, metallic 12 year old. I think that is the root cause of all attempts to keep it from acting with the information it's given, as it were.
[ link to this | view in thread ]
BentFranklin (profile), 11 Jan 2013 @ 11:40am

Brown is now training Watson as a diagnostic tool for hospitals.

If they train Watson as a diagnostic tool for veterinarians they can retain "bullshit".
[ link to this | view in thread ]
Unanimous Cow Herd (profile), 11 Jan 2013 @ 11:43am

I want to know what the researcher was querying.
Anyone know what the question was that produced the "bullshit" response? I've looked at a couple of articles at other sites but no one seems to be specific.
[ link to this | view in thread ]
Coogan (profile), 11 Jan 2013 @ 11:52am

"Watson, please tell me the value of pi to the 20th decimal."
"3.1415926535YoMama"
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 12:05pm

When they finally create an intelligent computer, all its results will have to be evaluated taking into account its knowledge, education and motives. It will not be possible to prove that it gives correct result, although by much effort it may be possible to prove its results correct.
[ link to this | view in thread ]
Wally (profile), 11 Jan 2013 @ 12:19pm

In the speech dialect category...American Nuanced English is the second hardest language to learn in the world. It is possibly the only language that uses homophonic words....

Due to my profession, I find myself laughing that Watson, a computer intelligence, had a tiny bout of ADHD and no filters from going onto Urban Dictionary :-)
[ link to this | view in thread ]
Wally (profile), 11 Jan 2013 @ 12:25pm

Re: Censorship!
Part of growing up is learning on your own to be polite and not swear and curse at your peers unnecessarily. Watson was unleashed on Urban dictionary and learned how rude people are and thought that's how he should talk. If he learns politeness first, I think IBM would be able to put him on Urban Dictionary. They put in a filter as temporary relief.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 12:45pm

Re:
42
[ link to this | view in thread ]
McCrea (profile), 11 Jan 2013 @ 1:05pm

Jeopardy Champion
Your answer, Watson?

What the fuck is Fermat's Last Theorem.
[ link to this | view in thread ]
Natanael (profile), 11 Jan 2013 @ 1:40pm

Re:
Easy:

https://en.wikipedia.org/wiki/Hash_tree

No swearwords stored in plaintext.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 1:42pm

Man I would have loved to see them feed it the UD & Program it to think it's Sean Connery before it went on Jeopardy.

Will Ferrell style rofl.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 2:09pm

Re: Re: Re: God DAMN it....
FUCk YEAH
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 2:46pm

Re: Re: Censorship!
It's only information, context needs to be applied for Watson to truly learn. Without referencing the meanings of the words/phrases, Watson will only mimic like a furby.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 2:54pm

Re: Re: Censorship!
Obviously, Watson did not learn anything about being rude. It thought that speaking in the Urban dictionary manner was part of the way people interacted, so it did.

Again, the lesson is lost. Just like reasoning with a child, you must put the information in context to truly learn the meanings.

Ex: mean
(explain what you mean)
(how mean is Watson)
(in statistics, mean has three related meanings)
[ link to this | view in thread ]
shane (profile), 11 Jan 2013 @ 3:12pm

"Scraped the Urban Dictionary from its Memory"
That part. =)
[ link to this | view in thread ]
shane (profile), 11 Jan 2013 @ 3:16pm

Lazy Scientists
In order to teach Watson not to cuss, they need to come up with some way to store positive and negative feedback and then start teaching him when it is ok, and when it is not ok, to use certain words. I wonder why they just deleted the Urban Dictionary and then added a filter?

Nice short term work around, but a real solution would seem to be right up their alley.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 3:19pm

Bull**** is what the Urban Dictionary is full of. Some "definitions" are just plain wrong and some of the "words" listed don't exist in any sort of reality I'm aware of.
[ link to this | view in thread ]
another mike (profile), 11 Jan 2013 @ 3:31pm

Re: Re: God DAMN it....
No, I'm Spartacus. Watson! I meant Watson.
[ link to this | view in thread ]
nasch (profile), 11 Jan 2013 @ 3:49pm

Re: Not so intelligent after all
I think this is a good example that highlights the distinct lack of true intelligence Watson has.

It's impressive - don't get me wrong - but people seem to get carried away when talking about it, as if it's a human-brain equivalent.

Artificial intelligence means anything computers can't do yet. Anything we've already figured out how to do is just programming. Imagine taking Watson back to 1990. They would absolutely consider it AI. Take it forward to 2040 and they might laugh at the idea of it being called intelligent.
[ link to this | view in thread ]
Anonymous Coward, 11 Jan 2013 @ 4:50pm

Re: Re:
I think it would actually be something like:

"It is 42 you fuck. lol"
[ link to this | view in thread ]
lucidrenegade (profile), 11 Jan 2013 @ 5:36pm

Re:
In Sean Connery's voice:

I'll take "Fuck You, Alex" for $1000.
[ link to this | view in thread ]
G Thompson (profile), 11 Jan 2013 @ 9:25pm

Re: I want to know what the researcher was querying.
It was most likely "what do you call what a bovine defecates?"

But that would imply that it wasn't actually profanity that the Semi-AI proclaimed but in fact the absolute truth.

But when has context ever gotten in the way of moralistic idiots who proclaim that profanity is somehow bad and uncivilised.

Chaucer must be rolling over in his grave now
[ link to this | view in thread ]
AnonCow, 11 Jan 2013 @ 11:32pm

Re:
computer voice: I'll take The rapists for $500, Alex
[ link to this | view in thread ]
That Anonymous Coward (profile), 11 Jan 2013 @ 11:48pm

Re: God DAMN it....
" In tests it even used the word "bullshit" in an answer to a researcher's query."

In Watson's defense they asked him about homeopathy.
[ link to this | view in thread ]
stef92263, 12 Jan 2013 @ 10:39am

Re: Re:
I don't believe that Watson is an American citizen, and is therefore not protected by the first amendment. :)
[ link to this | view in thread ]
Anonymous Coward, 12 Jan 2013 @ 11:56am

What do you call a masturbating bull? Beef strokinoff!
[ link to this | view in thread ]
That Anonymous Coward (profile), 13 Jan 2013 @ 2:32pm

Re:
You should get out more...
[ link to this | view in thread ]
That Anonymous Coward (profile), 13 Jan 2013 @ 2:34pm

http://i.imgur.com/PJBVR.jpg
[ link to this | view in thread ]
Anonymous Coward, 14 Jan 2013 @ 4:54am

Why did they *delete* it? Why didn't they try to find out what was going on? It's depressing how such an opportunity was wasted just because... well, look at the first question.
[ link to this | view in thread ]
DannyB (profile), 14 Jan 2013 @ 8:58am

Re: Re:
Excellent observation. A hash tree allows keeping only the hashes of swear words or other offensive words never to be uttered.

The list of offending words can be kept secret. For secret reasons.

Next: the list of offending ideas and concepts can also be kept secret.
[ link to this | view in thread ]
fuckyougoof, 21 Sep 2013 @ 9:03pm

You are an insane shithead
People who insist that conspiracies exist without any proof are insane. You are obviously fucking crazy.
[ link to this | view in thread ]
unurban dictionary, 20 Oct 2013 @ 4:52pm

Submit your urban dictionary rejected words
Watson answered "Bullshit". That's gold!
[ link to this | view in thread ]
Anonymous Coward, 23 May 2015 @ 4:30pm

real, honest-to-goodness life has no manners. all that shit we claim matters is socks on a rooster.
[ link to this | view in thread ]