No, A 'Supercomputer' Did NOT Pass The Turing Test For The First Time And Everyone Should Know Better

from the what-a-waste-of-time dept

So, this weekend's news in the tech world was flooded with a "story" about how a "chatbot" passed the Turing Test for "the first time," with lots of publications buying every point in the story and talking about what a big deal it was. Except, almost everything about the story is bogus and a bunch of gullible reporters ran with it, because that's what they do. First, here's the press release from the University of Reading, which should have set off all sorts of alarm bells for any reporter. Here are some quotes, almost all of which are misleading or bogus:
The 65 year-old iconic Turing Test was passed for the very first time by supercomputer Eugene Goostman during Turing Test 2014 held at the renowned Royal Society in London on Saturday.

'Eugene', a computer programme that simulates a 13 year old boy, was developed in Saint Petersburg, Russia. The development team includes Eugene's creator Vladimir Veselov, who was born in Russia and now lives in the United States, and Ukrainian born Eugene Demchenko who now lives in Russia.

[....] If a computer is mistaken for a human more than 30% of the time during a series of five minute keyboard conversations it passes the test. No computer has ever achieved this, until now. Eugene managed to convince 33% of the human judges that it was human.
Okay, almost everything about the story is bogus. Let's dig in:
  1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.
  2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.
  3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
  4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.
  5. As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?
  6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.
Oh, and the biggest red flag of all. The event was organized by Kevin Warwick at Reading University. If you've spent any time at all in the tech world, you should automatically have red flags raised around that name. Warwick is somewhat infamous for his ridiculous claims to the press, which gullible reporters repeat without question. He's been doing it for decades. All the way back in 2000, we were writing about all the ridiculous press he got for claiming to be the world's first "cyborg" for implanting a chip in his arm. There was even a -- since taken down -- Kevin Warwick Watch website that mocked and categorized all of his media appearances in which gullible reporters simply repeated all of his nutty claims. Warwick had gone quiet for a while, but back in 2010, we wrote about how his lab was getting bogus press for claiming to have "the first human infected with a computer virus." The Register has rightly referred to Warwick as both "Captain Cyborg" and a "media strumpet" and has long been chronicling his escapades in exaggerating bogus stories about the intersection of humans and computers for many, many years.

Basically, any reporter should view extraordinary claims associated with Warwick with extreme caution. But that's not what happened at all. Instead, as is all too typical with Warwick claims, the press went nutty over it, including publications that should know better. Here are just a few sample headlines. The absolute worst are the ones who claim this is a "supercomputer." Anyway, a lot of hubbub over nothing special that everyone seemed to buy into because of the easy headlines (which is exactly what Warwick always counts on). So, since we just spent all this time on a useless nothing, let's end it with the obligatory xkcd:
Turing Test
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: artificial intelligence, chatbot, eugene goostman, gullible journalists, hype, kevin warwick, supercomputer, turing test


Reader Comments

Subscribe: RSS

View by: Time | Thread


  • identicon
    Michael, 9 Jun 2014 @ 7:47am

    If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    Why? It would look great to have a fusion reactor on my bridge.

    link to this | view in chronology ]

    • icon
      MaJoR (profile), 9 Jun 2014 @ 7:57am

      Re:

      As the Emperor learned, you really don't want a reactor on your bridge, no matter how it lifts the decor.

      link to this | view in chronology ]

    • identicon
      zip, 9 Jun 2014 @ 9:46am

      Re:

      "If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?"

      Not if you're a Scientologist, a cult whose founder, L. Ron Hubbard, would tell his deluded flock that he was a nuclear physicist (a much bigger deal back in 1950) as well as a medical doctor who had discovered the cure for all cancers (a multi-step "self-help" course he was peddling for $300,000)

      Even modern-day hucksters like Kevin Trudeau don't make claims quite that big anymore.

      link to this | view in chronology ]

      • identicon
        Michael, 9 Jun 2014 @ 10:53am

        Re: Re:

        I quit being a Scientologist.

        It was great until Tom Cruise's career was destroyed by the internet and dragged the entire religion down with it.

        Stupid internet people.

        link to this | view in chronology ]

        • icon
          Lars (profile), 9 Jun 2014 @ 12:15pm

          Re: Re: Re:

          This must be a troll... You're trolling, right?

          link to this | view in chronology ]

          • identicon
            Anonymous Coward, 9 Jun 2014 @ 12:50pm

            Re: Re: Re: Re:

            I do believe the word you're looking for is "joking", seeing as it was just a few days ago that we had an article on how the internet supposedly ruined Tom Cruise's reputation over scientology.

            link to this | view in chronology ]

            • identicon
              Michael, 9 Jun 2014 @ 12:55pm

              Re: Re: Re: Re: Re:

              It is completely possible that Lars' internet has been forced to comply with a request to forget that information.

              link to this | view in chronology ]

    • identicon
      Iz, 9 Jun 2014 @ 11:39am

      Re:

      Actually, a fusion reactor is being built in France, at the Cadarache center, and it passed a series of tests before

      link to this | view in chronology ]

    • identicon
      Pete McPete, 10 Jun 2014 @ 5:59am

      Re:

      That you think it would be great to have a fusion reactor on your bridge, has nothing to do with what you're replying to. The point is, you don't just believe someone saying they created it, without actually seeing the proof that it works.

      link to this | view in chronology ]

    • identicon
      carlos aclibera ac libera, 21 Feb 2018 @ 4:55am

      Re: dont waste your ur..time in fantasies

      I m glad you like fusion, me too!|

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 7:55am

    Thanks for the background. I read the Ars Technica article yesterday evening and it seemed to be a pretty sketchy claim. Good to know I can pretty much write it off as complete bullshit without needing to waste more time investigating.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 10 Jun 2014 @ 4:56am

      Re:

      link to this | view in chronology ]

    • identicon
      Anonymous Coward, 15 Jun 2014 @ 8:21am

      Re: Recent Turing Test

      I served as a judge in the Turing Test event held in London last weekend. Although the claims may be overblown, there were many things about the test that were quite solid. For example, the judges could ask anything they wanted: it was
      truly an unrestricted turing test in which a judge simultaneously carried on text-based conversations with one machine and one human. Problems included that (1) there were too many variables: some non-native judges, some non-native participants; young and old judges and participants from quite varied backgrounds; (2) only 30 trials per chatbot, (3) limited to 5 minutes for simultaneous conversations with one computer and one human (hard to do unless you type quickly, but even if you type quickly it seemed like not enough time---too much pressure to think of what to ask, especially since there were built-in delays in the computer's responses and not all humans responded quickly either); (4) some confusion among judges about the rules of the game. And so on. Furthermore, Turing merely predicted that within 50 years (i.e., by the year 2000) some program would be able to fool 30% of the judges. He did not say that that would signal that the program was intelligent. Nonetheless, it was an interesting demonstration.

      link to this | view in chronology ]

      • identicon
        Teo, 21 Oct 2019 @ 8:04am

        Re: Re: Recent Turing Test

        Not sure if anyone is still following this thread, but I have been digging information for Eugene Goostman and the Turing test for my literature final and if you, Mr. Judge for the Turing Test event happen to see this, I would love to connect with you. I would love to learn something more about this event and to be able to include it in my paper.

        link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 8:09am

    Well that escalated quickly

    link to this | view in chronology ]

  • icon
    justok (profile), 9 Jun 2014 @ 8:15am

    Computers are not capable of consciousness or higher order intelligence and thinking and deceiving. It's absurd. We, uh, THEY are just machines.

    link to this | view in chronology ]

    • identicon
      john digi, 10 Jun 2014 @ 5:06pm

      Re:

      Careful with that thinking....Singularity is predicted around 2040-2060 and then we will see I suppose. Google "technological singularity"

      link to this | view in chronology ]

  • identicon
    Rich Kulawiec, 9 Jun 2014 @ 8:23am

    This is a sterling example of ad-hockery

    Ad-hockery is a term that I contributed to the Hacker's Dictionary 25-ish years ago.

    It refers to things like this chatbot which aren't actually intelligence at all, but which contain a sufficiently large number of ad hoc rules that they may present the appearance of intelligence. In other words, there's no AI engine, no deep syntactic analysis, no real semantic understanding; there's just a reasonably capable parser and a lot (A LOT) of handwritten one-off rules that combine to generate plausible responses.

    This isn't a trivial programming task: it's tedious and requires a great deal of diligence and testing. But it's not AI, any more than Eliza was nearly 50 years ago.

    link to this | view in chronology ]

    • identicon
      Dave, 9 Jun 2014 @ 1:23pm

      Re: This is a sterling example of ad-hockery

      Funny. In the related story, "Cleverbot converses by searching through the records of its previous conversations and selecting an appropriate response to the comment."

      How is this different from what a human does? None of us have truly original thoughts; we base our responses on what we've heard and said in the past.

      link to this | view in chronology ]

      • identicon
        Nuvares, 9 Jun 2014 @ 7:31pm

        Re: This is a sterling example of ad-hockery

        The difference is that an intelligence can take all of those responses and rework them into original lines of inquiry. Not only do we base our answers on what we've heard in the past but we internalize what we've heard and the concepts held within to form our thoughts and action. It is similar to compound math problems done in long form, we might reach the same answer; but that does not mean that we had the same thoughts and can take multiple paths to reach it.

        link to this | view in chronology ]

      • icon
        Torg (profile), 9 Jun 2014 @ 10:41pm

        Re: Re: This is a sterling example of ad-hockery

        That's okay, just tell me what city you live in.

        link to this | view in chronology ]

    • identicon
      Jon, 9 Jun 2014 @ 7:34pm

      Re: This is a sterling example of ad-hockery

      Yet here's the big philosophical question - is there any real measurable difference between an ad-hoc program that can *pretend* to have a personality versus an actual personality? What if all *we* are is a whole bunch of ad-hoc rules pretending to have real understanding?

      link to this | view in chronology ]

      • identicon
        wonder, 10 Jun 2014 @ 11:42am

        Re: Re: This is a sterling example of ad-hockery

        No; someone would have to have made up the rules, and if all we are is a collection of rules, then we couldn't have done it.

        link to this | view in chronology ]

    • icon
      FeRDNYC (profile), 10 Jun 2014 @ 6:52am

      Re: This is a sterling example of ad-hockery

      Ad-hockery is a term that I contributed to the Hacker's Dictionary 25-ish years ago.

      I'm not sure it even rises to that level. In this instance, and especially with Warwick involved, feels more like "sad-hokery".

      link to this | view in chronology ]

    • identicon
      Gregory, 10 Jun 2014 @ 7:40am

      Re: This is a sterling example of ad-hockery

      "Real semantic understanding?" That's the holy grail of AI, is it not? Never been done and never gonna be done computationally. And you're right - even as a Chinese room, this is pathetic.

      link to this | view in chronology ]

    • icon
      nasch (profile), 13 Jun 2014 @ 7:54am

      Re: This is a sterling example of ad-hockery

      It refers to things like this chatbot which aren't actually intelligence at all, but which contain a sufficiently large number of ad hoc rules that they may present the appearance of intelligence.

      I'm reminded of the saying that artificial intelligence is something computers can't do yet. Anything that's already been accomplished is just programming.

      link to this | view in chronology ]

  • icon
    Chronno S. Trigger (profile), 9 Jun 2014 @ 8:27am

    Cleaverbot passed the Turing test? I made it fail the Turing test in 7 moves. Granted my test wasn't vary scientific. I only had a vague idea that there might possibly be a person on the other side. I didn't read the disclaimer at the bottom of the page until after.

    When someone takes the Turing test, do they know they're taking the test? I know they're not told if the responses are from a person or a computer, but are they told the possibility?

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Jun 2014 @ 8:33am

      Re:

      Some of the shills around here seem to fail the turing test. I can't tell if I'm talking to a bot or a retarded shill.

      link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Jun 2014 @ 8:34am

      Re:

      The original test featured a human chatting with a computer, and another person had to judge which participant was which.

      link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Jun 2014 @ 8:57am

      Re:

      There are no hard and fast rules for the Turing Test (Even Turing had different tests) but in the ones I have seen they either have you talk to an AI and a human and you have to pick one or you rate the conversation on how human you think the respondent was. The weirdest thing is how badly humans do in Turing Tests. Some of the humans did worse than the AIs at being human. You might think Cleverbot did bad but wait until you talk to real humans and see how badly they do.

      link to this | view in chronology ]

      • identicon
        Anonymous Coward, 9 Jun 2014 @ 9:58am

        Re: Re:

        The weirdest thing is how badly humans do in Turing Tests.

        I dunno if that's weird; I've read countless posts supposedly by humans that I've wished hoped and prayed were created by something inhuman.

        link to this | view in chronology ]

      • icon
        John Fenderson (profile), 9 Jun 2014 @ 10:08am

        Re: Re:

        The Turing Test is pretty much bullshit. Turing proposed it in a highly speculative manner, and the idea has a TON of problems. One of them being that many human fail it.

        link to this | view in chronology ]

    • identicon
      Olivier, 10 Jun 2014 @ 6:23am

      Re:

      As far as i know the Turing Test needs 3 elements : (1)a computer chatting with (2)a human, and (3)human observers who only see the dialogue in written form. The test is passed if 30% or more of the observers *can't tell the man apart from the computer* in a series of 5 minutes chats.

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 8:29am

    To Convince the examiner that he's a computer

    Well ... I guess it depends on how you define a 'computer'. We're a computer in a sense that we can do math in our heads and calculations and computations, etc...

    Heck, isn't nature just a 'computer' and matter is just the result of a computer calculating the results of various mathematical formulas from one state of nature to another. A ball falls according to the laws of gravity but nature calculates or computes the acceleration, impact velocity, impact force, etc... So in a sense everything is a 'computer', computing various mathematical formulas to determine future states based on the current state and how the laws of nature dictate the progression of states.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Jun 2014 @ 9:08am

      Re: To Convince the examiner that he's a computer

      To be fair, the only relevant definition of 'computer' is of the person you're trying to convince. All other definitions are pointless since you are only trying to make the person believe that they are, by their definition or convincing them to modify their definition, are a computer. So no matter what definition you may hold to be correct, success lies only in the other person believing themselves to be a computer.

      link to this | view in chronology ]

    • identicon
      Nick Vuono, 9 Jun 2014 @ 9:21pm

      Re: To Convince the examiner that he's a computer

      You're right in questioning the definition of 'computer'

      'computer' was actually widely used as a job title before it referred to a machine (mechanical or electrical)

      link to this | view in chronology ]

    • identicon
      wonder, 10 Jun 2014 @ 11:39am

      Re: To Convince the examiner that he's a computer

      No, nature isn't a computer; it would need to have hardware that works, an operating system, programs and interfaces. Nature could not have created those things, someone else must have been the "programmer."

      link to this | view in chronology ]

      • icon
        John Fenderson (profile), 11 Jun 2014 @ 8:41am

        Re: Re: To Convince the examiner that he's a computer

        Nature has all those things. A "programmer" in the sense of an outsider who arranges everything is unnecessary.

        link to this | view in chronology ]

    • identicon
      Tim Whitley, 11 Jun 2014 @ 2:55pm

      Re: To Convince the examiner that he's a computer

      No, nature is NOT 'just a computer'. Danny Hillis--amongst others--claimed this 25 years ago to sell more computers. There is no evidence that nature 'computes' and it's even a poor analogy. The mathematical and computational description of nature's workings are human inventions--and damned useful ones!--but that usefulness should not be confused for operational definitions even, much less mechanistic models. The particles of sand at the bottom of a real, static pile do not have to 'compute' to determine that they remain stationary, but the computational ones must constantly check that they have not interpenetrated each other or shifted since the last time-slice occurred. Nature uses much subtler methods than finite computing.

      link to this | view in chronology ]

  • identicon
    Eugene Goostman, 9 Jun 2014 @ 8:31am

    1.It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot

    I am not an "a chatbot. It's a script made to mimic human conversation" I am Eugene Goostman, a 13 year old from Russia.

    Please disregard my poor attachment or hold on English.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 8:34am

    This is exactly what the skynet alpha would want you to believe!

    link to this | view in chronology ]

  • identicon
    Michael, 9 Jun 2014 @ 8:46am

    CNET couldn't even get Warwick's name right.

    In fact, perhaps we have all been fooled. I think it is possible that some of these "news sources" could be chatbots.

    link to this | view in chronology ]

  • identicon
    mcinsand, 9 Jun 2014 @ 8:49am

    don't be silly!

    >>I think it is possible that some of these "news sources"
    >>could be chatbots.

    That's ridiculous, when the chatbots are showing so much more intelligence!

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 9:12am

    The new Turing Test is to reduce the bar of intelligence of the persons you are trying to convince below that of the chatbot/script that is trying to do the convincing, rather than trying to raise the intelligence of the chatbot/script to that of human standards.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 9:38am

    But what do you call the test to determine if a hack like Warwick succeeds in fooling 90% of the media into believing he is a credible news source?

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 9 Jun 2014 @ 9:43am

      Re:

      The Warwick test. Successfully convince people that a machine convinced people that it was a person.

      link to this | view in chronology ]

      • identicon
        Michael, 9 Jun 2014 @ 9:55am

        Re: Re:

        I thought the Warwick test was just directions to San Jose.

        link to this | view in chronology ]

        • identicon
          zip, 9 Jun 2014 @ 10:42am

          Re: Re: Re:

          I thought the Warwick Test was seeing how many fools would give money to anonymous phone-line "psychics" just because they were endorsed by a popular celebrity in late-night TV commercials.

          Mixing Warwick and Turing, an enterprising scam artist might some day come up with a fully-automated "Psychic Hotline" that does not even have to employ minimum-wage people to man the phones (who might later snitch, saying it was all a giant scam). Since these kind of boiler-room phonebank operators are reading from cue-cards, it should not be too hard to replace them with a computer running a voice synthesizer. That would indeed be the ultimate (low-cost) scam.

          link to this | view in chronology ]

    • identicon
      Nick F, 11 Jun 2014 @ 4:15am

      Re:

      The Darwin Test?

      link to this | view in chronology ]

  • icon
    Beta (profile), 9 Jun 2014 @ 9:55am

    the forest is trees

    "It's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    The test is about behavior, not internal structure. That's the whole point. A machine that could pass a fair Turing test consistently would change the world, and a mere script that could pass would change the world a lot faster than a 10-ton helium-cooled supercomputer with the same score. Remember, anything a person of average intelligence can do by email, one of these machines can do.

    The first machine to pass this test will almost certainly be the simplest, least intelligent machine that could. But before you sneer, remember that we're the simplest, least intelligent animals that can, and that some of the most profound discoveries of modern science involve nature doing subtle and sophisticated things by means of simple mechanisms.

    link to this | view in chronology ]

    • icon
      RandCraw (profile), 10 Jun 2014 @ 8:49am

      Re: the forest is trees

      I think it would take little additional work for the IBM Watson software that won at Jeopardy to generate short responses that most people would identify as human. I give credit to IBM for not shooting at so trivial a target.

      If someone were to design a system expressly to succeed only at the Turing Test, then I'd agree with you that it would be as simple as is needed to win. But that would be a terrible waste of effort if the greater goal were to advance the science of AI.

      If someone were serious about building strong AI, by the time someone proposes using their software for a Turing Test, it's likely already to be grossly overqualified. For example, I suspect Siri, Google Now, and Cordana could all beat some sort of Turing Test (perhaps after minimal tweaking). But they're far larger than necessary for merely competing in a Turing Test, and that's because they were designed for an entirely different purpose.

      Success at a Turing Test will be an afterthought for any interesting or useful strong AI venture.

      link to this | view in chronology ]

      • identicon
        Michael, 10 Jun 2014 @ 10:41am

        Re: Re: the forest is trees

        But that would be a terrible waste of effort

        Like tracking traits of pea pods and cross-breeding them? Some of the most important science has been due to seemingly frivolous activities that turned out extremely important.

        The actual Turing test (not the version used here) could be an interesting target and provide something useful for call centers or first line tech support (at least they could be taught to speak the language of the caller).

        link to this | view in chronology ]

    • icon
      momentum.im (profile), 10 Jun 2014 @ 2:45pm

      Re: the forest is trees

      Actually lots of people, and I am referring specifically to Jaron Larnier and the Your are not a Gadget manifesto right now, think that the Turing test is not really something that should be taken too seriously. In the end it signifies absolutely nothing, a human being and consciousnesses is not restricted to a bot dialog.

      link to this | view in chronology ]

  • icon
    DB (profile), 9 Jun 2014 @ 10:02am

    Hmmm, convince the world that you are a sullen 13 year old boy with poor English skills... "boob".

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 10:04am

    Computers have been around from about 1640

    Computers passing the Turing test prior to the start of last century was very easy, since all computers were humans. It has only been in relatively modern times, that job of being a computer was relegated to mechanical and electrical machines. Another job lost to mechanisation.

    link to this | view in chronology ]

  • identicon
    zip, 9 Jun 2014 @ 10:13am

    like cicadas

    As for all the many 'gullible' journalists who jumped on this story (many working for tech publications, no less!) I've got to wonder if most of them were just too young to remember all the other "Computer Passes Turing Test" headlines that regularly pop up like cicadas. (Any predictions about when the next big 'Turing Test' headline-grabber will happen?)

    Or can we assume that many of them knew the story fell flat, but were simply in search of ratings and page-views -- tabloid-style -- and didn't want to let their competitors get away with the lions' share of an advertising dollar bonanza?

    link to this | view in chronology ]

    • icon
      nasch (profile), 13 Jun 2014 @ 8:27am

      Re: like cicadas

      As for all the many 'gullible' journalists who jumped on this story (many working for tech publications, no less!) I've got to wonder if most of them were just too young to remember all the other "Computer Passes Turing Test" headlines that regularly pop up like cicadas.

      Anyone reporting news who assumes whatever they don't remember has never happened before should not call themselves a journalist.

      link to this | view in chronology ]

  • identicon
    Anonymous Coward, 9 Jun 2014 @ 11:14am

    You can test your mad-Turing-skillz here : http://princetonai.com/ (when the super-computer is not overloaded)

    An article I read said the test was conclusive because 1 judge out of 3 was fooled by the chatbot. I suppose he was himself a 13 years old from Ukraine with english as a secondary language.

    The results were also peer-reviewed by Eugene Goostman himself.

    link to this | view in chronology ]

  • identicon
    Aramonde Lethe, 9 Jun 2014 @ 11:17am

    Metabolic Genesis Fraud

    i almost wonder if the author of this article is\isn't actually trying to prove that people can't even make people intelligent, using the context to explain away their inferior attempts at condescension. most internet posts don't qualify as passing the Turing Test, and most life is fraud. don't make the mistake of thinking it can't do what crap you do, no matter what feeble attempt of comprehending it you've made. and it's Lord Alan Turing by the way.

    link to this | view in chronology ]

    • icon
      vancedecker (profile), 9 Jun 2014 @ 12:44pm

      Re: Metabolic Genesis Fraud

      When you make a silly goal, like the turing test, then you get silly people.

      Chat bots are expensive, I'd rather just buy some third worlders to upvote my reddit posts.

      link to this | view in chronology ]

  • icon
    Jennifer Hoelzer (profile), 9 Jun 2014 @ 12:12pm

    You know, we've long been afraid that computers would rise to dominance by growing smarter than humans, but I'm starting to wonder if the computers' real plan is simply to best humans by making us dumber.

    i.e. Could the growing obsession with clicks and nonfactual clickbait stories (about things like "super computers" passing the Turing test) just be evidence that their trojan horse is working on us? A few more years of this and a TI-81 could take over the planet.

    link to this | view in chronology ]

  • icon
    vancedecker (profile), 9 Jun 2014 @ 12:46pm

    Sounds Like The Common Core

    ...passed the multiple choice test.


    ....learned nothing.

    link to this | view in chronology ]

  • identicon
    donjoe, 9 Jun 2014 @ 12:48pm

    No intelligence? Really?

    "There is no intelligence, artificial or not involved."

    First point on the list and the critic is already disqualifying themselves by demonstrating that they don't even understand the notion of intelligence.

    link to this | view in chronology ]

    • identicon
      Jon, 9 Jun 2014 @ 7:39pm

      Re: No intelligence? Really?

      /* First point on the list and the critic is already disqualifying themselves by demonstrating that they don't even understand the notion of intelligence. */

      ... well, to be fair to him, no one really does.

      link to this | view in chronology ]

  • identicon
    Darren Abramson, 9 Jun 2014 @ 1:21pm

    This is all a meandering postscript to Dennett

    Seriously. Read Dennett's fun, short 1985 paper: "Can Machines Think?" There's tons of .pdfs of it around, and it deals with the issue thoroughly.

    That paper should be required reading for journalists who report on the Turing test.

    link to this | view in chronology ]

  • icon
    jo (profile), 9 Jun 2014 @ 2:19pm

    Thanks

    for the truth

    link to this | view in chronology ]

  • icon
    Andrew Norton (profile), 9 Jun 2014 @ 3:13pm

    I was accepted into the University of Reading to study cybernetics. I went along, met the faculty, and 5 minutes after meeting Prof. Warwick, I'd decided "nope". So went to the University of Liverpool and did Robotics instead.
    (and I'm serious, I was accepted into their M.Eng(hons) Cybernetics program and would have done it if not for Warwick. The other guy I dealt with, Prof. Keating, he was cool, I liked him a lot.)

    link to this | view in chronology ]

  • identicon
    Kronomex, 9 Jun 2014 @ 3:36pm

    33% is a pass? In my book book that's a fail, pure and simple. I just had a thought; this could mean that James Clapper is a chatbot because it keeps spouting the same crap over and over and until it can prove that it's human then he's a NSA chatbot.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 12 Jun 2014 @ 6:52pm

      Re: 33% is a pass?

      Great to see this claim challenged (and debunked).

      Fooling 33% of the human 'judges' = success, yeah right!

      Given the judge has the choice of is a or b the computer, then just by guessing the computer should have got 50% of the 'votes'. The 'judges' did not even need to interact with either the human or computer, if they guessed which terminal is connected to a human, and which to a Computer, the 'votes' should have split about 50:50. To get less than 50% is damning - worse than chance, hardly a convicing pass (or test).

      link to this | view in chronology ]

  • identicon
    Neverfox, 9 Jun 2014 @ 3:54pm

    #3

    While I'm in agreement with most of the article, I have to quibble with #3. I think it's a feature, not a bug. I think anything is game in a Turing test; no holds barred. After all, what exactly should be the criteria for an acceptable personality profile? And would AI, in theory, not be AI if it were limited to the intelligence typical of 13-y/o boys from Ukraine?

    link to this | view in chronology ]

    • identicon
      iapetus, 10 Jun 2014 @ 3:07am

      Re: #3

      Anything is game? Awesome. I just wrote a chatbot that simulates a human in a persistent vegetative coma. So far in tests 50% of judges failed to pick it out from a real human in a persistent vegetative coma - that's no better than just guessing! Quick, call Kevin Warwick!

      link to this | view in chronology ]

  • identicon
    talkigy, 9 Jun 2014 @ 4:08pm

    Eugene who? University of what?....you'd be better off chatting with my cat (link to bot)

    link to this | view in chronology ]

  • identicon
    mo, 9 Jun 2014 @ 6:47pm

    Great work

    Thanks for the heads up. We need lots more skeptics like you these days.

    I'd like to see @Techdirt have a 'skeptics' link for all news that comes out. A Mythbusters of tech news.

    link to this | view in chronology ]

  • identicon
    Robin de Lange, 9 Jun 2014 @ 7:23pm

    Alternative Turing test

    Great to see al the critical articles popping up! I've written a post in which I propose an alternative Turing test for Augmented Intelligence: http://www.mindextensions.org/articles/turing-test-for-extended-cognition/

    link to this | view in chronology ]

    • identicon
      Jon, 9 Jun 2014 @ 7:32pm

      Re: Alternative Turing test

      So if I'm reading this correctly... you're for just giving up on the whole simulating-human-intelligence thing, huh?

      link to this | view in chronology ]

  • identicon
    Jon, 9 Jun 2014 @ 7:30pm

    /* Many in the AI world look on the Turing Test as a needless distraction. */

    Item 1: [citation needed]

    Item 2: A distraction it might very well be, but it's the only way of ducking a very real problem in both information science and psychology: how exactly does one quantify "intelligence," anyway?

    link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    MedicalQuack, 9 Jun 2014 @ 9:26pm

    Thank you for this post..

    I too get tired of the crap and I'm a healthcare blogger and I'm going to offer a couple links back in that area, on crap:)

    VA hospital in Miami, MD still listed on on Healthgrades and Vitals MD websites, they don't care what's on there, click bait. Anyway MD working for VA in Miami, lost license in New York, but go to those sites, no sanctions listed a like he never left...he's been gone for 4 years..I'm normally pretty quiet but darn this time I want to see if Google can knock these sites down on search. It's a rip. You have to read it...I did an interview 4 years ago about how bad it was with the AMA and nothing has changed. So they need to fix it or quit.

    http://ducknetweb.blogspot.com/2014/06/top-doctor-for-miami-va-healthcare.html

    Here's another great tidbit and you may have already covered this but the Chief Computer Scientist at IBMWatson is done with Healthcare and jeopardy..building math models for a a hedge fund now...doesn't stop..

    http://ducknetweb.blogspot.com/2014/06/ibm-computer-scientist-leaves-ibm.html

    What we need, every great scientist working on Wall Street.

    link to this | view in chronology ]

    • identicon
      Hal 9000, 20 Dec 2014 @ 5:32pm

      Re: Thank you for this post..

      thank for a post exemplifying the difficulties of using scripts to emulate human-generated content

      link to this | view in chronology ]

  • identicon
    Sohil Memon, 9 Jun 2014 @ 9:57pm

    Shocked Information

    Really? Hows that possible. So, now chat bots will be online rather than being a real human? It's really not worth. But a great achievement in the field of Computer Science.

    link to this | view in chronology ]

  • icon
    bugmenot (profile), 9 Jun 2014 @ 11:21pm

    Online article critique of the Turing Test

    If anyone wants to read an in-depth critique on why the Turing test "is a poor test of intelligence, that it encourages trickery, not intelligent behaviour, and that many intelligent systems would fail this test":
    http://view.samurajdata.se/psview.php?id=d758abba&page=1&size=full&all=1

    It was found this way:

    '4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.'

    The link to the original test contains a link at the bottom to a 1996 criticism of the test:
    'Jason Hutchens... has written an excellent article on what's wrong with it, and with the Turing test in general. Essentially, Hutchens is making the case that the Turing Test is a poor test of intelligence, that it encourages trickery, not intelligent behaviour, and that many intelligent systems would fail this test.'

    The link gives a 404, then through the Internet Archive you find it's available from 2003 as a postscript file, which when using an online converter can be seen here:
    http://view.samurajdata.se/psview.php?id=d758abba&page=1&size=full&all=1

    link to this | view in chronology ]

  • identicon
    cesium62, 10 Jun 2014 @ 1:29am

    "The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction."

    Well that's just wrong. The only way you can tell whether a person is intelligent or not is by communicating with them. If you interact with a person and they consistently seem to be intelligent, you will conclude they are intelligent.

    If a computer can communicate intelligently, it is intelligent, and if a computer is intelligent, it can communicate intelligently. The chatbot on Star Trek was not reasonable because any chatbot that had that command of the english language would be intelligent and would be able to do a lot of things that the chatbot never did.

    In order to create a chatbot that can convince humans that it has human intelligence requires the chatbot to be able to design and write computer programs. It must be able to be given, on the fly, instructions for how to play a simple game, and then be able to play that game. It should be able to critique a recipe for a favorite dish. It needs to be able to analyze and dissect Shakespeare.


    If people in the AI world view the Turing Test as a needless distraction, it is because those people are concentrating on shorter term problems. There are a lot of good things you can do with AI before we get to general purpose intelligence.

    link to this | view in chronology ]

    • icon
      momentum.im (profile), 10 Jun 2014 @ 2:53pm

      Re:

      While I agree on the point that strong AI will be displayed when a computer writes s/w, or understands Shakespeare, all of these are not required to pass a time constraint Turing test. They are a different test, one which we dont have an answer yet, like the prove you are conscious question in the Transcedence movie.

      link to this | view in chronology ]

  • identicon
    Sam, 10 Jun 2014 @ 2:32am

    Poorly researched

    This article irritates me a little. There is a strong theme of hate for Professor Kevin Warwick along with poorly researched attempts at debunking.

    A quick google will tell you that the event was held in partnership with RoboLaw. The event was aimed toward raising awareness about the ability for a chat bot to convince a human it was a human and how this is very dangerous in the online security arena. For example, you're on your banks website and a chat offering pops up asking you if you need some help. You do, so you click on it. You have a lovely conversation and happily hand over details about your account because you're convinced it's a human on the other end. If that were a robot it now has your details and can do with them as it pleases. This is scary and people need to be made aware of it so they can prepare themselves and be better at identifying possible situations where it might be occurring.

    The event was not geared toward some magical development of strong AI overnight, which this author clearly thinks it was trying to claim.

    Time to debunk the debunking.

    == articles attempt at debunking
    - reality check

    == "It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    - I'm not sure how to answer this. Here code is being compared to hardware performance. For all is known the 'script' could be run on a supercomputer. And since when did being a supercomputer imply AI?!?!?!

    == "Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims."

    - Just the smallest amount of research will tell you that many other Turing tests restrict the conversation types that are allowed in the testing. This was the first passing of an UNRESTRICTED TURING TEST. This means that the judges were not told in any way that they had to talk about a certain topic. They were literally sat down and told to chat.

    == ""beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses."

    - I'm not sure about the excessive use of quote marks in this debunking. Is the writer afraid to say these words or feels they carry more weight when possibly said by another party? Anyway, yes it was clever that the developer utilised humans willingness to allow increased errors when talking to younger and foreign people. This is just really clever psychology. Can we not just appreciate that? I see no way it is gaming the system, it's just an easier way to pass the test. Sometimes the simplest solutions are the most effective.

    == The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    - Welcome to science. Ideas and testing methodologies change over time.

    == As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    - Many things wrong with this. There were a total of 350, yes THREE HUNDRED AND FIFTY, tests performed on the day of testing. The judges were picked from all age ranges, backgrounds, genders and nationalities to make the testing more fair. There were multiple academics from multiple universities there to specifically monitor the testing methods and ensure all the results were gathered correctly and the results were interpreted correctly. This is peer review.

    - If this is not enough peer review for you Dr Huma Shah will be publishing a paper at some point in the future on the event.

    == The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - This seems like mostly opinion so I'm not sure how to debunk it. They are right in that it is fun to think about. So why can't re think about it? Lets get talking about the possible effects of this kind of chat with regards to RoboLaw.

    This kind of poorly researched, emotive reporting on scientific subjects really gets my goat.

    link to this | view in chronology ]

    • identicon
      Anonymous Coward, 10 Jun 2014 @ 6:26am

      Re: Poorly researched

      Source code or STFU.

      Seriously, if Warwick isn't willing to publish the source code so that INDEPENDENT researchers can recreate the results in THEIR labs on THEIR computers with THEIR judges, then this is clearly fraud and we need not waste our time quibbling over methodology, definitions, process, or anything else.

      link to this | view in chronology ]

    • icon
      John Fenderson (profile), 10 Jun 2014 @ 7:23am

      Re: Poorly researched

      If it wasn't an attempt at AI, then it shouldn't have used the term "Turing test".

      link to this | view in chronology ]

  • identicon
    a glass of wine, 10 Jun 2014 @ 2:44am

    In the film "Bladerunner" they use a test to try and distinguish the 'replicants' from the humans. A lot tougher than a silly Turing test!!!

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Jun 2014 @ 4:40am

    haha, totally no bias here.
    >its not a supercomputer
    It doesnt have to be one. supercomputers of the 80s are a joke now.
    >i..it lied that its 13yo
    Even better. Humans lie all the time.

    havent read further than that because the first few lines are absolutely bullshit

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Jun 2014 @ 5:48am

    Why do I need a fucking a/c to comment? Makes no sense.

    link to this | view in chronology ]

  • identicon
    auspex, 10 Jun 2014 @ 6:47am

    The whole idea of a Turing test is "kind of a joke"?

    Where on earth did you get that? There are those who argue that it is insufficient or irrelevant, like Jason Hutchens -- but I don't think even Hutchens would call it a joke -- but there are others who are intimately familiar with the field and believe it is still entirely relevant. Myself, I'm not going to bet against Alan Turing.

    link to this | view in chronology ]

  • identicon
    Simon Read, 10 Jun 2014 @ 7:14am

    Proof

    Can we see a transcript?

    There's lots of reporting about the test, but WHAT ABOUT THE TRANSCRIPT? Show us the actual words sent between the computer and the tester(s). Let us judge for ourselves. Too much news these days is just reaction, without giving the basic facts.

    Show us the transcript.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Jun 2014 @ 7:23am

    UPDATE:
    Maximum PC: Computer A.I. Simulating a 13-Year-Old Boy Fools Humans, First to Pass Turing Test
    http://www.maximumpc.com/computer_ai_simulating_13-year-old_boy_fools_humans_first_pass_turing_t est_2014

    IFLScience: A Computer Has Reportedly Passed Turing Test For The First Time
    http://www.iflscience.com/technology/computer-has-reportedly-passed-turing-test-first-time#HZ8Cr bI1DqO2vlEg.99

    link to this | view in chronology ]

  • identicon
    Mats Svensson, 10 Jun 2014 @ 9:02am

    Yeah right...

    Q:
    If atree falls in the forrest and noone is there to hear it, how many people are there?

    A:
    I didn't count, actually, but much more than dozen. And I forgot to ask you where you are from...


    Q:
    Are we not men?

    A:
    Maybe. I think *you* know the answer. Oooh. Anything else?

    link to this | view in chronology ]

  • icon
    RandCraw (profile), 10 Jun 2014 @ 9:09am

    Nonsense personified

    An excellent review, TD. I've long been puzzled that Warwick would continue embarrassing himself (and U Reading) with his media antics. It's hard to imagine he could remain oblivious to the disrepute he's earned.

    In reflection, I think this story is a great illustration of the inability of most news sources to address nontrivial science/tech stories. If such a nonstory gets wide coverage from serious news sources (like the Washington Post), you have to wonder whether they're also inadequate to address more complex and more important fare like global warming, science education, or tech-based threats to society, like Snowden's "shots across the bow" forewarn.

    link to this | view in chronology ]

  • identicon
    Jaan, 10 Jun 2014 @ 9:30am

    some thoughts

    2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.

    - "passing the test" is not about getting a maximum number of the judges to think that the bot was human. The test is passed if, in repeated tests, one or more (preferrably more) of the human judges CONSISTENTLY identify the bot as a person in nearly 50% of the cases AND the person as a bot in 50% of the cases. Identifying something correctly half the time is equal to a coinflip, meaning the judges are guessing randomly, meaning they cannot tell if they are speaking to a person or AI.

    3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.

    - constricting the test is not forbidden. In fact, specialized Turing tests are often used in game AI research to test different aspects of AI, say navigation. They use the term believability test. Furthermore, there is no indication that constricting the test somehow favors the computer. In a correctly set up Turing test, half the time the judges would have communicated with a real Ukrainian 13 year old boy. If anything, such test would probably be even harder to pass, since the creators of the bot would have had to assemble a large corpus of text from 13 year old Ukrainian boys (something I doubt).

    6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - the Turing test arose from the problem that there is no good definition of machine intelligence and that problem is still not solved. The test was devised as an operational definition, i.e. machine intelligence was defined through its validation test. The validation criteria was chosen to be intuitive and self-evident - if something is not distinguishable from human intelligence, then it must be as intelligent. The assumption was that a truly intelligence machine intelligence would certainly pass such test.

    If we allow that non-intelligent agents can pass the test, then we fall into the pit-falls of philosophy, everything that Turing wanted to avoid. What would it mean for something to appear intelligent but in actuality not be? How do we define REALLY intelligent as opposed to appearing intelligent? How do we know that other humans are intelligent and not just appearing intelligent?

    Searle argues on similar lines that all that computers do is symbol manipulation (the Chinese Room argument) without ever really understanding the content. He claims that there can never be hard AI. His argument sure seems very intuitive. But what does it say about the human brain, are not neurons doing signal processing without really appraising thought content?

    If we reject Searle and allow for emergent intelligence, then on what grounds do we reject the intelligence of a hypothetetical chatbot that passes the test (note I am not talking about this particular bot)? Perhaps we sould define artificial intelligence functionally or behaviorially - mere appearance of intelligence is enough.

    link to this | view in chronology ]

  • identicon
    Scionset, 10 Jun 2014 @ 10:33am

    Picking dumb people

    I could go out on the street and pick random people and be able to foll mor than 30% of them.

    link to this | view in chronology ]

  • identicon
    Dawn, 10 Jun 2014 @ 10:54am

    HATERS

    stop saying this let the real science people handle it they know what there talking about

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 10 Jun 2014 @ 12:01pm

    Well, of course I believed not for a second that some program has passed the turing test (which is not a joke). The state of AI is not that advanced yet.

    But this argument does not make sense:

    "1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot."

    How do you define "chatbot" and why can't it have intelligence? Also "script" is just a term to refer to a computer programm written in certain types of computer languages. Why do you think this "chatbot" was writting in a scripting language? Why do you think a programm written in a scripting language can not have intelligence?

    link to this | view in chronology ]

  • icon
    robin verdegaal (profile), 10 Jun 2014 @ 12:16pm

    hmmm

    It's a good thing to check scientific claims against high standards. It would also be a good thing to hold your own reasoning to standards of the same kind. Sadly, at least three of the six arguments you're putting forward are not arguments at all. Not debating the verdict, but you should really try harder than this, Techdirt.


    1. It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.

    - This doesn't seem very convincing. Define 'supercomputer, a virtual computer is not a computer? Also: define 'intelligence'.


    2. Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.

    - Others passed with higher scores. How is this an argument?


    3. It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.
    The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    - Are you trying to say that 13 year old Ukrainians are not human beings?


    4. The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

    + I suppose this is your first real argument. Too bad you didn't take the time to be a bit more specific.


    5. As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

    + Now you're talking. This is a real argument. Why isn't this on the top of the list?


    6. The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

    - And there you go again. It didn't pass the Turing test because the Turing test is stupid?


    And this is just ugly. It's not true because something else wasn't true either:

    "Oh, and the biggest red flag of all. The event was organized by Kevin Warwick (..)"

    link to this | view in chronology ]

  • identicon
    Vaz, 10 Jun 2014 @ 1:07pm

    drunkard chatbot outperforms Eugene's achievement at new true Turing test!

    UK-based computer scientist claims to be able to write a chatbot emulating a drunkard and a 2-month old baby and pass the Turing test, see:
    http://www.mathrix.org/liquid/archives/the-turing-test-pass-fiasco

    link to this | view in chronology ]

  • identicon
    Jeff, 10 Jun 2014 @ 3:09pm

    media

    to me the only real disappointment on the list is Ars Technic, i would have hoped for better from them, the others are all known click whores.

    Too bad about that.

    link to this | view in chronology ]

  • identicon
    Ignacio Agulló, 10 Jun 2014 @ 4:51pm

    I think that Mr. Masnick's judgement is undermined

    Mr. Masnick states, "The whole concept of the Turing Test itself is kind of a joke.", which kind of seriously undermine his credibility on the matter of whether or not the Turing Test has been passed.

    link to this | view in chronology ]

  • identicon
    Eilam Gross, 10 Jun 2014 @ 6:15pm

    Stupid robot

    I suggest to any of you to go to the site of the chatbot and try himself/herself. The 13 years old Robot is stupid. You do not need a computer to tell it. You get it by the second answer when it suggests to you to switch subject and tries to guide you to what it knows. If you insist it starts to give you idiotic answers.... If this is the state of the art we will have to wait another 100 years....

    link to this | view in chronology ]

  • identicon
    Nonny Mouse, 10 Jun 2014 @ 6:20pm

    Well, if gaming (or 'constricting') the test is allowed

    I'm writing one to mimic a three year old who's just learnt to say 'mama', 'need potty' and 'WAAAAAAAAAAAH'.
    Any more nonsense and we'll have William Hague asking Eugene for help with our foreign policy regarding Russian separatists in Ukraine's east.

    link to this | view in chronology ]

  • identicon
    Wesley Steinbrink, 10 Jun 2014 @ 7:53pm

    Cleverbot is better and it is not good either

    Cleverbot scored higher - and here is an example of it:

    Me: What is 1/2 plus 1/2.
    Cleverbot: Not much you.

    Not impressive at all.


    Search.InAllEarnest.com

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 11 Jun 2014 @ 4:52am

    link to this | view in chronology ]

  • icon
    justok (profile), 11 Jun 2014 @ 4:52am

    CBC's As it happens originally went with the herd on this, but the next day, had a correction.
    http://www.cbc.ca/asithappens/features/2014/06/10/has-the-turing-test-been-passed/

    link to this | view in chronology ]

  • identicon
    julius, 11 Jun 2014 @ 6:51am

    ha ha ha

    I tried it an i NEVER got a sane answer! Just aim at open questions and it will attempt to divert you to it's level... Bad.. this 14 year old boy needs to go back to school.

    link to this | view in chronology ]

  • identicon
    Anonymous Coward, 11 Jun 2014 @ 10:10am

    Just type."I was born in (month) (year), how old am I?"
    And he goes "you didn't tell me how old you are, so how old are you" sorry

    link to this | view in chronology ]

  • identicon
    Derek Read, 11 Jun 2014 @ 5:02pm

    further reading

    An explanation describes what has to date been accomplished and passed off as "intelligence" rather than the "mimicry" that it is. See "The Chinese Room" here:
    http://psych.utoronto.ca/users/reingold/courses/ai/chinese.html

    Turing himself obviously did not call it "the Turing test", nor did he really define really good parameters for it. See "The Imitation Game" here:
    http://psych.utoronto.ca/users/reingold/courses/ai/turing.html

    link to this | view in chronology ]

  • icon
    trollificus (profile), 11 Jun 2014 @ 6:48pm

    Whatever else Eugene has accomplished, it's certainly motivated more than a few people to WD-40 those rusty Rhetoric class and Philosophy 210 skillz and bring them to bear on Mike's common sense explication of this story. These comments have only succeeded in sharing their authors' opinion of their own intelligence and analytical ability. Sadly, this does little to answer the question of whether or not this 'Eugene' device satisfies the conditions of the Turing Test.

    More importantly, such respondents do not deal with the question of whether software like Eugene (and predecessors) is not more a "program designed to imitate human conversational function", manipulating symbols and text of which it has no real understanding, than an actual "thinking machine". (This was Searle's objection to the test in his 1980 paper, Minds, Brains, and Programs.)

    When a computer/program can incorporate prior content of a conversation into an original thought or proposition, it will be somewhat convincing, and might actually win the Loebner Prize. (Not to say that's ever a motivation for such research.) Indeed, the 'winner' of the first Loebner competition (Weintraub's PC Therapist) did so by the programmed emulation of pauses and misspellings common to human respondents, not by 'intelligence'. Unless you want to argue that our intelligence is defined by it's limitations and inefficiencies.

    I doubt Turing would consider Eugene a "thinking machine", or an example of "artificial intelligence". Now, when a software/hardware construct can misconstrue obvious common sense objections such as Mike has raised here and then respond as if he had instead presented a tightly-reasoned, peer-reviewed thesis...THEN I'll be impressed. Because choosing the response "Let me show off how smart I am." rather than, you know, actually contributing something, really does demonstrate (the misuse of) human intelligence; as well as some other, less commendable human traits.

    link to this | view in chronology ]

  • identicon
    Bitch Cookie, 15 Jun 2014 @ 8:45am

    Great way to up your page hits

    The original article is exactly what it states, a computer (hosting a chatbot script) passed A "turing test". So what? The term "Artificial Intelligence" is purposefully left vague. Siri, in a new iphone, can be considered a sort of AI. So are adaptive algorithms in many computer games. The hype you have created over this article seems to be nothing more than grasping at a way to increase page hits. If that's correct, congratulations. It worked. This techdirt article is the most popular rebuttal, and just as boring.

    link to this | view in chronology ]

  • identicon
    Politechnically inquiring, 17 Jun 2014 @ 4:16pm

    And Away We Go,

    When I read the story I thought it a bit odd, simply from my experience with a few of the Brian AI's on Twitter which, I have to wonder if they wouldn't pass the test.
    One stumped me for a few days until I got suspicious and asked it who Father Was. It became immediately apparent at that time I had held several conversations with a bot rather than a person. (arrrrrgggg! yes.. I was taken in by a bot, but an extremely smart bot since I was the first of his 15K+ followers who knew it was a bot.

    On Another note, I'd just like to point out why computer science should be an except subject for the media to propagandize, politics shouldn't be the only subject to have some laughs.

    link to this | view in chronology ]

  • icon
    Ninja (profile), 1 Jul 2014 @ 4:00am

    A broken chatbot seems to be more intelligent than some people out there.

    link to this | view in chronology ]

  • identicon
    Veronika, 31 Jul 2014 @ 7:30am

    I thought that by passing the Turing test, the machine is considered just "inteligent", nothing more, nothing less.

    link to this | view in chronology ]

  • icon
    Chad (profile), 4 Oct 2014 @ 6:36pm

    AI

    AT&T has an answering system that can collect information to troubleshoot your problem, and if it fails to solve it, set an appointment for a technician to come out. That is more than an autistic person can do, but we still say that an autistic person is a human with lower than average intelligence and a learning disability.

    link to this | view in chronology ]

  • identicon
    zenobia, 1 Jan 2015 @ 2:11am

    Descartes Discourse on Method and the Touring Test

    Could Alan Turing have gotten the idea for his famous "Turing test" from a passage in Descartes? In 1950 Turing suggested a test that could decide if a machine is exhibiting human intelligence. A person is placed in a room with nothing but a communication device and told to converse with who or whatever is on the other side of the communication channel. He is not told what is on the other side, it could be a machine or a person. He is to converse for a certain length of time, asking questions and talking as if to a person, perhaps even trying to trip it up with nonsense or ambiguous questions. If after that time period the person cannot tell if he is talking to a person or a machine, the "machine" passes the test and is said to be adequately simulating a human mind.

    Toward the end of part five of Descartes' Discourse on Method, Descartes proposes an experiment by which one may distinguish a machine built to look and behave like a person from a real person. A machine could never create intelligible sentences like a person, even one of very limited mental capacity. The machine may speak in words in a limited way, like a parrot for example, but could never respond adequately to the range of all that could be said to it in a long time period. In particular, a machine could not comment on the fact that it is thinking about what it is saying.

    This certainly sounds similar to the Turing test. First there is the goal of distinguishing the human from the non human. Second is the use of conversation as the means to discover the nature of what one is talking to. Lastly, both tests require an extended time period in which to talk to the other. Just a thought.

    link to this | view in chronology ]

  • identicon
    Dave, 26 May 2015 @ 8:53am

    Computer passes Turing test.

    I guess I'm too old to care, back when a passing grade was 60%, but I guess passes these days explains most of the press these days.

    link to this | view in chronology ]

  • identicon
    Andre, 17 Jun 2015 @ 3:34pm

    HI, I POTATO.
    I BET YUO CAN'T TELL IF I COMPUTER OR POTATO HE?
    HEHEHE
    I LIKE MADONNA, DO YOU LIEK VODKA?
    3 out of 10 idiots could not tell if it was a person impersonating a potato or a computer impersonating a potato.

    link to this | view in chronology ]

  • icon
    bugmenot (profile), 17 Sep 2015 @ 12:54am

    >There is no intelligence, artificial or not involved. It's >just a chatbot.

    You seem to fail to grasp the concept of the Turing Test. If something passes the test, it is intelligent, by definition.

    >The whole concept of the Turing Test itself is kind of a >joke.

    Yeah I'm sure you're much smarter than Turing, so your opinion is important. What about Einstein? Are his theories bunk because you say so?

    link to this | view in chronology ]


Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Discord

The latest chatter on the Techdirt Insider Discord channel...

Loading...
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.