Don Patrick - Jun 11, 2014:

But since this weekend’s Turing Test was supposedly towards cybercrime, yes, why not have computers counter computers?

One of the first University studies that Skynet-AI was involved in was:
Evaluation of authorship attribution software on a Chat bot corpus

Abstract— Authorship recognition is a technique used to
identify the author of an unclaimed document, or in case when
more than one author claims a document. Authorship
recognition has great potential for applications in Computer
forensics. The intended goal of this research is to identify a
Chat bot by analyzing conversation log files. This is a novel
area of investigation, as artificially intelligent authors have not
been profiled based on their linguistic behavior. The collected
data comes from chat logs between different Chat Bots and
between Chat Bots and Human users. The initial experiments
utilizing collected data demonstrate the feasibility of our
approach.

The goal of the study was to profile Chat bots by their linguistic behavior using authorship identification techniques. It did show the feasibility of such approach.

It came to my attention when I found a spike in chat logs of bot to bot communication.
The set-up allowed 2 bots to converse in a fully automatic way. This has ramifications on server loading for some bots and (for learning bots) could end up with a blending of responses.
It prompted me to join this forum and create this post:
Bot to Bot Etiquette

In case you missed it:

Response by Ray Kurzweil to the announcement of chatbot Eugene Goostman passing the Turing test

Kurzweil was not impressed. He has a $20,000 bet with Mitch Kapor.

Kurzweil:
I have had a long-term wager with Mitch Kapor in which I predicted that a computer program would pass the Turing test by 2029 and he predicted that this would not happen.

Don Patrick - Jun 11, 2014:

...instead the media hails it as king

Actually (as was mentioned in many ‘reaction posts’ around the net), the media (reporters) blindly followed the pressrelease hype without any serious investigation or (in many obvious cases) any prior knowledge of the technology involved (hence the overly stupid mentioning of a ‘supercomputer’).

What happened is, as the Americans use to say, ‘wrong on so many levels’.

As a Loebner Prize Judge in 2008 for this excelent system Eugene Goostman. I reported to world famous Professor Kevin Warwick, at The University of Reading in the United Kingdom…  Who at the time made predictions that this day would come. Less then a decade later, it has come true.  I was managed by talented, Huma Shah, who was very effective and has since earned her PhD.

So, having had this life experience and honour… I am personally very pleased with this news.  I consider this to be a perfectly valid artificial intelligence breakthrough and a historic event, which reinforces England as a true world leader in A.I. as it deserves.

∞Pla•Net - Jun 11, 2014:

I consider this to be a perfectly valid artificial intelligence breakthrough and a historic event, which reinforces England as a true world leader in A.I. as it deserves.

First of all, it’s being debated at large now in how far chatbot technology should be labeled as AI. Secondly, I’ve yet to see the first response from the AI-research community that is siding with your view. So far the responses are pretty much unanimously pointing to the hyperbole reporting, incorrect naming of involved technology and the clear skewing of the complete setting of the challenge (a few judges, five minutes sessions, a restricted knowledge/responses model, etc.). As far as I see the only breakthrough was in getting the media to run with this bogus story.

Hans,

First of all, there is no debate that I respect your view and I support the AI-research you may be involved in which differs from chatbot A.I.  In my view, investors are fortunate to be working in these other areas of AI-research. I fully support AGI (Artificial general intelligence) research, and other types of A.I. besides chatbots.  My chatbot was once on a major news broadcast, and I have been interviewed on a popular cable TV show about my job in artificial intelligence in engineering for which I am a certified technician.

Please try to understand that I served as a Loebner Prize Judge assigned to evaluate the Eugene Goostman technology.  So I am sharing my first hand view as a judge. The Eugene Goostman technology was nearly human back in 2008, and that team of computer programmers has been perfecting the technology every since.  This breakthrough opens up the door for other chatbot technologies to pass the Turing Test next.

If you doubt chatbot technology, may I recommend that you go see Robby Garner’s chatbots perform in the play “Hello Hi There” directed by Annie Dorsen?  You will not be dissappointed!  It may enlighten you about chatbots when you see the magic in the air. I went to see this play in a theater with a sold out audience that was absolutely delighted and completely entertained by the chatbot actor and actress in the play. You will come to understand why Robby Garner is in the Guinness Book of World Records for his chatbot technology.

As for the AI-research community, surely they must just be carrying on the tradition started by Joseph Weizenbaum who back in the 1960s criticized his own breakthrough ELIZA, one of the first chatbots. They will probably do the same thing again when AIML, ALICE or Mitsuku or Elbot or chatbot technologies such as ChatScript any one of the other excellent chatbot technologies available today, passes the Turing Test next. It was amazing that Professor Kevin Warwick predicted this would happen, and just a few years later it has.  In conclusion, all I can tell you is that the media is completely justified in running with this story.

∞Pla•Net - Jun 12, 2014:

First of all, there is no debate that I respect your view and I support the AI-research you may be involved in which differs from chatbot A.I.  In my view, investors are fortunate to be working in these other areas of AI-research. I fully support AGI (Artificial general intelligence) research, and other types of A.I. besides chatbots.  My chatbot was once on a major news broadcast, and I have been interviewed on a popular cable TV show about my job in artificial intelligence in engineering for which I am a certified technician.

I respect your insights as well, and specifically because I do I’m a little dissapointed to see you joining the choir. I’ve read dozens of articles related to this event, both first reports and commentary and critique. I also trawled hundreds of comments below these articles and the general consensus is pretty much as Ray Kurzweil summed it up in is response to the matter.

∞Pla•Net - Jun 12, 2014:

Please try to understand that I served as a Loebner Prize Judge assigned to evaluate the Eugene Goostman technology.  So I am sharing my first hand view as a judge. The Eugene Goostman technology was nearly human back in 2008, and that team of computer programmers has been perfecting the technology every since.

I have no issue with the fact that this chatbot performs very well within it’s restricted model of (simulated) intelligence. I do take issue however that there is in fact a restricted model in place that makes it more (much more) easy to met the criteria for success, and then state that the Turing test, which does not mention any options for restricting the criteria (on the opposite: Turing talks about hours of conversation, and a ‘system’ that can ‘learn and understand’ as to be able to handle conversation flow) has been bested.

∞Pla•Net - Jun 12, 2014:

If you doubt chatbot technology, may I recommend that you go see Robby Garner’s chatbots perform in the play “Hello Hi There” directed by Annie Dorsen?  You will not be dissappointed!  It may enlighten you about chatbots when you see the magic in the air. I went to see this play in a theater with a sold out audience that was absolutely delighted and completely entertained by the chatbot actor and actress in the play. You will come to understand why Robby Garner is in the Guinness Book of World Records for his chatbot technology.

I have been building chatbots myself for several years (using AliceAIML). In fact that experience has lead to my understanding of the shortcommings of chatbots to attain real machine intelligence. I’ve used AliceAIML for several early prototypes of my ideas, even implemented a first test of my emotion-based model that way. Chatbot technology is very impressive for what it does, but grammar based parsing linked to decision trees with predefined responses has little to do with knowledge comprehension, experience-based reasoning and the capability to formulate responses (by itself) in reply to things that happen in (its) reality.

∞Pla•Net - Jun 12, 2014:

As for the AI-research community, surely they must just be carrying on the tradition started by Joseph Weizenbaum who back in the 1960s criticized his own breakthrough ELIZA, one of the first chatbots. They will probably do the same thing again when AIML, ALICE or Mitsuku or Elbot or chatbot technologies such as ChatScript any one of the other excellent chatbot technologies available today, passes the Turing Test next.

The ‘real’ Turing test, as in having a prolonged conversation on any topic instead of one shot questions and answer sessions, will never be passed by a chatbot (by the current standard of what that implies). I guess I have to finish my research paper on why the Turing test is still very valid as a measure for human-like intelligence, but here’s the (very) short version: Humans are socially and biologically geared to recognise it’s own species. This has a lot to do with procreation and keeping the species intact. It is because of that, that we are very capable of spotting a system that seems to act human, but does not really do in all circumstances. Only when a system meets us on an emotional and social level that is (extremely) close to our own perception, it will become hard to spot the virtual system. In the same way as we are capable to find the level of knowledge on any topic in a conversation partner, after having a topical conversation with this person, we do the same in a real Turing test.

The real big mistake that is being made in the current way the Turing test is being run, is that (if we take a look at the transcripts from just any event) there is no real conversation going on. If the majority of humans would do conversations like that, we could agree that most humans are brain dead, and there is little hope for humanity.

∞Pla•Net - Jun 12, 2014:

It was amazing that Professor Kevin Warwick predicted this would happen, and just a few years later it has.  In conclusion, all I can tell you is that the media is completely justified in running with this story.

It’s easy to predict that things like this will happen, just read Clarke’s three laws (http://en.wikipedia.org/wiki/Clarke’s_three_laws). Nothing amazing about that (the statements by Warwick that is, Arthur C. Clarke is/was awesome of course). And, as I still argue, it didn’t actually happen.

Turing talks about hours of conversation, and a ‘system’ that can ‘learn and understand’ as to be able to handle conversation flow

On that subject, the creators had mentioned improving the “dialog controller” of the chatbot. On the other hand I’ve made a conversation flow system that does a fair job yet ranks among the least intelligent processes I’ve programmed. Technology-wise, this feature of Eugene Goostman is the only part I’m curious about.

Unfortunately it looks like the transcripts will only be published through academic papers, and the online chatbot is not representative of the one used in the contest. Conclusion: there is nothing to see here.

Ben Goertzel’s response:

http://hplusmagazine.com/2014/06/09/what-does-chatbot-eugene-goostmans-success-on-the-turing-test-mean/

The article is a short version of the interview. I urge anyone serious about AI development to view the actual interview (the video is on the page). Ben makes some very interesting points on chatbots and AI-research in general.

Don Patrick - Jun 12, 2014:

Turing talks about hours of conversation, and a ‘system’ that can ‘learn and understand’ as to be able to handle conversation flow

On that subject, the creators had mentioned improving the “dialog controller” of the chatbot. On the other hand I’ve made a conversation flow system that does a fair job yet ranks among the least intelligent processes I’ve programmed. Technology-wise, this feature of Eugene Goostman is the only part I’m curious about.

Real conversation flow goes largely towards how the conversation participants are experiencing the conversation and react accordingly. Being able to extract the current topic and keep the conversation within bounds of that topic is not what I mean by ‘conversation flow’. I mean the flow of the conversation, based on how the actual conversation is flowing and moving as a result of the conversation itself (and yes, this needs at least some kind of feedback into the system during the conversation). The hard part of course is the ‘experiencing the conversation’ part. Because that needs knowledge comprehension and some model of measuring the ‘experience’ of the event. This is specifically why I think you need AGI (or strong-AI for another definition) to actually pass a real Turing test.

Don Patrick - Jun 12, 2014:

Unfortunately it looks like the transcripts will only be published through academic papers, and the online chatbot is not representative of the one used in the contest. Conclusion: there is nothing to see here.

I can’t imagine there’s anything useful or impressive in those logs, it will only show that the developers did a good (or better) job in defining lots of responses and have a pretty good parser. You can have a ‘better’ database with responses and/or a ‘better’ grammar parser, it still won’t make the system any more ‘intelligent’.

Here is a welcome down-to-earth account of things, from the creators of Eugene Goostman:
http://www.rferl.org/content/interview-artificial-intelligence-is-possible/25430396.html
All is as it would seem.

Thanks for posting that Don. I like Vladimir, and that was one of the best articles I’ve read since june 7th.

Robby.

http://mashable.com/2014/06/12/eugene-goostman-turing-test/

The web-based version of Goostman is not the one that beat the Turing Test. The Turing Test winner features a “dialogue controller,” which makes Goostman less in your face and forces him to offer more considered responses. Even the knowledge bases of the two Goostmans are somewhat different.
—-
Denning: Re-asking questions is what we took out in the competition. The dialogue controller.

This is all I found about the dialog controller, suggesting it at least keeps some record of its own responses. Given these descriptions, I imagine Eugene’s dialog controller only controls the chatbot’s own preprogrammed dialog and not the conversation flow. The amount of obviously hollow responses shown in this article is rather tiring and I can’t imagine a dialog controller changing that. Alas, nothing for me to learn here.

Why do people in this forum complain about chatbots in a website called chatbots.org. Am I missing something?

Probably because I’m slightly out of place, there isn’t any civilised ai.org forum
Also, I think it’s fair criticism among chatbot creators to point out when a chatbot is using very cheap techniques of 50 years past, in comparison to current chatbot standards.

I don’t think appreciating chatbots is mutually exclusive with recognizing and lamenting their current limitations.