I will be submitting my software for the 2011 Loebner contest. I’ve taken quite a different approach to AI and believe mine may have a fair chance of passing the Turing test.

I just wanted to clarify some of the rules as I’ve been concerned about the idea of it becoming a guessing game (a 50/50 coin flip) when the judge cannot determine which one is a computer. Is the scoring system set up in such a way that both can be judged as equally human? And if so, can the computer win the contest for being equally rated human? (or must it somehow prove to be “more human than human?”)

Thank you for your time,

Bobby

There will be four judges.  Each will be presented with a paired comparison, one computer, one human.  The judges must decide which is which.  If 2 or more of the judges mistake the computer for the human, the program will have “passed” the test, the submitter will receive $25,000 and the Silver Medal, and the contest will move on to the A/V input stage.

Thank you.

Based on that, a perfectly capable, human-level AI will have exactly 50% odds to pass the test. Hopefully the contestant with that AI will not encounter many years of “bad luck”.

Hi Powerai,

This video might help to see how it works in practice:

http://www.chatbots.org/awards/announcement/loebner_prize_2009_video_report/

Erwin

I just wonder how can a judge make mistake to believe a chatbot is a human adult?
How can they ask questions? It’s so easy to judge between a computer and a human.
I believe 10 questions are enough to find out a simply pattern based chatbot.

Is there chatting logs of the past test?

10 ? ! I can think of dozens of different ways my first input could detect it was a bot !.....

human: count the number of words in the next thing i say to you, if it is more than 2, say yes, otherwise say no, but if it’s more than 10 then I want you to reverse the words.
human: dog, cat, fish
bot: pc

or

human: Will you answer no to this question?
bot: If I say no, then I am answering no to your question, but by saying no i’m wrong.  If I say yes, then i’m wrong also.

human: I’ll say 3 words, take the first word(letter?) of each, in reverse order and give me the word
human:  ride a canoe
bot: the word is car !

user: *W skasd *H* asfasfd *A* asdfsaf *T*  asfsdaf sdaf   *I* asdf *S* asdf *Y* asdf s *O* asdf   *U* asdf s—-  *R*  asf sis s *N* asdf *AME*  ?
bot:  My name is bob !

so Nathan, unless you have one VERY powerful truly ‘strong ai’ chat bot engine, and if they do these tricks in the Loebner contest (I have asked, received no reply), then I wouldn’t get your hopes up thinking of winning !

@Nathan: Judges don’t believe chatbots are human adults, at least, until now. Chatbots are getting better, but it’s still far from this ideal. However, some chatbots perform better than others. The best of them wins the Loebner Prize Broze Awards.

http://www.chatbots.org/awards/announcement/loebner_prize_2009_video_report/

Erwin, you had mentioned that their have been chatbots that have actually passed the Turing Test, in ‘limited areas’.  This must not have been in the Loebner prize, since it doesn’t acknowledge any winner so far.

What contest was it, i would like to see the chat logs

I actually don’t think they passed in contests. I’ll check it out! I actually would love to see the chat logs as well.

Hi Erwin, how can they evaluate which chatbot is better when all of them can not pass the test?

Erwin Van Lun - Aug 31, 2010:

@Nathan: Judges don’t believe chatbots are human adults, at least, until now. Chatbots are getting better, but it’s still far from this ideal. However, some chatbots perform better than others. The best of them wins the Loebner Prize Broze Awards.

http://www.chatbots.org/awards/announcement/loebner_prize_2009_video_report/

I have split this thread into a new one “Challenging issues in NLP” , since it was starting to have really not much to do with the LP.

Thank you, Victor. I was trying to do the same thing, but couldn’t quite figure out where to make the split.