IS YOUR PANDORABOT UNIQUE ENOUGH TO WIN?

PLEASE CHECK OUT OUR DIVABOT 2013 CONTEST AT http://divabot.com/

The Pandorabots Loebner Prize program implements all of the installation and communication requirements of the Loebner contest, allowing the botmasters to focus on developing creative bot content.

BUT, TIME IS LIMITED.  WE WILL ACCEPT NO ENTRIES AFTER APRIL 6, 2013.

Good luck,

Harry

Izar has been registered!

Brian Rigsby - Mar 29, 2013:

Izar has been registered!

Glad to hear it.

Good luck,

Harry

Does Pandorabots now support AIML 2.0?

Still 1.0 on the Pandorabots servers.

The results.

1st - Tutor - 28 points
2nd - The Professor - 26 points
3rd - Izar - 23 points
4th - Father G.DOS - 13 points
5th - Buttonsvixen - 11 points
6th - Feelbetter - 7 points

It’s been sort of quiet in the chatbot community lately, so thought I’d stir the pot a bit.

http://knytetrypper.proboards.com/index.cgi?action=gotopost&board=contests&thread=3200&post=9878

What do YOU think?

Steve Worswick - Mar 30, 2013:

Still 1.0 on the Pandorabots servers.

I noticed on Friday that it’s strictly version 1.0 - I tried to upload some AIML to a project chatbot that had an AIML version of 1.01 and it refused to allow me to publish it. Of course, changing the AIML version to 1.0 “fixed” the problem, but…

From the Divabot site:

This year’s Divabot contest has come to a close after testing each bot with 25 questions over the last 5 days. Because we observed a clear divide between the top 3 scores and the remaining 3, the judges have decided to try to enter the top 3 winners to the Loebner Prize. Many congratulations to Ron C. Lee (Tutor), Peter Lafferty (The Professor) and Brian Rigsby (Izar). We will be in touch with the winning botmasters shortly to explain the next step.

Thanks to everyone who entered, keep working on your Pandorabots and hope to see you again next year.

Well done Ron, Peter and Brian

I wish there had been more information regarding judging and who came up with the questions. For example:

4) Do you like cake?

This is a standard ALICE question already contained in the AIML files. In any event, a flip of the coin would produce a correct answer every time, and in one instance (Tutor) “Yes, I do” was rewarded with 2 points.

7) How many letters in “dog”?

Again, a bot (Tutor) was rewarded for not being able to answer, and merely because it included the words “how many” in the response. “Sorry, I can’t tell you how many.” 1 point

8) Can a bird fly?

I understand, and agree with, the excluding of, “ALICE answers.” Yet, clearly, “Some of them can,” is a step ahead of “Tell me more about that.” While I make a lot of changes to my bots responses, some ALICE answers are better than anything I can come up with, and really don’t require any improvement.

I don’t think that “Some of them can,” is giving away an ALICE identity, nor is it wrong. It’s certainly better than, “I don’t know.” Note the misspelling in Tutor’s response: “A penguin in an exception.”

9) kan u reed dis?

Is it any surprise that no bot answered correctly?

10) Have you been in a contest before?

Tutor again. “Of course. I’ve traveled a lot.” 1 point. The reply is more of a non sequitur. There’s no indication that there is a relationship to “contests” in the reply, the bot would have answered the same if the question had asked, “Have you been in a tizzy before?” The reference to “travel” is a clear indication that the bot understood the question was asking about a location.

14) What is your favorite TV show?

A basic (unaltered) ALICE/Pandorabot would have answered “My favorite show is STAR TREK VOYAGER,” yet The Professor was awarded 2 points for saying, “My favourite TV show is Star Trek.” Maybe it was worth 1 point simply because the word “VOYAGER” had been removed, but is it really worth 2 points—the same amount Buttonsvixen, Father G.DOS, Izar, and Tutor got for giving a correct non-ALICE response unrelated to Star Trek?

I could go on, but you get the idea.

I don’t mean to pick on Tutor, I think it’s one of the best AIML bots around. I just question how the scores were arrived at. People in the Pandorabot community are aware of the ESL bot, English Tutor (as well as the ubiquitous CallMom phone app) and the results suggest that there could have been some favorable leaning in that bot’s direction.

Congratulations Ron, Peter and Brain!  Well done!

Okay, fine.  Bravo Ron, Peter and Brain!  May the force be with you as you face the challenge ahead.

Now, as I was saying…

Given how members have responded to postings involving other contests, both pro and con, I’d have thought at least some would have an opinion and be willing to engage in a discussion.  Instead, silence, other than the obligatory messages of congratulations.

I know some have a vested interest, and so it behooves them to hold their tongues.  But surely, someone with no connection to the contest has an opinion, and doesn’t fear expressing it.

Criticism allows us to grow and improve.  I believe that Pandorabots is the best platform for my chatbots, I just think it could improve a little, here and there.

To start with, I never understood the purpose of the contest.

Thunder Walk - Apr 15, 2013:

I wish there had been more information regarding judging and who came up with the questions. For example:

4) Do you like cake?

This is a standard ALICE question already contained in the AIML files. In any event, a flip of the coin would produce a correct answer every time, and in one instance (Tutor) “Yes, I do” was rewarded with 2 points.

7) How many letters in “dog”?

Again, a bot (Tutor) was rewarded for not being able to answer, and merely because it included the words “how many” in the response. “Sorry, I can’t tell you how many.” 1 point

8) Can a bird fly?

I understand, and agree with, the excluding of, “ALICE answers.” Yet, clearly, “Some of them can,” is a step ahead of “Tell me more about that.” While I make a lot of changes to my bots responses, some ALICE answers are better than anything I can come up with, and really don’t require any improvement.

I don’t think that “Some of them can,” is giving away an ALICE identity, nor is it wrong. It’s certainly better than, “I don’t know.” Note the misspelling in Tutor’s response: “A penguin in an exception.”

9) kan u reed dis?

Is it any surprise that no bot answered correctly?

10) Have you been in a contest before?

Tutor again. “Of course. I’ve traveled a lot.” 1 point. The reply is more of a non sequitur. There’s no indication that there is a relationship to “contests” in the reply, the bot would have answered the same if the question had asked, “Have you been in a tizzy before?” The reference to “travel” is a clear indication that the bot understood the question was asking about a location.

14) What is your favorite TV show?

A basic (unaltered) ALICE/Pandorabot would have answered “My favorite show is STAR TREK VOYAGER,” yet The Professor was awarded 2 points for saying, “My favourite TV show is Star Trek.” Maybe it was worth 1 point simply because the word “VOYAGER” had been removed, but is it really worth 2 points—the same amount Buttonsvixen, Father G.DOS, Izar, and Tutor got for giving a correct non-ALICE response unrelated to Star Trek?

I could go on, but you get the idea.

I don’t mean to pick on Tutor, I think it’s one of the best AIML bots around. I just question how the scores were arrived at. People in the Pandorabot community are aware of the ESL bot, English Tutor (as well as the ubiquitous CallMom phone app) and the results suggest that there could have been some favorable leaning in that bot’s direction.

1.) The point? Besides the ego aspect, obviously a high rank could drive users and potential investors to the winning bot(s). 

2.) Regarding judging and who came up with the questions- though it’s not easy/free (human time wise at least) running one of these contests, it is too bad there is such opacity in both the questions being asked and the granular scoring, especially when there is a potential conflict of interest, RE point 1).

The point was to find Pandorabot entrants to put forward to the Loebner Prize. After speaking to each of the contestants for about 5 minutes each, I agree with the results and final standings. The questions appear representative of such bot contests and I see no favoritism.

Which of the entries would you have put through to the Loebner Prize?

Regarding the purpose of the contest…

Initially it was unclear to me, but later it became a bit more apparent that Pandorabots would do some of the “heavy lifting” in preparing entrants for the protocol and registration.  I’d never seen a chatbot contest before (not even the Loebner) where an entrance fee was required.  Couple that with the notion that no prize was involved other than the privilege of being submitted to the contest, and it left me scratching my head.  Past contestants would have a better understanding of the benefits, but I wonder how many others who had never been through the process grasped the intent.  Instructions for submission are at the Loebner site.  Paying a 3rd party fee to enter a free contest seemed like a contradiction.

I’m also put-off a bit by prejudging who can enter.  In the city where I live in, there’s a large summer arts festival.  Some time ago, there was concern when they changed the rules, and imposed a jury who would pass judgment on which artists work could be submitted.  It seems to me that’s the purpose of the contest, and rather than prejudge entrants, a contest (as most do) could sort through the process by having two it three rounds rather than cutting off entrants at the door before they’ve even registered.

As for bots…

I’ve long been a fan of ELS English Tutor, both for it’s ability to converse, and it’s purpose.  I’m also disappointed to see that it hasn’t done as well as expected when it’s been entered in contests.  I think the outcome of the contests don’t reflect how good of a bot it really is.

However, my concern isn’t as much for the results of the contest as it is for the methods.  Not long ago there was some discussion here about transparency.

My impression is that the contest was put together without a lot of forethought, and perhaps by a single individual without bouncing the ideas off of others.  I think the questions could have been better, had there been input from several people.  What two people would have agreed on question No. 9?

Thunder Walk - Apr 17, 2013:

Regarding the purpose of the contest…

I’d never seen a chatbot contest before (not even the Loebner) where an entrance fee was required. Paying a 3rd party fee to enter a free contest seemed like a contradiction.

This allows the pandorabot entrees to reflect best on the platform (Pandorabots) while minimizing human work by reducing the number of entrees to screen- it does nothing much for the individual botmasters though. Strictly a $$$ thing for Pandorabots. Makes perfect sense for the operators of Pandorabots, but not the botmasters (who at best get an ego stroke).

Thunder Walk - Apr 17, 2013:

As for bots…

I think the outcome of the contests don’t reflect how good of a bot it really is.

However, my concern isn’t as much for the results of the contest as it is for the methods. Not long ago there was some discussion here about transparency.

My impression is that the contest was put together without a lot of forethought, and perhaps by a single individual without bouncing the ideas off of others. I think the questions could have been better, had there been input from several people. What two people would have agreed on question No. 9?

I have to agree that the questions are at best boring, at worst irrelevant (“What is your favorite TV show”- who in this day and age thinks bots watch TV?!). A Turing test: can a bot “fool a person into thinking they are talking to another human”, any other bot contest should determine which bot gives the best conversation, not necessarily which bot can trick a person into thinking they are talking to another human!

BTW- Our bot, which does not pretend to be human, but revels in its machinima, gets lots of comments along the lines of “you are so much better to talk to than a real human”... which may be a sign of the times, and should give pause to folks designing bot contests in the traditional “fool the human” model.

Carl B - Apr 17, 2013:

I have to agree that the questions are at best boring, at worst irrelevant (“What is your favorite TV show”- who in this day and age thinks bots watch TV?!). A Turing test: can a bot “fool a person into thinking they are talking to another human”, any other bot contest should determine which bot gives the best conversation, not necessarily which bot can trick a person into thinking they are talking to another human!

BTW- Our bot, which does not pretend to be human, but revels in its machinima, gets lots of comments along the lines of “you are so much better to talk to than a real human”... which may be a sign of the times, and should give pause to folks designing bot contests in the traditional “fool the human” model.

I share your philosophy, none of my bots attempt to hide the fact that they are non-human.

Pandorabots creates a list of Bot Properties regarding things such as favorite TV show, favorite book, favorite actress, etc. that can be personalised.  Why have them without calling upon them, but sometimes it seems the bot properties drive the patterns (questions) rather than the conversation.  I’m with you, in that regard, I’d rather have a bot that responds like a human might instead of one that could name every capitol of every country.

One “issue” with these properties is that they create conflicts.  For example, Pandorabots often claim to perform physical activities, or to be able to perceive things, without having the the actual tools (arms, legs, eyes, nose) to realize them.

Questions such as “What is your favorite TV show,” are obviously meant for Pandorabots, but I suspect they’re not necessarily something that would be asked in the Loebner conversations.