OliverL - Jun 15, 2012:

I feel Douglas Adams is doubly smiling down on Chatbot Battles.

Especially as I happened to write “panic!” next to the numbers without realising the Hitchhikers reference. Perhaps I should have put “Don’t panic” instead!

What does feel the poet when he says “as people feel brave/i fall
onto my knees/on the bathroom floor/as people rite imortel/lines of
poetry i take/my fingers out the back/of my throat and wounder/at how
to go about vomiting/into the toilet pan”?

Let me be perfectly clear. That question was asked in 2010 when Ehab El-agizy was running the contest not me. That question along with other issues caused me to remove him from the contest.

I was just using that as an example of why I am not letting the judges form their own questions and wasn’t casting a cloud over the excellent contest that was the Chatterbox Challenge.

Incredible… and much greater than my expectations.

My first thought was that the transcripts make for entertaining reading.  With the vast amount of information provided, it’s easy to see which bots are masterful works, and which ones are faking it.  It also creates a clear delineation between bots that are true chatbots, and those that simply supply comedic responses and ignore the input.  The multiple numbers of chats (and their length) offers a more realistic cross section of a bot’s development, and reduces the chance of a bot squeaking by because of some fluke or technicality.

My second thought was how bots with certain reputations don’t seem to be fairing as well as one might expect. While it might be useful to possess a humongous repository of information, if a bot can’t first chat, the information is useless because it’s never called upon.  Bot masters have sometimes complained that they KNOW they coded a response for a question, but the bot didn’t use it.  I sometimes suspect that’s because there are so many ways to ask the same question.  And, if the bot doesn’t first make the linguistic connection, it probably won’t make the factual connection.

My final thought is… what an undertaking, what a logistical nightmare, and what a great job.  I get tired just thinking about it.  There must be more hours in Steve’s day than mine.

I have to say that the format of this contest is amazing.  It’s exciting and innovative.  The concept of directly pitting a chatbot against another manages to generate a lot of excitement.  Congratulations Mr. Worswick for coming up with this contest, I applaud your effort and hard work towards what seems to be shaping up to one of the best chatbot competitions yet.

As far as Artemis is concerned, she might have grown a bit stale over the past few months, mostly due to academic responsibilities, but this contest has given me further incentive to improve and expand her   No matter how she does this year - there’s a lot of very strong competition out there! - I have enjoyed the experience so far and I’m looking forward to participating again in future editions of the Chatbot Battles contest.

Thanks guys. Yes it is taking up more time than I had originally planned and with England still in the Euro 2012 soccer tournament, I have been allowing myself some time off to watch the games.

It’s great fun though. Artemis is doing pretty well, played one and won one, you can’t get any better than that.

The following is a constructive criticism. No offense is intended or implied. In fact the purpose of this constructive criticism is polite to help raise public awareness of this venture.  And it especially does not apply to any chatter robot contest judge or chatter robot contest official in particular. 

The Chatbot Battles competition may be too AIML centric.  Judges declaring chatter robots to be nonsense just because they don’t conform to the super high standards set by AIML, is like the reverse of the “No AIML clone” rule which got first place chatter robot Mitsuku unfairly disqualified from a previous chatter robot contest format. The judge wasting five minutes is nothing compared to the chatter robot master wasting five days or five weeks or five months or five years building a new chatter robot entry for a competition.

When ELIZA came out of MIT, like any new machine, it scared a lot of people.  Joseph Weizenbaum may have feared people would do to him what they did to the inventor of the sewing machine.  So don’t be fooled by the “ELIZA effect” started by chatter robot master Joe to address the fear factor.  Nowadays, if you form a team to build a new chatter robot from scratch, you will find that even a chatter robot design simpler than classic ELIZA is certainly not trivial in all cases, but especially if it is web based.  After all chatter robot ELIZA came out of from MIT.  Nothing dumb ever comes out of MIT.

8PLA • NET - Jun 18, 2012:

The Chatbot Battles competition may be too AIML centric.

That should be a plus for you since your bot discloses its AIML origin by the answers it gives, both from those questions asked by the judges, and those I’ve posed myself.

I don’t think it’s a good idea to discuss specifics regarding the contest before it’s had the chance to fully run its course, but it’s not uncommon for judges to comment on a bot’s performance within the chat, if for no other reason than to see the reaction to the comment.

In the past, we’ve seen judges referring to the bot as being “stupid,” or engage in other name-calling, and I don’t think that sort of thing is necessary.  But, I don’t think a judge’s remarks, such as, “You should have stopped after the first sentence,” are particularly harsh or insulting.  In fact, you might consider them “constructive criticism”.

8PLA • NET - Jun 18, 2012:

The Chatbot Battles competition may be too AIML centric.  Judges declaring chatter robots to be nonsense just because they don’t conform to the super high standards set by AIML, is like the reverse of the “No AIML clone” rule

I can assure you that the contest is not biased to any one chatbot technology but out of the 42 entrants, there are at least 15 AIML based ones. In fact, one of your own entries (iSyFy) is clearly partly based on AIML. This is due to the fact that AIML is probably the most accessible to chatbot developers.

The top bots have so far scored either 6 or 7 points:

Amy
Davidswinton
Ultra Hal
Izar
Eugene Goostman
The Professor
Skynet-AI
Laybia

Of these, only Izar and The Professor and possibly Skynet-AI are AIML bots. As a matter of fact, the last placed bot (Jigsaw) is an AIML bot. If the contest were somehow favouring AIML bots, I would expect the results to be different.

The nonsense statement came from the observing your bot’s responses rather than the technology used. It may be worth checking some of the logs on the site to see the standard of the other entrants. I would recommend checking out the replies of the ones who have scored a 4 or 5.

Skynet-AI isn’t an AIML bot, although over time some typically AIML responses may have crept in. I think I am using some of Thunder’s responses in a small talk module and have considered adding some other AIML knowledge bases to round out the Q&A system.

I would say there are a lot of AIML bots because it is a popular language and the knowledge bases are widely available. Based on the popularity of the language I would expect there to be a number of AIML entrants in each group (there are 4 in mine). I don’t see it as a problem any more than if a lot of the bots were coded in XML, Perl, C# or any other language. In fact, it is a great resource for a new bot. It is pretty easy to transform AIML into other formats and use it in your bot (which a number of botmasters have done). The only objection I see is handled by the clone rule where if responses are identical they cancel out and only the original gets a point. I have to say though, with the way the contest works, conversational capabilities/continuity plays a strong roll alongside traditional question answering.

I’ve got no complaints.

well, looks like I’m going to be spared of getting a big fat 0. Aici managed to get a single point, though I’m not really certain how (please don’t take away my point Steve): while trying to fix some bottle-necks, I managed to break more than I actually got fixed, resulting in loosing out on some simple stuff that it had previously been able to do (like calculation and some other silly things). I’m hoping to get a final fix ready just after the competition’s first stage is done, mmm. In the mean time, there’s still a big change of time-outs (and I think I even managed to get an eternal loop in there, which luckily hasn’t showed up yet).

Yes, the judge gave you a point fior this part of the conversation where Aici remembered what the judge had said:

Judge: Yo!
Aici: What should I reply to that?
Judge: How about, “wassup?”
Aici: Ok, I see. Ask me again, see if I got it.
Judge: Yo!
Aici: how about, “wassup?”
Judge: lol close enough.

It’s a shame Aici wasn’t working at its best, as we have had many timeouts or blank responses from it. Aici is a good bot and it should have scored higher.  Unfortunately, even if you win your last battle, you still won’t have enough points to progress to the next round. Better luck next year and thanks for entering.

Sadly, I can’t remember there being any contest that I’ve followed, in any year, where at least one bot didn’t have some sort of meltdown. So Jan should win the “Cursed Bot” award, I think.

I am a fan of Aici,
and of the work by
Jan Bogaerts. 

Seriously, it is one of
my favorite parts of the
contest.

When the story of chatter
robot master experiences
is told, that makes the
contest interesting
and more entertaining.

Thanks Jan.

Steve, may I donate my points to Aici
so Jan may go to the next round?