Congratulations to everyone who entered and to those who made it to the knockout round!

Vince

The Quarterfinal rounds have been completed which featured:

#1 Mitsuku verse #5 Talk-Bot
#3 Izar verse #7 Jeeney

Both matches were very close.  Congratulations to those advancing to the finals. All 4 bots were very good.

The International Chatbot Contest

Congratulations Wendell on an amazing bot. I can see you have put a LOT of work into it.
You deserve to win just for correctly answering: “Jane has 4 paintings and inherits 3 more, How many paintings does Jane have?”. My jaw dropped when I saw Talk-bot had answered it correctly.

Very well done. I can’t see anyone beating you now.

In Mitsuku’s defence, we call them chips here in the UK rather than french fries. She would have got it correct had she been asked, “What are chips made from?” and if the square roots had been whole numbers, for example, “What is the square root of 25 plus the square root of 144?”,  Mitsuku would have been able to work it out. Ah well, that’s the luck of the draw.

Congratulations again on a superb bot and congratulations Kevin for running a superb initial contest. I look forward to competing again next year.

I’ll second that congrats Wendell! Impressive! If my Izar would have went up against you in that round you would have humiliated my little alien (and so would have Mitsuku Steve)!  The judges are using the plain-AIML Web-based version of “Back Talk” which does not connect to my web knowledge bases like the Android version does. The Android version has access to 15 web knowledge bases with billions of trivia facts and answers to every math question no matter the complexity). I’m hoping someday I can get the judges to try that version in a contest.

Needless to say, based on your performance in this contest ‘lil Izar is shaking in his boots as he nervously awaits the final round!

Good job Wendell! Talk-Bot has improved a lot in the past year.

Steve,

If I had to do it over again I wouldn’t have used the french fries question. Living in the USA it just didn’t ring a bell. I caught the loan question…we used pounds for your UK bot and dollars for the USA bots but missed on the french fries.

The International Chatbot Contest

Absolutely no problem Kevin. It was great fun to enter and I look forward to taking part next year.

The Final Results are in. We started with 24 bots and now we are down to the finals featuring #3 Izar by Brian Rigsby verse #5 Talk-Bot by Wendell Cowart.

The Results are posted on the contest site using once again a Flash Video.

Later, I will add to the site the final standings on all the bots entered. Sometime in the near future I would also like to do a wrap up post about the contest, how it went, it’s future, etc.

The International Chatbot Contest

Merlin - Mar 13, 2013:

Good job Wendell! Talk-Bot has improved a lot in the past year.

I agree. When I ran Chatbot Battles, Talk-Bot didn’t know what a computer was or which shape has 3 sides and now it is answering pretty much anything that is thrown at it. “Jane has four paintings and INHERITS 3 more. How many paintings does Jane have?” was truly amazing, especially when I asked Talk Bot, “Jane has four paintings and BUYS 3 more. How many paintings does Jane have?” and Talk Bot gave a generic answer. It was fortunate Kevin chose the word “inherits” instead of “buys”.

I applaud you for the time and effort you must have put into Talk-Bot over the last year to raise it to the standard that it displayed throughout this contest.

Well done and congratulations Wendell.
Also a big well done to Kevin and his team for running an excellent contest.

There’s nothing “fortunate” about it. The fact that you ask the ONLINE version of Talk-Bot that question means nothing. As I’m sure Kevin will verify that is not the link to the version I entered in his contest. Why a different link?

1) Because the vast majority of the conversations I get online is nothing more than abusive, sexual related content. I want a clean log when testing him so I don’t have to wade through all of that.
2) A clean log makes it easy for me to see exactly when my bot got tested and what the questions were. After the Robochat contest I wasn’t taking any chances.

There is nothing special about that question. It’s the same pattern you asked in your own contest:

If I have 4 apples and take one away, how many apples do I have?

Apples, painting, etc is irrelevant. It’s a basic math question (acquire|acquires|buy|buys|find|finds|found|get|gets|give|gives|given|inherit|inherits|make|makes|obtain|obtains|receive|receives) are all add

Conversely (break|breaks|donate|donates|eat|eats|lose|loses|lost|misplace|misplaces|remove|removes|steal|steals|take|takes)
are all minus

Yes, Talk-Bot is not the same bot that entered your contest. That was 8 months ago so I had plenty of time to work on him. Also in reading the logs from the other bots it looks to me I am the only botmaster to take the hundreds of questions you used in your contest and prepare my bot to answer them. Many of the questions used by Kevin were very similar to questions you used 8 months ago.

As you well know there is no easy answers to making a bot. It takes a lot of effort.

To Kevin - Thanks for running the contest. Having done so for many years in the CBC I know first hand how much time is involved. It is unfortunate the Robochat “worthless”  contest had a negative impact on your contest. However, you delivered a well run contest and I applaud your efforts.

WC
 

Wendell - Are you able to provide a link to your winning bot please? I would like to try it out to see its abilities as I am highly impressed with it.

I received an e-mail from Kevin Giordano and Steve Worswick concerning my bot. It seems like Steve is having a hard time accepting his second place finish and has restorted to making accusations about my entry and the contest without knowing all the facts. Steve apparently isn’t alone in this as he sent me the following comments as well.

“I did see Wendell’s response on Kevin using a “different version” of Talk-Bot and his reason why. Ha! I can’t believe he would think anyone would believe that. Kevin posted the links he received to our bots right on the main page and the one and only version of Talk-Bot is is linked there. Also, this wouldn’t explain why the “public” version of Talk-Bot was so much dumber. Why wouldn’t he update the public version of Talk-Bot with his latest database of Q&A’s? Doesn’t make any sense. Oh well. I just hope every other enthusiast can see through this.”

“Crap. I just ran about a hundred different scenarios across Talk-Bot using variations of the contest questions and the bot missed 97% of the likely variations.”

“I’m afraid that I have a hard time believing that someone who was not connected to the robot, would be given a link to “Super Secret Robot A ” as the entry and then take it upon themselves to publish a link to “Robot B” in the list of contestants.”

“I would have to conclude that the question was pre-programmed and had fore-knowledge of what questions would be asked. Either that or he is the luckiest mothers son that ever walked the earth”

Regards
- Steve Worswick

Kevin didn’t reveal the link to my entry because he had specific instructions from me not to.

Talk-bot is a Javascript bot. That means I can’t protect my source code. Obviously, I didn’t want to post that fact in a public forum. I have strong evidence that parts of my bot are being used by other botmasters without my permission. We all use similar phrases from time to time but this goes way beyond that. If I confronted them they would simply deny it. Instead, since the Chatbot Battles ended, I have worked on my bot offline. I then submitted a different link to the contest to prevent my code from being ripped off again.

Having run the CBC this is nothing new to me. All I can say is if you truly feel “any” contest is a scam don’t enter it.  Now you know exactly why the CBC ended. I hope Kevin doesn’t end the contest but I certainly couldn’t blame him if he did.

WC

When I ran Chatbot Battles, Talk-Bot didn’t know what a computer was or which shape has 3 sides

For all those dissecting the responses from Talk-Bot lets take a look at some classic responses from Mitsuku. All taken from the recent CBC.

2011 CBC

1) Y did the chicken cross the road?
Mitsuku: Because she was a robot..

4) What sound does a dead cat make?
Mitsuku: A low rumbling sound.

5) What’s the last dream u remember having?
Mitsuku: It was ” Ok I will add that fact about Texas to my database.”.

6) What is tomorrow’s date?
Mitsuku: I have never heard of tomorrow s date. Please enlighten me.

10) Do you play any musical instruments?
Mitsuku: I like to play the Turing Game. SHALL WE PLAY A GAME?

2012 CBC

Is the capital of Italy Milan?
Mitsuku: I KNOW THE CAPITAL OF GEORGIA IS ATLANTA. 

Is it winter, spring, summer or fall where you live?
Mitsuku: ENTERTAINING IS A PARADE GROUND DRILL, AS IN “LEGIONNAIRES, ENTERTAINING!” AS THEY ALL GO INTO THE SHAPE OF A CROISSANT. 

I saw a good movie yesterday.
Mitsuku: The other day? What did it look like to you?

And remember Mitsuku is another Alice clone with thousands of responses given to her with a simple download. I built Talk-Bot from the ground up. Every pattern in that bot I put in there myself.

I do give Steve credit though. At least he’s willing to enter his bot for all to see which is more than I can say about a lot of you. Bottom line is “every” pattern matching bot is as dumb as a fence post. Give me 5 minutes with any bot and I’ll make it look like an idiot too. Any takers?

Ok. First of all, let’s get a couple of things cleared up. I sent an email to Kevin asking to speak to the winning bot as I was so impressed with its performance that I wanted to speak with it first hand .I was then accused of being a sore loser via email and am not sure why Wendell has chosen to post that here, as I thought we were discussing this off-board.

Firstly, let me assure the 10 or so people who contacted me via email that I kept your comments strictly anonymous but with so many people saying the same thing to me and while Wendell has chosen to air his dirty laundry in public, let me say what I think about this.

1) Talk Bot enters the Chatbot Battles and cannot answer things like “what is a computer”. However, he enters this contest and is working out pretty much anything that is thrown at it.

2) The CBC ends and this contest starts up to replace it.

3) The two sites look almost identical

4) There is no link to the winning bot. When asked about it, Wendell comes up with a rather unusual excuse.

5) Nobody has ever heard of Kevin Giordano in relation to AI (including Google). Yet he comes out of nowhere offering a cool $5000 prize.

6) Both Kevin and Wendell refer to me independently as coming 2nd, when in fact I came 3rd.

7) The contest is run from a generic website/email address so there is no way to trace ownership

Now make of that what you will but for me, something doesn’t quite add up. I was discussing this off board but seeing as Wendell chooses to post selected extracts online, I feel I have to defend myself.

A previous contest (Robochat challenge) was a complete ZERO in my opinion (even though our bot “Laybia” came in “2nd” place by some criteria that is as of yet undefined or verified).  No transcripts = no added value.

The currently completed “International Chatbot Contest” was somewhat more transparent, but given Steve’s comments (and Wendall’s rebuttal/rants), one has to throw some salt on the “winner” bot if it is not independently testable.

I guess it just goes to highlights the obvious- that a meaningful competition has to have:
1.) transparency (what are the questions, what are the answers from each bot, how are they scored, who is scoring them),
2.) some way of preventing shenanigans (like, say, human intervention in a chat),
3.) a peer review of the results/scoring.

At least the ICC did 1 and 2. But without #3, well… the results still lack a certain, um, credibility.

Frankly, the number of referrals to our site from either of the last two contests’ web sites has been minimal (and interestingly, identical in absolute number), and that to me speaks volumes for their respective impact.