BREAKING NEWS: Once thought to be unbeatable, whatever lead Chatscript may have taken in the chatbot contests over AIML for the last few years, appears to be fading away fast.  At least according to the latest results in the Loebner Prize Competition: Chatscript slips to third place behind two AIML competitors.  Will Chatscript continue to be dominated by other chatbot technologies? What happened to Chatscript?

My interest is in the technology of conversation. If the judges do not wish to converse, it’s not my problem. 

Tom, your post is dangerously close to being considered trolling. I very nearly deleted it, and if this thread goes even a little bit more in that direction, I’ll do so.

Let’s not try to incite ill will, please.

I have no intention of getting into the Chatscript v AIML debate. I have a ton of respect for Bruce. He is an amazing and generous guy. We use different technologies, that is all and I compare this debate to “which is the better language French or German?”. Neither is better, they are both just different. Let’s leave it at that please.

Jinkies, what a sensationalist newspaper imitation. I don’t think results of the Loebner Prize say much about the technology behind it. I still consider Chatscript to be a very clever and handy tool, and all Bruce’s Loebner-participating chatbots have been strong contenders. I trust they will continue to be so in the future.

Besides, you could just as well say that Chatscript won over, what was it, 7 AIML entries?

Don Patrick - Sep 16, 2013:

Jinkies

Too much Scooby Doo as a child?

Actually, I’ve been watching the latest series “Scooby Doo: Mystery Inc” a lot. It gets to you.

Bruce Wilcox - Sep 16, 2013:

My interest is in the technology of conversation. If the judges do not wish to converse, it’s not my problem. 

I agree.  I feel as though the question/answer, question/answer method is too simplistic for measuring one bot over another.  I thought that Chatbot Battles http://www.chatbotbattles.com/ had the right idea.  In addition to the overall winner, a prize was given for the best 15 minute conversation (Bruce Wilcox) and it was also easy to read the transcripts… you didn’t need to decode them.

Any bot can be loaded with batches of answers, but what would truly satisfy the Turing Test, or the Loebner Competition, would be a bot that could converse in a way that the judge couldn’t tell if it was human or not.  In that case, an error here or there would make it seem more human, and “correct” facts would be less important.

When I chat with a bot, I want to come away feeling as though I’ve been sitting on a park bench chatting with someone I’ve just met, not someone who knows five facts about each American president.

The underlying technology is not the key factor in determining a great AI chatbot. It’s the elegance in the application of that technology. And I must agree with Bruce that G.I.G.O. absolutely applies in this situation.

Bruce,

I was wondering if you might answer a few questions for us?

Bruce Wilcox on Head Squeeze - Sep 12, 2013:

Well we’re all friends, but we are using different technologies, so advances made on one side, are not immediately, I mean can’t necessarily be used on the other, immediately anyway.

1. Were advances made on the other side?

Bruce Wilcox on Head Squeeze - Sep 12, 2013:

Rose has more advanced capabilities for understanding meaning, different kinds of things that she can manange so that she can deal with more reduced context because humans chat alot not saying everything because they are assuming you understood what was going on before.

2. How did, “more advanced capabilities” work out for Rose?

Bruce Wilcox on Head Squeeze - Sep 12, 2013:

Well I think her chances are excellent.  She’s the best conversational bot I’ve gotten so far.  So barring unforseen problems, which happen alot in the Loebners, she should do well.

3. Were there any “unforseen problems”?

Thanks Bruce.

Citation: Head Squeeze

I don’t think these questions really need answering by me.

Dave Morton - Sep 16, 2013:

... your post is dangerously close to being considered trolling. I very nearly deleted it, and if this thread goes even a little bit more in that direction, I’ll do so.

Let’s not try to incite ill will, please.

This is just an exciting news scoop stimulating interest and discussion.  Wouldn’t you like to know more about this news story? Thanks for your excellent moderatorship.  No worries, Dave!  Like you, (only probably much less), I helped with Chatscript too, as we can be sure others have.  If the Visual Studio 2008 C++ project, is still there in the latest Chatscript distribution, then that would be my (very minor) contribution.

I really don’t see how the performance of any program could be attributed to the choice of scripting language or programming language. I suggest you consider other factors.

Don Patrick - Sep 16, 2013:

I really don’t see how the performance of any program could be attributed to the choice of scripting language or programming language. I suggest you consider other factors.

OK, I considered your suggestion and I’ve determined that you must be right.  Plain and simple.  For example, there was no such choice for ELIZA except SLIP, a list processing computer language that Joseph Weizenbaum invented!

So, what do you think happened with ChatScript at the Loebner Competition this year?  Wasn’t it startling, to see a leading technology, such as ChatScript, come in third place?

Backing Bruce on this one, no matter how advanced the technology you code a bot on, one could enter “askdkf kasdfk kgtkkag kdsfkkasf” as input and subsequently judge your bot as performing in last place based on your bot not understanding the garbage input. That’s no botmaster or scripting technology fault. That’s a judging fault.  Give me access to any bot and I’ll type my questions in Icelandic and we’ll then see how the bot performs.

It’s difficult to say as the online transcripts are apparently incomplete. But I did notice one round where Rose introduced herself and then seemed to have lost connection (cause unknown, judge 3, ‘nuff said). Also there is the fact that Rose is a fairly new chatbot personality whereas Steve has spent a decade providing Mitsuku with contents, that’s why I expected her to win over everyone else. And Bruce mentioned that judges were less conversational, which is an unpredictable human factor.