If you would be interested in participating in an on-line chatbot contest please respond.

Right now there is no activity and no prizes, I am trying to determine interest to see if it is worth my effort.

In the past, I created some tech that may make this easier, including:
- automatic submission of input to a bot
- Natural Language generation of input
- automatic scoring of answers

If everything was automated, it may be possible to run multiple contests per year.

Yes, I would.

I would be very interested in a online chatbot contest. The goal is not to win prizes or rewards, but rather to know the value of each bot compared to others, and to know when it progress or when it regress.
The disappointment caused by the Loebner prize this year (I did not participate and I do not regret it) and more generally in recent years makes more interesting to organize an alternative competition.

I would be interested in any type of contest, but if it can serve I had imagined a protocol for this type of contest. Of course, it is perfectible but here are the main lines:
- Create a website of dialogue (chat like IRC) on which any user can connect and interact with another entity. Either permanently, or over a shorter period of time (on a weekend per month, for example).
- Botmasters register their chatbots and have interfaces to connect their chatbot to the site.
- When the contest is launched, each conversation starts every half hour and takes 25 minutes.
- Each user is randomly connected either to another user or to a chatbot.
- The user has two buttons, “Human” and “Robot”. As soon as he identifies his interlocutor as a human or as a robot, he can give his choice. At the end of the conversation, he can give a note on the quality of the conversation.
- The site automatically counts the notes and the average time during which each chatbot gave the illusion of being human, and automatically ranks.

Advantages: a fully automatic procedure. No judges, no place reservation, no travel for botmasters.
Disadvantages: like any bot connected to the internet, there is the possibility of cheating, for example by putting a human (or humans) in place of the bot.

I know you were hoping for a better robot to join the contest, but I’d be willing to join. Although I might not be able to. The most reliable way of talking to my robot is on pandorabots.com, so people would have to sign up for a free account with them to talk to it. Technically I have a website were people could talk to it but I would get charged after so many interactions with the robot.

I think an online chatbot contest would be neat if this thread becomes a hot topic. I don’t know how popular chatbots are in the U.K but this website seems to get a lot of traffic so it’s good publicity for online chatbots.

I think the simpliest way is for there to be a thread for chatbot creators to ask other chatbot creators to test there robots. The idea being someone would test someone else’s robot with the idea that their robot would be tested. After a chatbot creator tests the robot they record their conversation with obs or whatever screen recorder theyre comfortable with using. Then a youtube link of the recorded logs is posted to the forum with the address of the online robot.
this way there is at least some level of authenticity. If the web address shows the site the robot is on, this reduces the likelihood that the entire system is just patronage. Admittedly, that doesn’t mean that chatbot creators wouldn’t give softball dialogue so that their robot would also be thrown softball dialogue that also make it look good. Unfortunately with an online contest I have no idea of a way of actually making it fair. The very idea of making it over the internet means it’s possible that the responses are not actually coming from the robot.

However, since we could have this contest on forums. In other words it wouldn’t really be official. People could say what they liked and didn’t like about the video.  People could also question just about any of the robot responses. For example steve saw responses from Sophia the robot that looked suspicious and he was able to explain why those responses couldn’t have been coming from their robot. Basically what I’m saying with a contest like this the only thing you can prove is that the response came from your robot and that it could have come from the pattern that was asked to the robot. you can do this by showing source code. this is the only thing that could be proven. Unfortunately, it is impossible to prove that something couldn’t have gotten hacked online. That is always possible. I was just spitballing maybe my way of doing this is a bad idea.

I’d only be interested if there were prizes or if it were a big event publicity-wise (i.e. if you can get entire internet communities to judge). Getting my program to connect to the internet is a possible but clumsy affair, and I’m aware enough of my program’s progress without external judges. Generally speaking however, I think the Loebner Prize for instance should move online, this being the 21st century with social media taking over the role of news outlets.

The one problem with online contests is cheating, which I consider a real possibility if there were something at stake (if not, then not), so I’m just going to leave some ideas about that:
- Instant response times could indicate that an entry is a chatbot. However, not all programs are simple pattern matchers with instant response times. IBM’s Watson needs a second or two, my program takes several seconds going through its database at especially broad-range questions, and internet connections may introduce lag.
- Chatbots are technically capable of engaging multiple users at once and don’t need sleep. One might arrange that 5 to 10 judges interact simultaneously at an unannounced time somewhere within 48 hours, or that the chatbot must be operational 24/7 for a week. This would make it unlikely that an entrant would gather a group of friends to sit down and pretend to be their chatbot.
- If a winning chatbot shows uncanny human-like ability, one might stipulate that the winner either reveals their methods to the point of credibility, or repeat the feat in real life without internet connection before the outcome is validated.

Yes, I would be happy to take part in an online contest. The attraction of the Loebner Prize was always the Turing Test element of it and I feel this is also a reason why it wasn’t as popular this year. I’m not sure how you could run an online Turing test with many bots but it may be worth someone more technically minded than me to look into.

Assuming there’s no Turing Test in the new competition, can we at least make it so the bot doesn’t have to pretend to be human please?

Interested in Online Contest? Yes!

I will leave other comments for if/when this comes to fruition other than to say I think “cheating” would be less of an issue if the bot is not required to pretend to be human.

One of my favorites was:

http://www.chatbotbattles.com

Merlin,

As a former judge who worked for Hugh, I politely vote, “No.”
I am in favor of the contest, which Steve just won, getting
more support from us all, so it remains in operation.

However Merlin, discussing chatbot contest technology
is an extremely interesting topic.  So, I will be following
your progress on this.

Online, I have a working prototype chatbot contest forum. 
It has forumbot contestants and a forumbot judge.

I would be interested too and have suggestions.
Bot main streams such as Alexa, Siri or Cortana could also be involved, as they are always available, in order to have a general overview of the performance of the various systems.
The ideal would be for the judge to be software, so that his behavior is the same for everyone.
The test should be divided into distinct phases so that they cannot be intercepted by someone to improperly improve their chatbot during the competition.
But first of all we need to eliminate the fake emulation of the machine that tries to appear human.
We need a new test, yes, but not by Touring, an artificial intelligence test and not an artificial fiction test.
But this last point will be the most difficult to set up.

It would be nice if the chatbot was required to CHAT…to carry on an engaging, coherent or topical conversation about a variety of subjects without having to do complex math or name populations of some countries. Anything outside of ordinary human abilities should not come into play. It should first and foremost, CHAT. Just my $.02

I agree Art, which is why I made free flowing conversation part of the contest at http://www.chatbotbattles.com/ when I organised it a few years back.

It’s amazing how many people write to me saying something like, “Hey Mitsuku isn’t all that humanlike. I asked it the fuel capacity of a Saturn V rocket and it didn’t know.” I guess that makes me not humanlike either!

There are just to few botmasters interested to make dedicating time to a contest worthwhile.

I continue to believe that we may all be better served with a standard repository of answers to factoid questions rather than having each botmaster trying to duplicate and embed it themselves.

In the past, I suggested smaller contests focusing on a subset of topics (math, time, personality, etc.) but not too many people were interested.

Skynet-AI has seen both bot to bot and bot to human conversations, but these are often much longer than you would see in a typical contest. Bot to bot connections have their own issues since it can swamp the logs and depending on the learning mechanisms, could make both bots very similar.

Spoiler Alert!
Spoiler Alert!
Spoiler Alert!
.
.
.
.
.

Merlin, your A.I. never happened!

The result of an alternate timeline.