Firstly, let me confirm the final of the 2018 Loebner Prize will be in the expected format, 4 half hour rounds with 4 finalists, 4 confederates and 4 judges. LPP2 will be used.

Having said that, I should also clarify that I don’t think we’ll be able to run the Loebner Prize as it is currently understood any more. This isn’t the end and it may be a good thing leading to a much stronger and more active community.

The main problems are thus:

1) Hugh Loebner promised to update his will to leave his estate to the AISB purely for the purpose of running the competition, that will was never located. No one in the AISB suspects malice on Hugh’s part.
2) The competition would always have benefited from sponsorship, but now it absolutely requires it, and the simple fact is that the competition in its current state, and current location, is a difficult sell to sponsors.
3) Certain aspects of the competition take a lot of time to organise and exclude many people for reasons that are not relevant, for example, the size limits, offline-only rule, and the “20 questions” selection process.
4) The “finals” day also ensures only a small number of the submissions get to be demonstrated in public, where a more open structure would reward relative newcomers, and diversify the crowd.

There may be more but I’m writing this off the cuff as I feel this discussion should have started earlier.

Everyone in the Loebner Prize organising committee respects and values the regular entrants, and the chatbots.org community, so we want to keep you all involved and content with the way we take the competition. Let me briefly describe my vision for 2019.

I would like for the AISB to make an open invitation to all software and robotics teams to submit anything which behaves in a quintessentially human way, for example: a chatbot, poetry generator, graphic artist, music composer, imitation-game player. All submissions which meet a quite attainable quality standard will be displayed and demonstrated in an exhibition somewhere more accessible than Bletchley Park, space permitting. Funds will be available to help exhibitors attend with their submission, e.g. travel awards, and computers for pure software submissions. A panel of judges will be selected each year from industry, academia and the media to award a number of prizes to the submissions they deem to be of a particularly high standard, I don’t know what the prizes will be but we can guarantee that one of the prizes will be “The Loebner Prize” for best chatbot.

There is nothing stopping us doing the conventional judge-chatbot-confederate conversations at the exhibition, and as it will be more of a collaborative event we can be much less formal in the selection, security, and restrictions, on chatbots.

This will turn the current stressful finals day into a more of a celebration, where we will not be limited to demonstrating only the best four chatbots, where more of the community will be able to attend and will have more reason to attend, and there will be greater exposure to the public and media.

I hope you agree this is a good direction to go in the face of an existential crisis of the Loebner Prize. I’m comforted by the fact that I know the community here will give me their honest opinion on the matter.

I personally (and I think I can speak for everyone here) would LOVE to see The Loebner Prize continue in one form or another. If that means selling the name to a sponsor and turning it into something like The Walmart Prize then so be it.

I agree that it needs wider publicity than the current arrangement at Bletchley Park to make it attractive to a sponsor. The history of Alan Turing and the place itself makes it an ideal location but only for that reason. Most of the visitors to the Loebner Prize room just happen to have wandered in there by chance and have no clue (or interest) as to what is going on. When I’ve been there, I’ve seen that a tourist group of old women just aren’t going to care about humanlike conversational AI and prefer to reminisce about the war instead. However, when tech-savvy children visit, they seem to love talking with the demo bots and watching the contest.

Holding it as part of a larger science or tech event would make perfect sense to me. A dedicated, knowledgeable audience would attract more publicity, more sponsorship and even a slot on TV. I’m sure Channel 4 in the UK would love this sort of quirky contest, similar to “Child Genius” or any other of their niche reality shows. Have you seen some of the e-sports contests? They play to packed out arenas. I’m not saying we could hope for those numbers but I would confidently guess that a few hundred would be interested in watching it at the event if they knew about it.

Personally, I dislike the “pretending to be human” element of the Loebner Prize and feel it would be a better use of our time to run a “best chatbot” contest instead. Let’s face it, no chatbot is going to convince a reasonable judge that it’s a human over a mammoth 25 minute session and it seems strange to try and make humanlike chatbots when they usually give themselves away after the first 4 or 5 interactions.

Let the judges talk to each bot and vote for which they liked best. This will do away with the prohibitive LPP protocols which prevent many people from entering quality chatbots. No Bruce this year and only 11 entries :( Let’s just allow the developers to make their own interfaces for 1 judge to chat to 1 bot. No need for human confederates and sure, I think we are all trustworthy enough to submit online bots without the temptation of cheating. You could even have some kind of award for best interface as part of the event.

I also agree that we shouldn’t limit the contest to just 4 finalists, as it’s pretty much the same old faces each year. Let them all take part, assuming they pass a basic level of competence. Anyone remember pi bot from a few years back? This just makes the chatbot scene look bad and so I would still advise a basic screening set of questions first.

The possibilities for this event are endless, as the contest in its current form has become a tad stale and it’s good to get the ball rolling to try to elevate it to the heights it deserves. I’m sure a good marketing team could have this very unique event at the forefront of the tech world.

One final thought is that it would be good to keep Hugh’s memory somewhere in the event if at all possible. This is his legacy and without him, none of this would have ever taken off in the first place. Perhaps keeping his image on the awards or projecting his picture on some display screen or even an announcement at the start of the event would be a fitting tribute?

I can’t say this is unexpected.

1) Agatha Christie sends her regards.

2) I’m going to point out another reason why I suspect the Loebner Prize is a hard pitch: It is my understanding that there is something of an unofficial boycott in academic circles, in part because it’s too far out of reach of the state of the art and thus serves little practical purpose but to give media an opportunity to wail about how bad A.I. is. A tale of failure does not reflect well on the sponsor, and the end result of strictly human-or-not Turing Tests is always failure. It would take a change in judging criteria to amend that, rewarding programs for how good they are at certain things instead of how bad they are at being human. There have been suggestions in the past of granting various awards or titles for the most conversational, or funniest, or most technically capable programs. One may question how it is still a Turing Test if it has different criteria, but I think it is quite in the spirit of Alan Turing’s diverse arguments that the Imitation Game served to illustrate.

3/4) More than 4 finalists/exhibitors would be more than welcome if some minimal quality is observed. A more open exhibition would certainly serve my primary interest of demonstrating my program’s capabilities. Opening the contest back up to the internet would double the entries with the return of Cleverbot and the like. Especially if you want academic students on board, and if you want to include poetry generation, one can’t do properly without access to online resources. While I don’t have as much faith in people not to cheat, I suspect it won’t be a great issue as long as the monetary incentive remains modest.

You might want to get in touch with Dartmouth, they’ve been holding poetry generation and other art-related ‘Turing Tests’ for some years now. They suffer from a low amount of entries which I believe is in part due to stipulating the submission of source code, which mostly attracts student entries who’ve whipped up some neural nets in a few weeks time, but it seems to me there’s opportunity to join forces.

Lastly, just speaking for myself, the kind of event that I will not enter is one where I have to put great effort into aspects perpendicular to my research (i.e. if I have to make my program pretend to be human), without at least a chance of covering the lost time (read: rent) with prize money. I’ve been struggling to come up with excuses to keep entering the Loebner Prize as it was.

As for possible online cheating, I think the speed of responses will come into play here. Rather than having to currently fake typing speeds with the new LPP or deliberate typos, backspacing etc with the old LPP, we should get the bots to reply as though they are bots i.e. quick responses to input.

Any chatbots that take a few seconds to type a simple message may indicate foul play.

It is high time the contest was opened up to chatbots in general and not just ones that try to fool the user into thinking they are human. While it is true that academics frown upon the lowly chatbot, the contest/showcase does not have to be an academically sanctioned even- more like a ‘Chatbot Showcase /Convention’ than a ‘Chatbot Symposium’.