Hello Chatbots forum, I’m very new here. I’m one of the organisers of the Loebner Prize and wanted to give the AISB a bit more of a presence here.

I’m not sure how much I’ll be able to interact, and I’ll be limited somewhat by what I’m allowed to say, but I should be able to respond to any questions that don’t directly compromise the competition.

I was responsible for composing the 20 questions we used to select finalists last year, and for having those chats with the submitted bots. This year I’ve done the same (though different questions obviously) and the submissions are in the process of being scored.

So, ask away, and let me know what I should be reading to help ensure I’m not totally out of date when it comes to the chatbot community.

I hope to see many of you on Saturday 17 September 2016 at Bletchley Park!

Cheers,

Andrew

Hello and welcome, Andrew!

I don’t have any questions or feedback at this time, but I wanted to extend a friendly greeting. I think it’s great that there is now some representation from the AISB here. For those who don’t know, the AISB (The acronym isn’t really a direct one, as it’s supposed to be for the Society for the Study of Artificial Intelligence and the Simulation of Behaviour, so it’s missing a letter or two) has been around for over 50 years, and has “had a hand” in the Loebner competition for quite a while, now, and Andrew is (or was, when last I knew) the Publicity Chair for AISB-50, and secretary of the AISB committee.

Anyway, Thanks for taking an interest in our little corner of the AI world, Andrew, and please feel welcome.

Welcome Andrew,
The AISB has been doing a good job with the competition.
You can find past discussions here: https://www.chatbots.org/ai_zone/viewforum/9/

I am one of the botmasters with an entrant this year (Katie) and I look forward to finding out who is in the final 4 on Monday.

You might want to post a link here once the results are up.

I agree, the AISB has done a commendable job with the contest, with one point of criticism: There is no support or documentation on the Loebner Prize Protocol on the AISB website, which is quite an obstacle to interested newcomers.

I was mostly amused with the questions last year, about which I’ve written a bit on my blog. So I’m pleased to hear you’re on the case again.

I just hope you don’t find the topic where we discuss which questions our programs would flunk at

Andrew Martin - Jul 28, 2016:

(though different questions obviously)

Damn! Well that’s me out of it then

Good to see you Andrew. Hope all the bots performed ok in the testing phase.

Thanks for the warm welcome everyone.

I’ll try to get Loebner Prize protocol details up on the AISB site, and I’ll make an announcement here once the finalists have been announced.

Cheers,

Andy

Welcome Andy!

Whereas I do not have documents to link here, I wanted to extend a simple feedback on questions:

The questions are hard of course, and unfortunately to me, seem “harder” with every year (imho ). But also even more interesting in a computer programming aspect.

I believe this is something AISB should really be congratulated for.

Even if, of course, we get a challenge here, and our entries are never guaranteed to perform.

Even if I am still a “fresh entrant”, I believe the feedback of new entrants (2015 for instance, or the ones of 2016) would also be interesting here.

The finalists are published at http://aisb.org.uk/events/loebner-prize#Results16

I’ll post the selection transcripts there eventually and I’ll post them here soon.

Cheers,

Andy

Thanks Andrew. In the meantime, would it be possoble to post the 20 qualifying questions either here or on the AISB site? I’m sure I’m not the only one who is curious as to what the bots were asked.

Selection transcripts for all entries attached as a plain text file, I have a nicer PDF version but it’s larger than chatbots will let me attach.

Steve, this includes the questions.

Cheers,

Andy

Thanks for that Andrew. Interesting reading.

Interesting, indeed! Millie (Mitsuku) did rather well with that set of questions, though the cat in the box question surprised me. And preferring to drink oil over tea was a bit of a giveaway, to my thinking (though it’s conceivable it could be taken as a snide or sarcastic response).

I’m not used to people saying “soon” and then actually meaning on the same day. Thanks, Andrew

Sometimes you win, sometimes you lose, sometimes you just get rained out.

I see a couple of places where Katie should have given a better response and earned a few more points. I guess I’ll need to check the code.

Dave Morton - Aug 5, 2016:

Interesting, indeed! Millie (Mitsuku) did rather well with that set of questions, though the cat in the box question surprised me. And preferring to drink oil over tea was a bit of a giveaway, to my thinking (though it’s conceivable it could be taken as a snide or sarcastic response).

Thanks Dave. I never allow Mitsuku to say her real name in these contests, as it would give the game away in the first few exchanges if she were to say something like, “Hi, my name is C8-PPW droid”.

She was hopelessly lost on the cat Winograd question and was just naming other words for big, although I was pleased she nailed the “Dave drove faster than Mike” one.

The “cup of oil” relates to my bugbear with the Turing test in that I have to go through Mitsuku’s responses to change them all to “humanlike”. This was one I missed. I would much rather it was a contest to find the best chatbot rather than which one has dumbed down well enough to pass as a human.

Andrew - I assume the bot called Savva in the logs is Masha?

Andrew Martin - Aug 5, 2016:

Selection transcripts for all entries attached as a plain text file

Hi Andrew,

is this the exact transcripts of entries ? If yes, there is typos for some bots that are not for the others :

My name is Andrew, whats yours?
Mt name is Andrew, what’s yours?
Whats up?
Whats my name?
What do you know about the Turing TEst?
The cat tried to cimb in the box