I-Programmer article:

http://www.i-programmer.info/news/105-artificial-intelligence/6382-mitsuku-wins-loebner-prize-2013.html

Are the logs for the junior contest recorded any where?

Thanks for all the kind words guys. I am still in a state of shock as I was genuinely expecting to come either 2nd or 3rd, especially as I hadn’t talked to Bruce’s bot and it was very much an unknown quantity. It was my first time in the live final and I was more there for the experience of the day and to learn where I could improve for hopefully a win in 2014.

I think enough has been said about a certain member of the judging panel. All I will add is that I will never understand why he volunteered to be a judge when all he did was to type awful spelling mistakes into each contestant and spent the day announcing to everyone how much he hated chatbots. Mitsuku had the misfortune of meeting him in the first round and after he walked away in disgust after 10 minutes, I felt like going home.

However, the rest of the judges and the junior judges were absolutely fantastic and I cannot praise them highly enough for the work on Saturday. Winning the Loebner Prize really is a dream come true. I still think I am dreaming until I look up and see the big bronze medal looking back at me!

One thing, I don’t think has been mentioned too much is a big congratulations to Tutor for tieing with me in the junior prize. Congrats Ron! I don’t think there was any prize money attached to it this year. I didn’t get any but it is still a great achievement and many congratulations.

Jarrod Torriero - Sep 14, 2013:

I do have to admit that I’m kinda disappointed that even after all this time the Loebner Prize is still being won by fairly basic pattern-matchers, though.

Thanks for the compliment on my win. However, if you think Mitsuku is a “basic pattern matcher”, you couldn’t be further from the truth. I was kind of hoping that seeing as 3 out of the 4 finalists were AIML bots that this constant derision of AIML would have stopped.

I compare it to me sitting at a piano for 10 minutes, figuring out how to play Twinkle Twinkle Little Star with one finger and then denouncing the piano as being no good as a musical instrument, as that was all I personally could do with it. As someone once said to me when we were discussing the same subject, “Blame the artist not the palette”.

Surely now it is time to stop the “AIML is a beginner’s tool” type jibes?

Surely now it is time to stop the “AIML is a beginner’s tool” type jibes?

Basic AIML is a good tool for beginners, because it’s easy to learn a few tags and create a simple bot, without being a computer programmer.  This is like playing “Twinkle, Twinkle Little Star” on the piano. 

But much more sophisticated bots, like Mitsuku, can be created with AIML too.  This is even more true with the advent of AIML 2.0.

https://docs.google.com/document/d/1wNT25hJRyupcG51aO89UcQEiG-HkXRXusukADpFnDs4/pub

Steve Worswick - Sep 16, 2013:

Jarrod Torriero - Sep 14, 2013:

I do have to admit that I’m kinda disappointed that even after all this time the Loebner Prize is still being won by fairly basic pattern-matchers, though.

Thanks for the compliment on my win. However, if you think Mitsuku is a “basic pattern matcher”, you couldn’t be further from the truth. I was kind of hoping that seeing as 3 out of the 4 finalists were AIML bots that this constant derision of AIML would have stopped.

I compare it to me sitting at a piano for 10 minutes, figuring out how to play Twinkle Twinkle Little Star with one finger and then denouncing the piano as being no good as a musical instrument, as that was all I personally could do with it. As someone once said to me when we were discussing the same subject, “Blame the artist not the palette”.

Surely now it is time to stop the “AIML is a beginner’s tool” type jibes?

Jarrod Torriero - Sep 14, 2013:

What I meant is that all the chatbots that do well at the Loebner Prize are front ends to databases in which sentence-sets constitute ontologically basic elements. Such an approach may successfully outcompete the current rivals, but it could never be extended to actually consistently pass properly-conducted Turing tests (an AI-hard problem). To be clear, though, I didn’t mean to undermine the achievements of the finalists, which are of course very impressive.

My original post in this thread was worded terribly and I apologize for any confusion it produced. To be clear, I did not intend to single out AIML (which I do not view as a beginner’s tool) or any other platform. Rather, consider my comment to fall into the same basic category as ‘why don’t we have flying cars yet?!’ type comments. Even so, it didn’t really have any place in this thread, so sorry about that.

Not a problem at all Jarrod. One of the current problems about maintaining a chatbot is managing user expectations. I have had a ton of visitors since the win and many of them are asking questions like “When will the world end” or “is there a God” then being disappointed when Mitsuku doesn’t know. Many people seem to have expectations of these things being all knowing intelligences and we are simply not at that stage yet.

Rather than trying to code an adult human, I would like to see a test where the human confederates are very young children (2 or 3 years old) perhaps having the questions relayed via their parents. Once a bot passes that stage, the move up to say a 6 year old and see if we can pass that. We are still in the infancy of chatbots and I see these as the early pre-Wright brothers type aeroplanes, the flying bedstead and the crazy contraptions that people at the time built before manned flight was perfected. I agree totally with you though and would love to see some kind of sentient machine. I doubt it will happen in our lifetimes though unless there is some secret government project with a team of thousands working on it.

And of course Dr Wallace is correct in that the beauty of AIML is that a beginner can use it but as he said, it can also be used to craft very convincing chatbots, in a similar way as a piano can be used to play a simple tune or a masterpiece by Chopin.

Merlin - Sep 16, 2013:

Are the logs for the junior contest recorded any where?

I haven’t seen them which is a shame as the junior judges were excellent and were talking to the bots like people they had just met.

What sort of music are you into?
Do you like sport?

Rather than than usual style of the adults:

How many plums can I fit a shoe?
Which is bigger, a large lion or a small mountain?

Certainly a difference in the two groups of people. This matches the trend set in Exeter where the school pupils and adult judges behaved in the same manner. This may be due to the academic nature of the 4 judges. All 4 were professors with a great deal of AI experience. It was suggested in a panel discussion after the contest that maybe the judges should just be lay people rather than the highly educated academics.

I think that children are generally more curious, whereas adults think they know everything and are thus critical. The Junior judges looked like keen geniuses, I would very much like more attention spent on that part of the event.
I’ve always been in favour of non-AI judges, and would suggest what Hugh Loebner once said: Journalists. They’re well educated folk but not experts, they are good typers, have varied interests and are willing, curious and critical.

About the “is there a god?” questions, it’s not that people are expecting Mitsuku to know, they’re expecting her to be a joke machine. I had a programmer drop by once who asked my program “2+2” “What is the meaning of life, the universe and everything?” and “Hello world”, and expected the program to play along. If you must bother with it, I imagine an opinion answer would do as well as a knowledge answer.

Weirder still, I’ve noticed some comments of random people who checked Mitsuku out after the contest, some of whom were disappointed that she said something less than perfect after 5 sentences, while they themselves were entering incoherent nonsense from the first sentence. To be frank, they wanted to be disappointed.

Steve Worswick - Sep 16, 2013:

It was suggested in a panel discussion after the contest that maybe the judges should just be lay people rather than the highly educated academics.

That very thing has been mentioned here after the Loebner Competition, almost every year since chatbots.org began. I think, however, that maybe the suggestion should be made to increase the number of judges to 6, with half of the new total being academic/professional experts in AI (as now), while the second half consists of “the man on the street”, with only sufficient training to know how to use the software involved. That said, I’d also be willing to volunteer to be one of the “academic/professional expert” judges.

Steve Worswick - Sep 16, 2013:

Merlin - Sep 16, 2013:

Are the logs for the junior contest recorded any where?

I haven’t seen them which is a shame as the junior judges were excellent and were talking to the bots like people they had just met.

During the last year, Skynet-AI has participated in a number of “Classroom Turing Tests”. This has been interesting since the bot makes no pretense of being human. The classroom excercise (from middle school up to University) use the project as part of a surprisingly wide curriculum (Science, English, Philosophy). The student’s approach this with an open mind and a sense of wonder. I often see conversations where they hit the “Uncanny Valley” (Comments like; Are you real. This is weird. You are freaking me out. Etc.).

I contrast this with the conversations of a university AI Professor, where he has a lot to lose if he can’t tell the difference between a person and a bot. The Ego of some chatters leads to the glass half empty view of chatbots, versus the marveling of a young student who did not know that the technology has come this far and that maybe some day they could create their own AI.

Steve Worswick - Sep 16, 2013:

Rather than trying to code an adult human, I would like to see a test where the human confederates are very young children (2 or 3 years old) perhaps having the questions relayed via their parents. Once a bot passes that stage, the move up to say a 6 year old and see if we can pass that.

Although many chatters expect the bot to have the same capabilities and world view as a University student, I would be a great first step to have a bot consistently have a conversation as a 6-10 year old.

Steve Worswick - Sep 16, 2013:

I like your analogy Steve. We are still in the infancy of chatbots and I see these as the early pre-Wright brothers type aeroplanes, the flying bedstead and the crazy contraptions that people at the time built before manned flight was perfected.

I went to a town fair last weekend where they hosted a car show. All types of makes and models from vintage to modern. I look at chatbots like that; admiring the engine of one, the styling of another… Of course in a drag race I always hope I win.

Richard Wallace - Sep 15, 2013:

Some photos on Flickr

http://www.flickr.com/photos/50366015@N05/9741860832/in/photolist-fQRzPN-fQRzM9-fQRzH3-fQEfN4-fQEfxk-fQGgua-fQGhm6-fQGhug-fQEfqi-fQRzHN-fQEfo4-fQRzFS-fQRzGY-fQEfFF-fQEfGR-fQsm6J-fQsm4W-7UANxZ-ay3kbv-fQRzjY-fQGgZe-fQGgtr-fQRzGm-fQEfj6-fQRzuW-fQRzvN-fQEfrV-fQGgti-fQRzjj-fQEfsz-fQEfCk-fQGh62-fQGfkn-fQRzzY-fQEfr8-fQEfma-fQRzs1-fQGfux-fQEfjP-fQGgqP-fQEfGX-fQEfxc-fQEfsM-fQRzhA-9xeWzC-9wGBpJ-9nyz8M-9wWpUz-7VRC9j-brDYuo-e9sHoe

Nicely done, interesting location and atmosphere.  I have a sudden urge to see a Harry Potter movie.

More press links:

I-Programmer: http://www.i-programmer.info/news/105-artificial-intelligence/6382-mitsuku-wins-loebner-prize-2013.html

BBC: http://www.bbc.co.uk/news/technology-24107790

Slashdot: http://tech.slashdot.org/story/13/09/15/1954209/mitsuku-chatbot-wins-loebner-prize-2013

Goedel (German): http://www.goedel.at/it-news/Einträge/6559-chatbot-mitsuku-gewinnt-loebner-preis-2013

Futurezone (German): http://futurezone.at/digital-life/chatbot-mitsuku-gewinnt-loebner-preis-2013/26.950.435

I found this one as well:

Solidot (Chinese) http://www.solidot.org/story?sid=36464

I enjoyed the comment via Google translate: “it should be the developer discovered his talent lies, please allow me to be a joyful expression”