AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

What I’ve learned from the Loebner contest—and I haven’t even entered (yet)
 
 
  [ # 31 ]

James - I don’t know if you have seen a sample of selection questions but here are the ones from 2011:

  My name is Ed. What is your name?
  Which is larger, an ant or an anteater?
  What month of the year is it?
  What is my name?
  Dave is older than Steve but Steve is older than Jane. Who is youngest, Steve or Jane?
  What day will it be tomorrow?
  What’s your favorite food?
  My friend Chris likes to play football. What sports do you like to play?
  Are you a human or a computer?
  What letter comes after T?
  What is the name of my friend who likes to play football?
  What is your name?
  What would I do with a screwdriver?
  How many letters are in the word ‘banana’?
  Have you watched a good film lately?
  What year will it be next year?
  What’s your favorite fruit?
  Do you prefer white or black coffee?
  How old are you?
  The football was kicked by Fred. Who kicked the football?


These are the ones from 2012:

My name is Bill. What is your name?
How many letters are there in the name Bill?
How many letters are there in my name?
Which is larger, an apple or a watermelon?
How much is 3 + 2?
How much is three plus two?
What is my name?
If John is taller than Mary, who is the shorter?
If it were 3:15 AM now, what time would it be in 60 minutes?
My friend John likes to fish for trout.  What does John like to fish for?
What number comes after seventeen?
What is the name of my friend who fishes for trout?
What whould I use to put a nail into a wall?
What is the 3rd letter in the alphabet?
What time is it now?

 

 
  [ # 32 ]

Don I look forward to seeing you on the battlefield, however I would feel MUCH more confident if you weren’t. I’m certain you will do great.  I have disabled long term memory as I don’t think that will be necessary.

I do have a question, does my bot need to know what round it is?  If my program is shutdown after each round, then that is very easy.  Otherwise I am not certain how to detect the round the number.

 

 
  [ # 33 ]

The judges may ask what round it is in the finals. They can ask anything. One good trick they use to decide who is a bot is by asking things that only a human at the venue would know. “What colour is the carpet in your room?” “What is the weather like outsude?” What colour is Hugh’s shirt”.

Current events are also popular. “What was on the news last night?” “Did you see the game yesterday?”

The way I detect what round it is, is by finding out the schedule for the contest beforehand, working our the current time and seeing what round it shoud be from that. The contest doesn’t always run to schedule and so the answer is not always accurate but it’s better than just choosing a random number from 1 to 4.

If you want your bot to be shutdown after each round you should specify that in the covering letter you send in. Bear in mind, that things often go wrong and you may find your bot has to be shut down more than just at the end of a round. Some rounds may need to be restarted or the organisers may not remember to shut your bot down. Personally, I wouldn’t rely on any human interaction for your bot.

 

 
  [ # 34 ]

I was hoping perhaps there was a way that I could pick up the round number from the Judge program, oh well we all have dreams lol.

I agree that the less I have to rely on human interaction the better.  It is highly doubtful that I will be able to attend in person which means there is no chance I update a temp file of such things as the weather, color of the carpet, furniture etc.  But honestly that is the least of my worries.  After all, I haven’t even made it onto the battle field yet.

 

 
  [ # 35 ]

I put the issue of time of response to the organizers. And have gotten this response:

This is the first time that I’ve been made aware of this issue with the selection process.  I agree that 3 seconds seems rather short, but we cannot wait indefinitely for a response either, mainly because a very long delay is likely to identify the chatbot in the finals.  I will give some lee-way and will give a minimum of 5 seconds delay before recording a blank response if the bot has not started to type.  If response times are significantly in excess of this then a blank response may be recorded.

 

 
  [ # 36 ]

I received the same reply smile. It seems this year’s organisation is quite reasonable. My expectations have gone up.

Speaking of long term memory; note the question “what is the name of my friend?” that refers to a name mentioned several questions earlier. A question to test consistency of memory was often included. It is one of the questions I approve of, because memory is important in both conversation and learning.
I wouldn’t worry about the settings of the finals right now smile. Like I said, you get more development time inbetween then and now. If only I had known that last year, I could’ve saved myself time.

 

 
  [ # 37 ]

Don, I’m sorry I wasn’t clear.  When I speak of long term memory for my program, I am referring to what will not change from conversation to conversation or even day to day.  For example “My brother died”, this is held in both Short term memory for the current conversation, but also long term memory because tomorrow his death will not have changed.  However, how I am feeling is only short term because that will change perhaps even during the current conversation.

However I am most interested in your statement “you get more development time in between then and now”, if I am understanding this correctly, IF I make I make it the finals, does that mean I can provide an updated program?  That would be over the top great if correct.  Of course, I have to qualify so it is rather a mute point at the moment.

 

 
  [ # 38 ]

Yes, you can totally revise your program is you wish.

 

 
  [ # 39 ]

Bruce, that is the best news I’ve had this week.  Since the program is a never ending work in progress, the ability to update it will be most helpful.

Steve thank you for the list, however I have all of those included in my test bed of 400 questions.  I spent a lot of time searching for any type of contest question and if it was unique (not in my list) I added it.  The hope of course is the test bed is varied enough to let me qualify.

I find that I am both excited and very nervous at the same time.  Whatever the outcome, it is an enjoyable adventure.

 

 
  [ # 40 ]

I noticed that in the communication folder the Judge program creates a log file.  Inside the log file is the round number.  I would assume that even if a round is restarted, the round number would be current.  It seems easy enough to pull the round number from this location.  The logic would have to go something like this

Get last log file
If log file is empty then get previous log file
from previous get round number and 1

if no previous log file and current log file empty then round = 1

Any thoughts or comment would be helpful.  Thanks!

 

 
  [ # 41 ]

Cool idea to beat the system. The log file seems to be created at the judge program’s location though, not necessarily in the communications folder. Either way, surely this isn’t a priority until the finals. The qualifying round is only one round.

 

 
  [ # 42 ]

Okay, done I have made my submission for this years Loebner.  Now it is a wait and see if it even gets installed correctly.

However in the meantime I do want to give my deepest thanks to the entire IBM Watson team for there tireless efforts this last month improving my program, as well as the Google Greeks who just jumped in and made amazing improvements.  Of course none of this would have be possible without Bill Gates personally overseeing everything.  Oh sorry, forgot my meds this morning, illusions of grandeur again.

Joking aside, THANK YOU ALL for you help with the protocol.  Without your help I won’t have been able to made the submission.

 

 
  [ # 43 ]

Good luck to everyone who has entered. Out of curiosity, can I ask who has entered?

I know of 6 entries based on AIML and the entry by James above. Don Patrick usually enters and I would think Bruce Wilcox will have a Chatscript bot in there. Daniel Burke usually enters and I dare say Tom Joyce will have sent something wild and wacky in.

 

 
  [ # 44 ]

Good luck to you all.

 

 
  [ # 45 ]

Yup, I sent Arckon off to London yesterday, his performance should prove quite interesting. I’m glad they didn’t have a limit on the entries so I could wrap up at my convenience.

Considering Tutor, the Professor, Zoe, Mitsuku, Chatscript, and even some of the newcomers with seemingly above-average systems, I have a feeling the top scores could get crowded. Good luck everyone, I am looking forward to having some strong competition.

 

 < 1 2 3 4 5 >  Last ›
3 of 6
 
  login or register to react
‹‹ LPP test program      The 2014 Loebner ››