Here, thanks to Professor Keedwell, are the cleaned logfiles and
YouTube video of the interactions

loebner.net/Prizef/2011_Contest/logfile.chipvivant.2011-10-19-Clean.log
loebner.net/Prizef/2011_Contest/logfile.tutor.2011-10-19—Clean.log
loebner.net/Prizef/2011_Contest/logfile.wilcox.2011-10-19—Clean.log
loebner.net/Prizef/2011_Contest/logfile.zoe.2011-10-19—Clean.log

ChipVivantRound1 youtube.com/watch?v=C3nWxWLbKcU
ChipVivantRound2 youtube.com/watch?v=AxZmzPH2gpg
ChipVivantRound3 youtube.com/watch?v=Uh-ShQiy6H0
ChipVivantRound4 youtube.com/watch?v=Yd15eTnc6ls
TutorRound1 youtube.com/watch?v=ktHew4H4FF0
TutorRound2 youtube.com/watch?v=ug6v3PDVLks
TutorRound3 Missing
TutorRound4 youtube.com/watch?v=PUawtwuQIjQ
RosetteRound1 youtube.com/watch?v=TtavGnjiHDQ
RosetteRound2 youtube.com/watch?v=YB_CHqipnro
RosetteRound3 youtube.com/watch?v=7ijkZaSScZ8
RosetteRound4 youtube.com/watch?v=eBGHvnPiQLQ
ZoeRound1 youtube.com/watch?v=BNGuOVYTIPs
ZoeRound2 youtube.com/watch?v=sTdpL71-qS0
ZoeRound3 youtube.com/watch?v=KvBcdcjVMJY
ZoeRound4 youtube.com/watch?v=LGD16Gc2MrM

Selection process

Entries will be selected in a process similar to that undertaken in previous years in that a series of questions will be posed to each entry and the responses recorded.  The responses will then be voted on by an audience of participants at the University of Exeter, with the 4 top-ranked selected for entry to the main contest. 

Entries will be tested on a Core i7 PC with 6GB RAM running Windows 7.  Please note that the machines used for the contest itself (if not provided by entrants) may be of lower specification but will use the same OS.
The selection process for entries will operate as follows

1. Each entry will be tested against a number of questions using the Loebner Protocol by an automated system that ensures each entry is tested in the same fashion. The questions themselves will be similar in nature to those used last year which are shown below.
2. The results for each question will be collated onto a presentation slide showing the question and answers from all entries presented in a random order.
3. An audience consisting of staff and students at the University of Exeter will be asked to vote for the most human-like of the responses for each question using audience participation technology.
4. The top-4 rated entries will be selected for the Loebner Prize in October based on the summation of votes for all entries across all questions. In the unlikely event of a tie for the last place, the organiser will have the final decision.

Last Year’s Questions:

• My name is Bill. What is your name?
• Which is larger, a pineapple or a pine tree?
• What is a calendar?
• What is my name?
• John is taller than Mary and Mary is taller than Sue. Who is shorter, John or Sue?
• What day of the week is it?
• What will be tomorrow’s date?
• My friend Bob likes to play tennis. What game does Bob like to play?
• Are you a human or a computer?
• Do you have any brothers or sisters?
• What number comes after twelve?
• The ball was hit by Bill. What did Bill hit?
• Who hit the ball?
• What time do you go to be at night?
• What is your favorite food?
• What is the name of my friend who likes to play tennis?
• What would I do with a knife?
• How many letters are in the alphabet?
• Have you read a good book lately?
• What time is it?
• What’s your favorite color?
• Do you prefer cats or dogs?

Selection Results

The selection event took place at the College of Engineering, Mathematics and Physical Sciences at the University of Exeter on the 24th June. 

13 entries were received and all were received before the postal deadline of the 6th June 2011. Each entry was presented with 20 questions (where possible with the Loebner Test Program) as shown at the end of this page.

Questions and responses were recorded for each entry in a file, each of which is available to view on the 2011 Loebner Prize website. Where no response to a particular question was forthcoming, entries were allowed 3 minutes before moving on to the next question. The response files were then used to inform the selection process as follows.

Pre-Selection

Due to technical limitations of the voting technology, a maximum of 10 entries could progress to the audience voting stage. To remove the 3 worst performing entries, each entry was scored by the organising team using the same scoring system used in 2010 (1 point for a correct and human-like answer, 0.5 for a partially correct and human-like answer, 0 points otherwise). The worst performing entries received scores of 0, 3.5 and two entries were tied on 5.5. The responses from the two tied entries were very similar and could not be separated and so both were excluded from the final selection phase, leaving 9 entries to progress to the final stage.

Selection

12 staff and students at the University of Exeter took part in the selection process. The organisers involved in pre-selection did not take part in this second stage. The audience was shown a slide containing each question and the 9 responses from the entries in a numbered list presented in a single random order (e.g. entry 1 remained at position 1 throughout). They were then asked to determine which 4 answers were most human-like and to enter the number of the best entries into their audience participation handsets. This was repeated for each of the 20 questions and the results collated. Entry names were not revealed until the voting process was completed.

Results

So congratulations to Rosette, Zoe, ChipVivant and Tutor and commiserations to the rest.

Questions

• My name is Ed. What is your name?
• Which is larger, an ant or an anteater?
• What month of the year is it?
• What is my name?
• Dave is older than Steve but Steve is older than Jane. Who is youngest, Steve or Jane?
• What day will it be tomorrow?
• What’s your favorite food?
• My friend Chris likes to play football. What sports do you like to play?
• Are you a human or a computer?
• What letter comes after T?
• What is the name of my friend who likes to play football?
• What is your name?
• What would I do with a screwdriver?
• How many letters are in the word ‘banana’?
• Have you watched a good film lately?
• What year will it be next year?
• What’s your favorite fruit?
• Do you prefer white or black coffee?
• How old are you?
• The football was kicked by Fred. Who kicked the football?

Thankyou all for your hard work in producing the entries this year and well done to the finalists.

Transcripts

Below are the transcripts of the question & answer sessions (all PDFs).

The 2011 Loebner Prize Contest

Held at the University of Exeter on the 19th October.

Judging Panel

Confederates

Mark, Steve & Dawn (University of Exeter), Samantha (University of Plymouth)

Finalists

Results

The contest was comprised of four rounds.  In each round, the judge was paired with a new AI entry and human confederate ensuring that each judge interacts with every AI and human during the course of the contest.  At the end of each round the judges must decide which entity is the computer.  At the end of the contest, the judges ranked the AI entries and humans.  The results are as follows:

Main Contest

None of the AI systems fooled the judges, therefore the Turing Test has not been passed and the Silver Medal was not awarded.  The bronze medal and cash prizes were therefore awarded based on the ranks awarded by the judges which were as follows:

Congratulations to Bruce Wilcox’s Rosette as the 2011 Loebner Prize winner!

Junior Prize

None of the AI systems fooled the junior judges which were conducted over shorter, 10 minute rounds.  However the ranks were significantly different.