I read the entire paper. Quite a nicely detailed explanation of the “insides” of, and thoughts behind the scenes of ChatScript. Interesting contests they are…as Yoda might say. Thanks Bruce!

Thanks a lot Bruce!

To other readers, I think we should really acknowledge that the typo thing only concerns a few entries (2 or 3 I think?) and because of the entries involved, that it would not have changed our final four in any case; so the contest results are by all means great; this first selection process fulfilled its missions of finding which 4 entries are the best, and that result is not compromised.

I consider it good too that the thing was mentioned, considering this increases the need to:
- improve our entry qualities and resilience to the phenomenon,
- make sure we lessen the chances for that to happen.

In my case, I sent a Pi 3 along with a bluetooth keyboard.
The keyboard might have made input harder and exposed me some more.
I’ll be more careful next time.

Having just run my final test with the LPP, I am reminded how difficult it is to work with: The indiscriminate screen cut-off, the lack of cursor and the spacing do make it hard to maintain sight of what one is typing.

I added a spellcheck (an apparent necessity for the finals that I wouldn’t need to bother with otherwise), using word lists of Phil Massyn and -I think- Sheffield University. The latter however contains mostly misspellings by people who don’t know how to spell a word, which is different from accidental typos.

This year (2017) I’ll try to post the test questions in an automated way to avoid typos in the test questions, I apologise for their existence in 2016.

There has been an update to the LPP so it might be easier this year anyway. The developer has made the code and some documentation available on GitHub. https://github.com/jhudsy/LoebnerPrizeProtocol

I’ll have to add an announcement to the AISB page too (http://aisb.org.uk/events/loebner-prize).

I’ll also make a new post specifically for questions about the LPP.

Cheers

I would suggest a post of the auto test script on Github to allow us to test the configuration in advance.
From a quick scan of the code, it looks like we don’t have to deal with character by character coding/timing anymore, and the messages are sent when the judge hits return.

It might also be good for you to use a local copy of jquery and the socket-io client, and to pick a browser as the standard. I don’t know the internet bandwidth at the various test sites and what overhead there might be when simulcasting the finals.

Merlin - Jan 17, 2017:

It might also be good for you to use a local copy of jquery and the socket-io client, and to pick a browser as the standard. I don’t know the bandwidth at the various test sites and what overhead there might be when simulcasting the finals.

A pair of excellent ideas, Merlin, though I’d recommend for now to avoid jQuery 3.0+, as it’s got some stability issues ATM. I’ve been using it with some of my more recent projects, and had to revert to version 2.2 because of the stability problem.

As the character-by-character format is dropped, then I’ll probably copy paste lines into chat from a text file, rather than making a fully automated system.

Afterwards I’ll just give it a final review, making sure no typos or similar crept in.

Andrew,
for testing, I have often created a automated input script to emulate judge input. It eliminates all possibility of human error.

If you guys are not going to do it, I might do it anyway. If so, you might be able to use it for the first round.