Don Patrick - Jul 12, 2014:

P.S. Your final submission won’t require a LAN connection, will it? (Just pointing out the small print)

Your submission will probably have access to a LAN, but not a WAN (Wide Area Network, AKA the Internet). and I suspect that the LAN it might be connected to will almost certainly be secured against “intrusion”, thus preventing contestants having the opportunity to “change the odds”, if you know what I mean.

The setup usually consists of 12 PCs connected via a dedicated hub/switch.
4 PCs for the bots, 4 for the humans and 4 for the judges.

Unlike the Turing Test in London recently, the bots must be running from one of the dedicated PCs or I believe you can use your own equipment with the approval of the organisers. This means that your bot cannot use the internet to answer questions. No links out to Wikipedia and the like.

I was surprised that in the recent Turing Test at London, the bots were hosted remotely rather than on site, as this increases the possibility for cheating. I’m not saying this happened but the temptation must have been very great.

I should have explained, the LAN I use for testing is an ordinary ethernet-based local area network using ethernet cables and a data switch, no wireless or WIFI or anything. I presume Don’s warning is against using any kind of wireless networking or internet connection so I am OK (Uberbot runs on a standalone Windows PC and doesn’t need an internet connection). Anyway I am glad I tested as I found a couple of bugs that would have caused problems. So thanks again for the links, Steve and Don.

A couple more questions:

(1) When running the Judge program, I noticed my bot’s output just ‘runs together’ without any line breaks in the Judge program’s upper window. This is because my bot doesn’t add CR/LF to its output, on the basis the Judge’s input isn’t guaranteed to be terminated with CR/LF so why should I terminate my output? Is this OK? Or should I always terminate my bot’s output with CR/LF, even if the input isn’t so terminated?

(2) When implementing the ‘browse’ function in my Uberbot to find the LPP folder, can we assume drive letters are mapped in Windows? Eg will there be a drive letter permanently mapped to the LPP network folder? Maybe this is a question for Prof Keedwell, but I would be interested in anyone’s thoughts.

1 - It would look better if your output had line breaks. One of the entries in 2012 has the answers posted one after the other without any line breaks and was very difficult to read.

2 - In Ireland last year, the organisers browsed to shares on networked PCs rather than drive letters but yes, I would clarify that with Dr Keedwell.

What I had in mind was that the selection round judge would probably have both the chatbot and the judge program run on the same computer. In fact, I’m counting on it because my “select a folder” window requires a perl interpreter to be on the computer, because I’m not smart enough to know an easy way of doing so in c++. Unfortunately that too was a question Dr. Keedwell did not respond to, so I’m just guessing that whatever worked last year will work again.

If you have no reliable means of telling when the judge is done with his input (sentence analysis, timers…), I guess it’s okay not to output any carriage returns either. From the messy state of previous Loebner transcripts I’d say more bots have done without, nothing in the rules says you have to. It does make it harder to tell where one answer stopped and another began, increasing the risk of human error. Maybe if you output two spaces instead of a return?

Don’t count on that Don. I would assume the selection process setup was the same as the actual contest. What would happen if a bot worked great while the judge/chatbot program was on one machine but failed to run on the actual day because they were on 2 PCs? It would be a no show and the contest would be 1 bot down.

If you don’t know when the judge has finished typing, how do you pass his input to the bot? Most people tend to go for something like “if the judge hasn’t pressed a key for 3 or 4 seconds, assume he has finished typing and process what he has typed so far”.

I’m only counting on the perl interpreter being installed, or being installable, on the computer that runs my program. If not, then I am at the mercy of the organiser to read my instructions containing 4 fallback methods of hooking my program up to the communications folder, or the judge program to my program’s default communications folder.
The finals would have me on site (if luck would have it), which is always a fix to everything. I appreciate the concerns though, I’ll look into something better if I can spare the time.

Okay, I found a few minutes time, copied some c++ code, and that seems to do the trick too. Thanks for the encouragement

Glad to hear it Don. I was just checking the rules:

Entries must be self-contained and not require installation of third party software or libraries.

The testing/contest PCs are usually base build Windows 7 with nothing else on them.

Steve - based on what you said about the carriage returns, I think I will add them to the Uberbot’s output to make it easier to read. It seems there is no adverse effect in including them so I will.

Also that’s an interesting point about knowing when a judge is finished with their input. In the Uberbot I allow a choice of (1) wait for a return character or (2) wait for a pause of N seconds where N defaults to three seconds. The input method defaults to method (2) as from reading this forum you can’t rely on return being pressed.

From what Don said a third possible method comes to light. If your bot can process its input and generate output in near real-time, you could process the input every second or so. When the input hasn’t changed for a second or more *and the output looks sensible* you could assume that’s the end of the judge’s input for the current question. This would be a very good approach. Unfortunately the current state of my AIML interpreter doesn’t allow such luxury!

Steve - presumably “Entries must be self-contained and not require installation of third party software or libraries” permits third party software if it’s included with the entry as part of the entry’s installation? Otherwise that would mean you couldn’t have entries needing PHP, Perl, SQL or anything else?

I interpret the rules Steve points out as “As long as all we have to do is press one button”, so no separate manual installations of “product X” next to the installation sequence of your chatbot. But in last year’s case I figured they already had the perl software installed because they couldn’t run the judge program without it anyway. Plus I had provided a superfluous copy, plus these are AI professors we’re dealing with. They may be very busy people but one can expect them to manage two-step instructions. Anyway, my troubles are solved.

We also discussed a few possibilities for auto-entering input here:
http://www.chatbots.org/ai_zone/viewthread/1620/P38/

Indeed. If your bot needs SQL, PERL, Java or anything else apart from what comes on a base build of Windows 7, it must be installed via your setup file with no interaction from the organisers. The organisers will not follow instructions such as: “Now set up SQL using the parameters below” or suchlike.

Be wary of assuming anything Don. In 2012, I think around 9 of the 14 entries were not tested because the organisers had trouble following the instructions to load/run them. Hence the new rule of “we are just going to double click on one file. It’s up to you to get it working”.

The waiting of 1 second if the input looks sensible could be dangerous. Imagine someone typing, “My name is Steve” then pausing for a second before continuing “and I am from the UK”. The bot will make sense of the first part but totally ignore the second.

Thanks Steve - so I understand the general idea is you can install what you want, but you have to supply it and install it as part of the bot’s setup, and not expect the organizers to install it for you. That makes sense actually, otherwise it could take them ages.

I feel more comforted now! All I have to do is include SQL server in my installer now (which is causing me some difficulty!)

The idea of ‘continual evaluation every second until it looks right’ was really just a thought experiment. I know it wouldn’t be practical for my bot. Maybe other bots using different technology could use it.

Predicting the judge’s behaviour is half the fun though, Steve
The massive failing to run 9 of 14 entries was my first impression of the organisation involved and had me quite worried the first time I entered. I only learned later that the problematic entries all operated on the same system. Is that correct?

I hinted to the option of continuously examining the input mostly because I have the impression that Hugh Loebner and others expect this from AI. In my own opinion, that ability is overrated. Basing the timing of one’s reply on when a sentence seems “complete” is also potentially in conflict with good manners, so I don’t even do that as a person.