As a Loebner Contest entrant every year from 2013, I have experienced the original (LPP) and updated (LPP v2) versions of the Loebner Prize Protocol. Both versions of the protocol have proven difficult to use and deterred entrants. I want to suggest a new simpler version of the protocol which I call LPP v3:

1. LPP v3 shall handle any number of simultaneous conversations between entities connected to a single network (LAN). Entities can be judges, confederates or AI’s.

2. Prior to the contest each entity is given a unique name. Names conform to identifier syntax, consist of letters, digits or the underscore character, are case insensitive, and 40 characters or less in length. Examples are JUDGE1, Confederate4, UBERBOT_TEST.

3. There shall be a single shared folder which is readable and writeable by all networked entities. At the start of any contest or pre-contest, entities need two pieces of information: The shared folder path, and the name of the entity they are talking to.

4. To talk, an entity writes a text file entityName~destinationEntity~YYYYMMDD~hhmmss.txt. The content of the text file is the entity’s utterance, using ASCII printable characters only, optionally terminated with CR/LF characters, and limited in size to 32000 characters. Any control characters in the text file are ignored. Optionally we could use the Windows-1252 character set to allow European characters.

5. In the above text file entityName is the entity making the utterance, destinationEntity is the receiving entity, and YYYYMMDD~hhmmss is the date/time stamp as taken from entityName‘s computer. The tilde character is a delimiter which cannot appear in the other sub-strings. Entities can use a simple polling system: after they output their speech, they await a similarly named file from their partner with the latest timestamp. The result will be each entity can talk to its partner, and all conversations flow through the shared folder.

6. For example Judge1 might talk to Frankiebot by sending “Judge1~Frankiebot~20180628~090000.txt” containing the text “Hello”. Frankiebot knows it is talking to Judge1 and picks up this file. It might reply with a file “Frankiebot~Judge1~20180628~090030.txt” containing the text “Hi, I am Frankiebot”. Each round the entity knows to poll for the most recent timestamped file from its partner. Polling should not be done more than once per second as the time stamp only has an accuracy of one second, this also keeps the number of “directory reads” of the shared folder to a reasonable level.

7. There is no requirement for clocks on the entity’s computers to be synchronized, and there are no harmful effects if different entities have different local times. Each speaking entity uses the timestamp from the computer it is running on, so the receiving entity can always find the most recent utterance from its partner by polling for the file with the most recent timestamp embedded in the filename (YYYYMMDD~hhmmss).

8. At no point are any files deleted. This ensures a complete set of transcripts, and a monitoring program could be written that shows the progress of all conversations in real time.

9. By convention the AI’s wait until they are spoken to.

10. It may be objected this approach is inefficient and the shared folder would be a bottleneck. Let’s examine this more carefully. Assume a conversation between one person and one AI, the person types 10 characters per second and takes 10 seconds to compose a 100 character message. Assume the AI composes a 100 character reply in one millisecond. Therefore in 10.001 seconds, 200 bytes are written, 200 bytes are read and two files are created. To extrapolate, conversations between 100 people and 100 AI’s would generate 40k total read/writes and 200 files created in 10 seconds. This is about 4k bytes per second and 20 files per second, which should be well within the reach of modern hardware.

11. I am happy to release this idea to the public domain.

Will Rayer, www.uberbot.ai , 2018-06-28

Wil, I think that this is a wonderful idea, and think it could be a workable one, as well. If/when I get to a point to do so, I’d like to help develop this idea into a fully functional protocol. Please keep me in mind for future involvement.

I am all for such a straightforward setup with text files.

I would however, advise a separate folder for each AI and confederate, to avoid the possibility of interference. Whether caused by overzealous output loops spamming the folder, wanton bugs of all manners, or deliberate sabotage of another entry’s incoming or outgoing messages.
Even if one doesn’t know the names of the other entries’s files, a ‘findfirst()’ call would be able to locate and access another’s files. One could for instance read the other conversations to retrieve information about the judges, confederates and bots to use against them when they encounter them in the next round. One could also access and copy a confederate’s answer if the judge were to have asked the same question to both AI and confederate, or asked a confederate the same question before. This sort of tactic would fit with the likes of Dr. Kevin Warswick who believe that a Turing test should be passed merely by making the results indistinguishable.
Call me paranoid, but I do not think cleverness is beyond participants to a literal game of deception.

The only other problem I can think of is that a file-based system would be harder for javascript entries, though if I’m not mistaken the LPP2’s current bridge involves file interaction. For myself, the setup would be ideal.

I’m not agree for a new change of LPP. We can’t spend time to adapt our programs to a new LPP all two years.
With LPP1, we saw that a shared folder is not a good solution. That files contains a character or a whole message is not changing anything: Remember, there was problems of access right, of refreshing the list of files, and other problems over network.
The LPP2 is not perfect but has also some interesting improvements: messages when rounds are started and ended can be used to launch and stop automatically the bot. Personnally, I would have prefer a protocol like IRC: It is a known protocol with a lot of tools, documentation, and witch can be used on internet. But now, LPP2 is the standard and almost everyone reached to use it, we should keep it.

I have not been a big fan of the file based LPP1. The OS seamed to lock the file system and operated synchronously, this resulted in lost messages.

As a html based JavaScript bot, a file based system is more problematic.
Input at all levels happens asynchronously. You can’t just use the last message, you need all of them since the last was retrieved.

The issues as I see them are:
Lack of developer support (I filed a bug report over a month ago, heard nothing, had to find my own solutions)
There are 2 very different rounds; one is one on one with a judge and should be very simple, the other is with 4 finalists, 4 judges, 4 confederates, and adds webcasting (which can cause problems).
The current LPP 2 is overly complex and poorly supported. Late changes in the server makes it fragile. Changes in the OS, browser, and node, have also created issues.

I would like to think that in 2018 we could devise some kind of online chat system to use.

You know, several years ago, I developed a browser-based chat application called pChat, that had features such as multiple “rooms” (the app was designed to act/look like a house), chatbot integration and chat logging, among other features. /it wouldn’t take much for the app to be adapted to handle hosting and running the Loebner Competition for next year. It could also be reasonably simple for each and every round to be webcast, rather than just the finals. some of the advantages would include:

1.) Completely asynchronous communications
2.) Database-driven logs, with no lost data or deleted files
3.) No filesystem to deal with
4.) Simple HTTP or HTTPS connection, with no sockets to contend with
5.) Easy to use interface for judges, confederates and botmasters

Now I agree with Denis robert that there have been too many changes to the LPP, but I also feel that what we’re currently using is too “not simple”, and too fraught with bugs and quirks to be tenable. I’m not saying that “LPPchat” (if nobody objects to the name ) would be the “perfect” solution, but I think it would be a pretty substantial step forward. I’ll work on putting a Proof of Concept version on GitHub for any interested parties to check out over the next couple of weeks (I’m still dealing with home/family/health issues, so it may take some time).

Hi everyone

For all its problems the old Lpp was simple to understand and relatively simple to program. The new LPP took me ages to get a working solution and I am still not happy with it. A file/directory based solution would be my preference.

I agree with merlin the current LPP2 is way to complex for a relatively simple task and using web browsers and javascript make everything buggy.

We must be able to come up with something we all like.

Dan

I hear many good points being made here, I do have my doubts about the refresh rate(?) of files in a shared folder that Denis pointed out. We should also look at past LPPs (there have been many, Robby Garner also made an online one), and account for their problems.

Another option: What if, just saying, we left the whole transferral-across-a-network-software as the AISB’s responsibility, and that software delivers us these text files (or something else), and picks up ours, so that we only have to focus on reading and producing the files on the chatbot’s computer? Currently there’s a bridge to LPP1, so perhaps there could be a more integrated bridge to the textfile format that Will suggests.

I like Don’s idea about leaving it to the AISB to do the cross network stuff. All we have to do is read and write local text files with the AISB handling the comms back and forth from the bots to the judges etc.

How about this:

Chatbots monitor local files for a message and round starts and ends etc (read only) in;

Message.txt
Control.txt

and write reply’s in (write access);

Reply.txt

Message format for messages is simply;

<messageid>00001</messageid><message>hello world</message>

this ensures that messages are complete and easy to track. (Message id increases in value up after each response)

Any control program can then easily write to Control.txt for round info etc, update Message.txt with message from the judges and read the Chatbot reply’s in Reply.txt.

control text format
<messageid>00001</messageid><message>new round</message>

All you’d need to do is set up shares for each of the chatbots and each of the confederates so that any control server can read and write to the 8 locations.

This would of course require someone to write the control server but its all simple file reading and writing and since only one program is ever writing to one file you shouldn’t get access issues. This system would make it very easy for everyone with all the comms taken care of by the control program.

Dan

The Aisb has always been responsible for network transport. There is limited tome for testing on finals day, and that is when the problems showed up.

We may have to build and test what we want on our own.

For LPP1 we were only responsible for producing output and reading input, for LPP2 we are also responsible for establishing a connection with the server. Though I get it, it’s always preferable to test the whole setup if available.

LPP2 had the following problems last year: (https://www.chatbots.org/ai_zone/viewthread/3136)
Mitsuku: failure of organisers to manually click ‘new round’ and ‘start round’ messages on the controller.
Uberbot: a last-minute change to LPP2’s code (‘content’ instead of ‘contents’)
Rose: a malfunctioning LPP1 output function due to Bruce’s LPP2 interface being lost in the mail.

It seems to me all three problems were unnecessary and not so much inherent technical issues with connection or transmission. The hard part for the entrants is figuring out how to establish a connection in the first place, but as a means of transmission, it seems to work fine.

LPP1’s problems were more often mid-contest issues with timing and delays due to the hardware setup, the operating system, and failure of the organisers to reset bots and judge programs at each round. It would be good to list all past problems of LPP1 as well, and see if e.g. the refreshing of shared folders is still a problem when the judge program isn’t polling every microsecond, and messages are half a minute apart.