I am new to this site, and hope that I am posting in the right forum.  So far I am both amazed and thankful for how friendly and helpful people have been.

I am hoping to enter the Loebner this year, however even with the help of some I’m still having a difficult time with the protocol.  However, to date just getting ready for the Loebner is rewarding on several levels.

1.  The current version of my bot is better than ever before.
2.  The challenge to come up with solutions of “how the heck to answer that one!”, is rewarding when you figure it out.
3.  I have always had respect for those who have won.  However as I prepare for the contest myself I have a better understanding of what everyone faces, and my respect for all has increased.
4. My self confidence improves as the program improves.  Some of the previous Loebner contest questions are tricky (for a computer), but as the program works them out and answers correctly, yes I did that, and it is a really good feeling.

I am well aware of the critics of the Loebner, the “bad judge” from last year for example.  However while he made his point quickly, he offered no solutions (at least that I am aware of).  At least the people who prepare and enter the contest are looking and working toward solutions.  It’s easy to tear down, it is much more difficult to build it.

Technology today is where it is because individuals, like members of this community, stepped out, took the risk, didn’t accept status quo.  And isn’t that what the Loebner contest is really about, stepping out taking a risk to improve the programs.

Because the Loebner simply exist, my program has gotten better.  I hope this starts a dialog of what others have experienced both preparing for and entering the contest.

Jim

Wellllll, there are some good and some bad sides to the Loebner Prize. For me the whole deal of turning my program into a fictional human and compulsive liar is very counterproductive, but because of that I limit preparation to two weeks, in exchange for the benefits:
1. The strong competitors in the Loebner Prize are a challenge, which gives me a temporary but much-needed motivation boost for a month or two.
2. While the Loebner Prize temporarily witholds me from making leaps forward, its lower level of conversation forces me to perfect minor basics of my program’s language skills. And an improvement of the basics is an improvement of the whole.

To me, the Loebner Prize isn’t so much a goal as it is a platform to publicly demonstrate what I’ve made in a formal setting. It is however a double-edged sword, as a satisfying demonstration of my program’s linguistical and intellectual abilities requires intelligent questions to be asked. That didn’t happen last time, so I’ll just have to see what this year brings.

as for getting ready for the protocol…. what OS and what language are you running in?

I’m running Windows 7, and the bot is programed in .net.  I got the judge program semi-working today. I downloaded a loebner client test program and between the two I am starting to get a better idea of what is happening.  I don’t believe what is required is that complex,  just a little difficult to wrap my head around.

Don is right of course, the contest is a double edged sword.  Looking at logs of Siri, logs chatbots and some of the Loebner questions, if appears to me that “intelligent questions” are indeed a rare thing, which is a shame.  At the same time, this can be true of the real world.  Recent I stopped in our local 7 -11 and had a short but interesting conversation with the clerk.  I expressed that we were just returning from visiting the McDonald Observatory, and the reply I received was, “I wouldn’t want to go there, I’d be afraid of learning of what I eat”.  I smiled and left, a bit puzzled and then it dawned on me, this person first confused “McDonald” with McDonalds” and secondly they had no clue what an observatory was.  Thus “intelligent conversation” can be lacking in the real world as well.

The overview of what is happening is easy.
There is a directory. The judge program adds files named by a sequence number, a character or special character word, and the suffix judge.  You get the files and sort by lowest ascii sort of name, decoding only judge files and getting the characters.  You write into the same directory using a sequence id of your own choosing, the character or special character name, and the suffix other.  Sequence ids will rise during a round, both yours and the judges.

Lovely story about MacD.

after you read a filename, you delete it.

The judge program adds files named by a sequence number, a character or special character word, and the suffix judge. 

Doesn’t the judge program add named folders, not named files? I just tested Uberbot with the judge program from last year and I’m sure it’s folders.

Another strange thing I found is the names of the keys seems to be case sensitive. Eg I created a folder ‘return’ and it was ignored by the judge program, but I renamed it to ‘Return’ and it worked.

Yes, the judge program creates folders.  Huge help to know to delete the folder after you read it.  THANKS!!!

Yes, its folders. Sorry for the misspeak.
The names of the special characters ARE case sensitive. This list is at the bottom of a judge program.

What you have to decide is when you have enough input.
I use the rule that if 5 seconds of silence have gone by and I have a . or ? or !, then I run with it.
If I dont, then I run with what I have after 8 seconds of silence.
Unless I have nothing, in which case I remain silent.

Bruce thank you.  I think I am beginning to get a handle on it.  Also if I understand correctly, when my program installs, it should also install Perl, just to be on the safe side.  My program uses .Net 4.5 which the installer normally downloads from Microsoft, but with no internet connection I should have it in my “support” directory in case needed.

The timing is a shame because one of the things I have constantly worked on is the program responding within 1/4 of a second.  That is the “average” human response time in conversations.

From the documentation://

To detect a key press by the judge, the program must detect, within the communications directory a sub-directory with the same format, but extension “.judge” and then must remove or delete the judge’s sub-directory from the communications directory.

I should point out that I accidentally used a wait time of 4 seconds last year (instead of 3), and the judge in the selection round continued with the next question before the program had responded to the first. As a result, the answer to the first question awkwardly ended up in the second, and the first question was judged to be “blank”, therefore received 0 points.

This “wait” time seems both critical and unpredictable.  Don, if I am understanding your correctly, and please correct me if I am wrong, if your program does NOT detect any new key press within 3 seconds, then it is assumed that the Judge has finished entering his question?  It seems that the judges were good about entering “.” and “?”, was this consistent?

By the way THANK YOU, all of you.  I’m actually beginning to feel like I am starting to get a handle on this and that is only because of all of your help.

Now is not the time of course, however after the contest perhaps Hugh would be open to a change in the judges program. Perhaps a “send button” and nothing is sent until pressed.  It could continue to create a folder for each character, but resolve the timing problem.

Correct. After about 3 seconds without receiving keypresses from the judge, my program processes what it’s got. Long enough to allow for a pause for thought mid-sentence, but not long enough to suspect a malfunction. I intended to set it to 3 seconds last year but miscalculated. Optionally you could vary the time depending on the presence of punctuation or sentence structure or supernatural mindreading powers, but sometimes questions span two sentences.
The selection round judge has so far always been civilised enough to use proper punctuation, the judges in the finals not always. But you get some extra development time inbetween the selection and finals.

The testing program may, or may not, terminate the utterances with periods, question marks, or returns.

Should any contest judges read this: Yes, you could deliberately undermine natural timing, but it would make for a pretty dull contest that I couldn’t possibly take serious. Last year’s judge did pick up on the timing after that one blank though.

You’re welcome, and yes, I too hope the judge program will be improved. Enter presses would also solve the HORRIBLE UNREADABILITY of the transcripts and online broadcast. Just sayin’.

Unless your program uses PERL, there is no need to install it just in caee. Your installation only needs to install what your machine needs for your program.

As for timing, that’s strictly an individual choice. You can RESPOND as quickly as you want as the speed of your response is not what will make or break you winning. Deciding when the judge has said enough for you to respond is the stickier issue.  If you get a complete sentence at some point, you might respond to that, but if they were continuing typing, you might want to respond to their full input. EG
“yes. do you like fruit”

RIGHT NOW all this is unimportant. The qualifiers are NOT judges, they are complete sequences of sentences with punctuation. You get potentially multiple sentences in one blow followed by infinitite patience waiting for your reply. Once you qualify, then you have to think about this mess

Bruce, thank you for making my life a bit simpler.  That helps.  My only real concern then is net 4.5 which if required can be pulled from my “support” directory during the install, and the program will do that automatically.

Before posting this thread, I was beginning to lose hope of even entering, thanks you and others I think I will at least be able to enter.  However it goes with my program, I do wish everyone the best of luck in the contest.