I thought the deal with the blanket entry of pandorabots depended on each entry being separate from ALICE. The online version of Izar definitely has some ALICE code in there. With only 4 finalist spots available, I don’t think it’s fair that the AIML developers not only get a chance for entry, but they have somebody else write the Loebner protocol interface for them that they all share.

I mean no disrespect for them individually, but as a group I feel that they enjoy an unfair advantage.

Robby.

I think that the term “unfair advantage” is a bit strong, and not entirely accurate, Robby. Yes, Izar probably has some “ALICE clone” responses. That’s understandable, given that the original ALICE AIML set had literally thousands of “identifiers” that are nearly impossible to weed out (trust me on this. I’ve been weeding them out of Morti’s responses for many years now, and still find one from time to time ). When asked the same one thousand questions, you and I are inevitably going to give the same responses to at least a few of them, so does that make us clones? I think not. As to the LP interface, I don’t have enough information to determine whether that constitutes an unfair advantage. If the interface is only available to Pandorabots botmasters, then perhaps you could be correct, but even then, it’s only a minor advantage, as virtually every botmaster who entered the competition is fluent enough in their preferred programming/scripting language to create their own interface, given the freely available specifications and judging script for the Loebner Prize.

By the way, I’ve split this part of the thread into another topic, so as not to get the original thread off topic, and because this really has nothing to do with the meeting being discussed.

For the avoidance of doubt, Uberbot has no connection whatsoever with Pandorabots. Also I coded my own client program, which handles direct client input and the LPP.

As a developer who continues to lose his placing to majorily AIML-based chatbots, it is my opinion that Pandorabots does provide an initial advantage, but not an unfair one. They just build on existing technology. Comparably I employ code libraries that programmers before me have crafted, and others use ontologies and NLP parsers that are the result of decades of work by large corporations or universities, not to the developer’s own merit. Alice as I know her is a chatbot that fails to be relevant 50% of the time, it therefore takes developer effort to create a good entry from that. Although it continues to elude me how Izar manages to score so high when it doesn’t seem to be doing anything special technically, it is clear that much effort has gone into its elaborate and quirky responses.

The unfairness of swamping the Loebner Prize with AIML entries ended when the contest lifted the restrictions on the amount of entries. Theoretically it doesn’t matter how many there are: if they are all mediocre then one better A.I. system can overtake them all. That this is rarely the case is in my opinion because chatbot systems are built for and thus best suited to sound human (as most of the responses are custom-written by humans, they will always sound human)

If I recall, it took me two days to program a C++ Loebner Prize Protocol from scratch. The disadvantage to newcomers lies in the utter obscurity of documentation on the LPP, with which this forum helps participants where it can. I dare say the organisation has failed in that this year (by offering no explanation on the website), but Pandorabots has been one of the parties offering guidance to newcomers on this forum.

Yes, Izar does contain some Legacy AIML responses, and Dave is correct when he indicated that if two people are asked the same question they may respond with the same exact answer.  It simply doesn’t make any sense to change a response if the response that was present in the original A.L.I.C.E. responses doesn’t need changed.  For example, for the question “What is Impeachment,” Dr. Wallace had A.L.I.C.E. respond “Charge against a public official of improper conduct.”  Izar retains this response. 

As far as having an “unfair advantage” for using parts Dr. Wallace’s excellent work, feel free to give your bot the same “unfair advantage” by incorporating the A.L.I.C.E. responses into your own bot as Dr. Wallace has made the canned responses public domain https://code.google.com/p/aiml-en-us-foundation-alice/ if you feel this would provide you a better bot.  I’m curious why you haven’t taken a few hours of work to do this for your own bot if you feel the A.L.I.C.E. foundation responses provides such an advantage?

It is very interesting for me to follow your thoughts and experience on that subject.

But know that myself am not using AIML, and even if I am a Loebner contestant like any other (ie not even in the Final Four), I do not think myself at a disadvantage.

The reason behind it is simple:
Current entries and technologies haven’t convinced any judge that they were humans.
So the game is still open

I would naively propose the following:
Aim at the silver medal (the bronze is only for “midterm achievement”), and consider advantages or disadvantages when looking from that point of view. I consider it to be the point of the Loebner Contest.

We know we must refine things to get the thing done.
So the raw status we get by using this or this technology would not be as relevant, I think, as “what we change to it” to get to the silver. From that conclusion, I understand chances are good even for non-AIML contestants.

Edit:
Wanted to add that I find it awesome that Dr Wallace made that available.
During the last Loebner Live event, there were talks about whether the competition should be “competitive” or “more collaborative”.

I believe more collaborative, would mean less individual incentives, in exchange of more scientific progress. Anyway, I like that kind of thoughts, and what Dr Wallace did that helped people go forward.

We currently benefit both from the individual incentives *and* the collaborative tools. Sounds like good to me.

Don Patrick - Nov 16, 2014:

Although it continues to elude me how Izar manages to score so high when it doesn’t seem to be doing anything special technically, it is clear that much effort has gone into its elaborate and quirky responses.

For the record, Izar competes in all contests in a neutered form using far less than 1% of its brain using strictly AIML responses using code through or compiled from Pandorabots.  The majority of my coding efforts are only available through my on-line version app called Back Talk available exclusively on Android devices. This interface to Izar can answer literally any question and stays relevant by searching through and parsing a dozen knowledgebases, image searches, etc.  I look forward to the day when our bots can compete on-line and then we really would have an unfair advantage!

Christophe Finas - Nov 16, 2014:

It is very interesting for me to follow your thoughts and experience on that subject.

Wanted to add that I find it awesome that Dr Wallace made that available.
During the last Loebner Live event, there were talks about whether the competition should be “competitive” or “more collaborative”.

I believe more collaborative, would mean less individual incentives, in exchange of more scientific progress. Anyway, I like that kind of thoughts, and what Dr Wallace did that helped people go forward.

We currently benefit both from the individual incentives *and* the collaborative tools. Sounds like good to me.

Yes, some interesting remarks to my initial post. There is a difference between collaboration and going to the grocery store to buy cheese.  If I collaborate with someone and we both have ideas and share our efforts, perhaps we can make an advance that either one of us could have achieved on our own.

However, if you rely on using all the same parts as everyone else, and follow a recipe of packages to accomplish basically the same as everyone else in your development community, then I think I misspoke earlier.

It is a disadvantage if every bot has the same dialog, contributed 10+ years ago to a community of people who are not interested in collaborating, so much as taking a little of this and a little of that from other people in the same category or platform. That is a huge disadvantage if you honestly think you are competing for the silver medal.

I will flex backward and give you the advice that you make your bot portray a 13 year old boy with English as a second language. Add some AIML libraries to that and see if that is indeed the right recipe for this contest.

These are just thoughts folks. Again, I don’t mean to insult anybody.

Robby.

My only concern was the network issue that kept splitting the judge’s input into separate messages. I saw no other problems with this year’s contest.

The AIML community is a very sharing one. Dr Wallace created the LPP interface and I bundled it together into a nice easy “double click on this exe file” for the organisers. I run a contest beforehand to select the best AIML bots: http://www.divabot.com as otherwise, all 160,000+ botmasters would fancy their chances and then yes, it would get silly.
I also give out lots of free AIML which contain some of Mitsuku’s tricks for others to learn from.

If we say no bots can be based on ALICE then how far back do we take it? You must write your own operating system?

The only way to make progress is to share.

The subject here is very divisive.

Let’s analyze that closer.
We want unique solutions and attempts.
But we exist in a world of shared technologies.

I think the question Robby asks is he same that you did Steve:
how far back do we take it?

All correct me if I am wrong, I am just trying to see where the debate can reach its constructive point:

Robby puts the red line on the “answer sharing” part.
I don’t think this means that he puts it on the technological part, but on the answer sharing only.

AIML users here already mentioned it was almost impossible to remove these answers which are common in their entries. This is talking about practical means and not the red line subject, but it also adds to the debate. Sorry for giving a summary, maybe it can help. And again, correct me if I am wrong. I have been already making mistakes in my posts so that could just happen again.

Where do I stand myself?

I want unique solutions (meaning: I would intuitively think that it would be the best thing), but given the practical circumstances explained above, I don’t think the rules have yet to outlaw bots who “share answers”. AIML has brought many nice bots to Loebner, and I think all ended up with their own results and overall memory.

No programmer, judge or organizer would want to have the same answers coming from several bots of the final four. We all would be puzzled to see that. We understand people are trying to do it right. We have not seen (yet) any bad usage of the “shared memory components”, although I may be missing some insight about this.

We could just expect people to try to make their entries as unique as possible.
A gentleman rule. Maybe there needs to just emphasize that point.

We are judged on our ability to make machines think and “reproduce human behaviour”, and this would just be a quality note (that we actually all understand, I think).

Robby, what rule did you exactly have in mind?

Maybe I have this confused with the Chatterbox Challenge.  I’ve been pretty much inactive in all the contests with a few exceptions. 

I’m not saying collaboration is bad, just that if you all select the same components, it’s like a formula one race, and comes down to more about the judge’s (driver’s) performance than the bots. That’s all.

I have used AIML since 1995 or so, so I am just as guilty of incorporating ALICE by the grace of Dr. Wallace. Don’t get me wrong. I just think there is still room for an individual programmer’s innovation, here. I think Bruce Wilcox has innovated ChatScript into something that I am very interested in learning about.  He has also made his program open source, and perhaps next year we’ll see more than one CS bot entered.

I am doing other things now. I entered contests in the past, and it is fun, and I encourage everyone to participate fully. The benefits are great.  I’m just an observe now. And it’s easy for me to armchair quarterback everything, but it all comes down to the participants who make it happen.

Congratulations to all who entered this year.  I think there is progress here, it is not just a show, or a photo op.

Best,

Robby.

I have to say I had a great day. The Loebner was well run by Dr Keedwell, the AISB and the university of Bedfordshire. It was a real pleasure to be there. Many thanks to them all not forgetting Hugh.

I don’t have a problem with lots of people using AIML in their bots. As Steve says the best way to progress the field is to share. Even if lots of bots use AIML and that makes it harder for different technologies to get a chance to be in the competition, so be it. AIML and Chatscript are both fine AI engines and lots can be learnt from both of them. The only issue I saw at the event seemed to be the Loebner protocol had delays where sentence would arrive in chunks.

“my name is fred what is your name how are”

“you today? did you have a “

“good day.”

If a bot gets this sort of thing in chunks it makes it very difficult to make sensible answers.

AI is a complex field, I wonder how the finalists would feel about preparing an article explaining a little about how their bots work and how they prepared for the competition and what is new this year?
Maybe the AISB organisation could publish it?

Dan

Daniel Burke - Nov 17, 2014:

I wonder how the finalists would feel about preparing an article explaining a little about how their bots work and how they prepared for the competition and what is new this year?
Maybe the AISB organisation could publish it?

I’ve just had an email from the AISB suggesting something similar.

Brian Rigsby - Nov 16, 2014:

This interface to Izar can answer literally any question and stays relevant by searching through and parsing a dozen knowledgebases, image searches, etc.

That’s interesting. But even with just AIML responses Izar has been getting top places. Still makes me wonder what your secret is

I’m not sure I would call access to Cyc/Wordnet/google/wikipedia/wolfram alpha etc an unfair advantage either, as I’ve seen other programs do that too. I would however consider it a bit of a sport killer. I’ve seen chatbots who basically answer all questions with “Hang on, let me check Google for that”. At that point I wouldn’t be sure whether I should credit the chatbot author or the AI of Google and Wolfram.

The Divabot contest was only partly to root out Alice-clones, and there were maybe 4 of 25 questions designed to check for default responses. The rest of the questions seemed intended to find chatbots who would do well at the Loebner Prize. Personally I’m glad Pandorabots has this pre-contest, as I think the Loebner Prize would be swamped with a great deal more AIML entries of people just giving it a go and wasting precious time of AI professors. Additionally there is an entry fee for the Divabot contest, that I like to think of as payment for the LPP interface.

The main reason for the entry fee for the Divabot contest is to only encourage botmasters who have spent some time on their work to enter. If we were to accept anybody, I’m sure most of the 160,000+ ALICE clones would fancy a go.