Haris Lodhi - Feb 14, 2013:

But, if a judge asks about the prime minister of XYZ country, that should be known to most of the people on web and it would be a valid question then!

I would have to disagree with this statement. If you asked me to name the prime minister of Canada, I would struggle without having to double check it on the internet.

We have already tested all the participants and the results are in the phase of re-evaluation and compilation for avoiding any mistakes or discrepancies and just to make sure that there are no issues with our calculation as well.

Is this for the final round? I have had no visitors to my bot asking any judge type questions and have been doubly sure to check after the first round. I get an email sent to me whenever anyone accesses the link I sent for the competition and apart from a few people (I assume other entrants) generally chatting to it, nobody has asked it any style of questions posted on your website.

Dear Mr Lodhi,

I agree that “Who is the Prime Minister of [xyz]” is a question that would be handled by any bot which has an external connection to most online KB(s) and should be able to handled. RICH produced this;

What’s your name:hi
Vince: Hello
What’s your name:who is prime minister britain
Vince: Sorry, I couldn’t think of that right now. I’ll get back to you.
What’s your name:who is prime minister of britain
Vince: According to Internet sources, The Prime Minister of the United Kingdom of Great Britain and Northern Ireland is the head of Her Majesty’s Government in the United Kingdom.

Slightly obtuse, but valid

Your question produced this;

What’s your name:who will be the next prime minister
Vince: I wasnt sure of exactly what to say. . . ” The King of the Frogfishes. Hi! How are you doing today?  ” , sometimes I fall back on things that Ive heard elsewhere.

“Who will be the next Prime Minister?” implies that the Bot and the Judge are both residing in or somehow connected to a place that they both are aware of without it being named . We refer to this as “world view”. Mitsuku and Steve have a different localized world view than RICH and I being in Great Britain whereas we are located in the USA. If you asked me “What did you think of the Presidents speech”, I would know by inference that you meant the “State of the Union” speech given by the President of the United States. One is asking for fact, the other is asking for opinion based on localized world view. The fact that Steve was able to trace the IP back to someone located in Great Britain would seem to bear this out.

Vincent Gilbert

I am 100% sure these questions didn’t come from the RoboChatChallenge, as they were asked after the results were announced to a bot that I didn’t submit as an entry, the user was based in the UK and I doubt anyone knows much about One Direction apart from British teenage girls.

But this begs the question, what questions were asked of the bots? I do not understand the secrecy behind it. As the first round has been and gone and there is no way to post the transcripts, surely we can see what the questions were?

Haris Lodhi - Feb 14, 2013:

But, if a judge asks about the prime minister of XYZ country, that should be known to most of the people on web and it would be a valid question then!

Without looking them up, name the current Prime Ministers of the following countries.

  Armenia
  Azerbaijan
  Belarus
  Cameroon
  Central African Republic
  Chad
  Côte d’Ivoire
  Guinea
  Equatorial Guinea
  Gabon
  Kazakhstan
  Mozambique
  Namibia
  Peru
  Rwanda
  South Korea
  Sri Lanka
  Tanzania
  Togo
  Uganda
  Uzbekistan
  Yemen

Thunder Walk - Feb 14, 2013:

Haris Lodhi - Feb 14, 2013:

But, if a judge asks about the prime minister of XYZ country, that should be known to most of the people on web and it would be a valid question then!

Without looking them up, name the current Prime Ministers of the following countries.

  Armenia
  Azerbaijan
  Belarus
  Cameroon
  Central African Republic
  Chad
  Côte d’Ivoire
  Guinea
  Equatorial Guinea
  Gabon
  Kazakhstan
  Mozambique
  Namibia
  Peru
  Rwanda
  South Korea
  Sri Lanka
  Tanzania
  Togo
  Uganda
  Uzbekistan
  Yemen

Are you kidding!! don’t you know the prime minister of US, UK, Russia?? and this was just an example not an exact question from the competition… And I think bot-masters do maintain a database, where they can integrate them… so people don’t have to remember!!

Um… Haris? The USA doesn’t HAVE a Prime Minister. The closest thing we have is the President himself. And to be honest, the overwhelming majority of Americans don’t know who the Prime Minister is for any country, let alone those on the list that Thunder provided. Sadly, I have to count myself as part of that majority, as well. The information isn’t essential to my daily existence, so there’s no point in keeping that information handy. As for keeping a database of that sort of information for use in our chatbots, some botmasters may have such a thing, but many do not, and the only use for such a database would be for contests anyway, since most people who use chatbots for conversation don’t care about who the PM of Namibia is.

A full fact database will run in the multi-gigabyte range.

I have long felt that instead of each bot having to duplicate redundant facts/dictionary entries/wikipedia, bots should move to a good search API. I have long used that approach in my bots (although some people don’t like it). In the future, you will find less bots embedding the databases directly.

The problem is not that we are knowing the Prime Minister or not. The problem is “what is a good chatterbot” ? Is it a chatterbot which has a lot of knowledge or is it a chatterbot which has intelligence ? For me, a bot which call wikipedia or google is not useful, because we can go directly on wikipedia or google. What I would like to reach is another thing.

I have to agree with Denis, in that, for me at least, a chatbot is best when it gives good conversation; not when it can rattle of every known fact on the planet. To my mind, being able to carry on a “convincing”, unstructured conversation is a far better gauge of success than the ability to name the political leaders of obscure countries.

I agree with Dave, it isn’t likely that a visitor will remain with any bot that can’t hold an unbroken conversation for very long.  To that, I’d add that it helps if a bot has a sense of humor, and that it can produce an occasional surprise, the way that Morti and Mitsuku can.

And, while it’s not necessary to be able to name each of the moons of Jupiter, having the ability to occasionally access Wikipedia, Google, or (my favorite site) Zombo.com, to find answers, is probably better than saying, “My data base doesn’t contain that information,” over and over again.

In any event, if the knowledge your human guest is seeking is important, they’d probably just search the Internet for it, rather than ask a chatbot.  No one asks my bots factual questions unless they’re testing them.

Thunder Walk - Feb 15, 2013:

...having the ability to occasionally access Wikipedia, Google, or (my favorite site) Zombo.com, to find answers, is probably better than saying, “My data base doesn’t contain that information,” over and over again.

Ok, I admit it, Thunder. That’s TWICE you got me with zombo-com. Shame on me for not remembering the first one.

Thunder Walk - Feb 15, 2013:

No one asks my bots factual questions unless they’re testing them.

That’s my point exactly. And because of that, I question the validity of using more than one or two of those sorts of questions in the context of a competition, or of placing too much emphasis on them while scoring a chatbot in a competition environment.

@SuperDaveMorton

lol… Sorry my friend.  Zombo.com is a good web site to have your bot send those abusive visitors.

I have an idea for a new thread regarding contests, and I’ll save it for later, but I think there are some ways to produce a bot that seems smart without it having to know everything.

Denis Robert - Feb 15, 2013:

The problem is not that we are knowing the Prime Minister or not. The problem is “what is a good chatterbot” ? Is it a chatterbot which has a lot of knowledge or is it a chatterbot which has intelligence ? For me, a bot which call wikipedia or google is not useful, because we can go directly on wikipedia or google. What I would like to reach is another thing.

I agree with you Denis. I think you got the point that we are trying to persuade.

These are all excellent points as it relates to the field of AI.
Personally Im thinking of shifting my focus from AI to Time travel in order that I may travel 36 generations into the future and discover the answers to these questions

[1] Why can the transcripts for the first rounds not be posted?
[2] Failing the posting of the transcripts, can you tell us on which day(s) the judges attempted to visit the bots that were disqualified?
[3] When Will ALL of the transcripts be posted including the first round?
[4] who won

return safely, and post them here

V

Haris Lodhi - Feb 19, 2013:

Denis Robert - Feb 15, 2013:

The problem is not that we are knowing the Prime Minister or not. The problem is “what is a good chatterbot” ? Is it a chatterbot which has a lot of knowledge or is it a chatterbot which has intelligence ? For me, a bot which call wikipedia or google is not useful, because we can go directly on wikipedia or google. What I would like to reach is another thing.

I agree with you Denis. I think you got the point that we are trying to persuade.

I think we all understood; we just disagreed with the example given.

There are also some differing opinions regarding what makes a good chatbot (or chatterbot) and the value of knowledge vs intelligence.  I’d first ask, “What do you expect a chatbot to be used for?”  Do you seek something that can solve any and all of the complicated problems of the world, or simply a program that can emulate something like a conversation between two sentient beings?