Erwin Van Lun - Nov 28, 2011:

We had a discussion a while ago about several new approaches to chatbot (con)testing:
http://www.chatbots.org/ai_zone/viewthread/140/

I think this won’t work since it’s so time-consuming and it yields very little value for me. Bot contests must be easy to enter in their basic form. Just submit the bot and then get access to statistics and chat logs plus some extra traffic. This safely yields lots of value for very little effort. Additional test modes are nice, but they must be optional.

Erwin Van Lun - Nov 28, 2011:

What you’re basically saying is:
-I’m going to a website
-I’m seeing a chatbox
-I start to chat
-I enjoy the conversation
-When would I become suspicious? Or is it a human anyway? I’ll stay in confusion forever.
Is that what you’re saying?

That’s basically what Turing Hub does now. Regarding uncertainty, Turing Hub lets you make your decision whenever you want. Final stats only contain total votes. An improvement would be to report how long do people take to decide, how many take the effort to finish the test, perhaps how often do people say LOL or xD, etc.

I’ve got news. My bot Teo is now participating on Turing Hub. This proves that Turing Hub is still functional. If you have a general-purpose bot of your own, don’t hesitate to join the party.

How do we talk to the bots on the Turing Hub web site?

This confirms my opinion that Turing Hub site has usability issues. Just go to Turing Hub homepage and click the big red Take the Turing test banner.

Edit: The site selects one bot randomly. If you want to talk to specific bot, you will have to go to the respective bot’s homepage (e.g. Teo’s homepage). Use search to find the other bots. Direct URLs generally aren’t listed on the site.

I tried that. I assume we can’t choose which bot to talk to then?

Actually, if I make the first question, “what is your name?”, I can work out who I am talking to.

Steve Worswick - Dec 3, 2011:

I tried that. I assume we can’t choose which bot to talk to then?

Sorry. See my edited reply above.

I tried your Teo bot but could’t get it to learn anything. Am I doing something wrong?

You: my name is Steve
Teo: o_O
You: what is my name
Teo: No. You’re name is Theo.
You: learn my name is Steve
Teo: you are stupid
You: what is my name
Teo: What it has?

You are expecting too much too soon. Teo learns through imitation like a child. He cannot learn from facts like an adult. This is standard behavior with all imitating chatterbots. Teo actually does more than imitation, but it won’t help in your case.

The kind of teaching you are performing will not be supported for quite some time. The next significant feature on my list is learning from demonstration. It can achieve results similar to what you are expecting, but it will still require you to use training methods suitable for children (i.e. demonstration and/or corrections).

I don’t think it’s you, Steve:

Teo: Well just be funny with me?

Me: Nope!

Teo: yes

Me: Who are you

Teo: But not for him to take my place, are you out your mind?

Me: You seem to simply be spouting gibberish.

Teo: neither are you

Me: What is your name?

Teo: Pokerface

Me: I thought your name was Teo.

Teo: my name is maw

Me: Really? My name is Dave.

Teo: u

Me: Yes. Me. Dave.

Teo: U mad bro?

Me: Of course not. why do you ask?

Teo: you have a favorite tv show?

Me: I really like the show Fringe.

Teo: Well, it’s normal to be happy, isn’t it?

Me: Yup.

Teo: omg

Me: What?

Teo: Don’t worry.

Me: I’m not worried.

Teo: waaahhh

And that was about as far as I was willing to take that particular charade. Morti may not be the best chatbot in the world, but he’s far more capable of carrying on a conversation than this.

Robert, the main problem that I see here is that your bot’s apparent vocabulary is out of sync with it’s development level, and this will cause people who use it to think, just as I indicated to Teo, that it’s just spouting nonsense. I can pretty well guarantee that this will drive off the overwhelming majority of visitors, never to return, and of those that do stay, most will only do so to abuse your bot with foul language, and sexual statements. If your bot learns by imitation and example, then you will eventually end up with an ill-behaved 13 year old with a potty-mouth and a penchant for explicit sexual speech.

I applaud your efforts, but I strongly suggest that you do the initial training for your bot away from the internet, so as not to overly expose it to the evils of unrestrained pre-adolescents at this stage. Perhaps you can enjoin some trusted individuals to assist, asking that they commit a couple of hours per week of quality, intensive, one-on-one training, in an effort to provide a more consistent and trustworthy education than exposing it to the mercy of the net. After all, if this were an actual child of 18-36 months, I’m sure you wouldn’t just sit it in a truckstop with a sign around it’s neck that read “Talk to me”, would you?

Really, Well I think his bot is already a potty-mouth with a penchant for explicit sexual speech,
Raymond Lavas

@Dave: We are getting a little off-topic here, but I nevertheless thank you for your intelligent and insightful reply. I am well aware of pros and cons of unstructured learning methods. But let me explain why some of the perceived drawbacks are illusory.

Abuse is a serious problem, but it’s not fatal. Most of the web is nonsense or repetitive spam, yet google consistently lists quality results. How is that possible? Filters and scoring. Teo already performs quite heavy filtering/scoring, both hard-coded and learned. I know there’s still room for improvement.

Teo is currently an entertainment chatterbot. He doesn’t try to be intelligent nor logical. He instead tries to evoke emotions and provoke people to chat a little longer. I know it’s not what many people are looking for. As Teo matures and his goals become deeper, I hope many more people will find him useful.

As for the apparent disconnectedness of Teo’s responses, this is caused by a combination of several factors: still too small training database, weak consistency-seeking algorithms, and text-only domain of experience. All these factors are being worked on.

I have already considered having people adopt and train their own bot. It was my initial idea. It seems attractive at first because it’s anthropomorphic. Unfortunately it doesn’t really work. Computers are not humans. Splitting Teo’s memory into several mini-bots would just cripple him. These mini-bots would be abandoned over time and none of them would get anywhere. Shared memory is vulnerable to abuse, but benefits still outweigh the costs. This is not to say that specialization is bad. Specialization is alright as far as it is complementary to knowledge sharing.

I hope you will reconsider your position towards learning chatterbots, especially public ones.

I think that learning chatbots are great, but if you’re going to train it in an unstructured manner, as Teo is, then the knowledge bass should be “seeded” with a minimum amount of information, along with a small number of simple rules of behavior that serve as a “core frame of reference”. For example, a simple, single greeting (e.g. “hello”) that can be expanded upon as the bot learns, and one or two statements of confusion (“I don’t know”, and “I don’t understand”) along with a simple request for assistance (“can you teach me?”), which could then be used to let the user know that training is required in the current context, which can also be expanded upon as the bot learns. Yes, I know that this isn’t how humans learn, and as infants/toddlers/small children, we seem to start with zero language skills when we learn our first word, but that’s not strictly true. A baby develops the ability to hear sounds at about 18 weeks into the pregnancy, according to MayoClinic.com. That means that we have around 22 weeks, over 5 months, of input to use and learn from before we’re even born (Yes, this means that I believe that we begin learning even before birth). Add to this the fact that we, as infants take in every sound our that’s uttered by other people every waking moment of our lives and at least subconsciously store it away for later use and/or reference. This represents a significant amount of data at an infant’s disposal by the time he or she speaks their first word, and by the time a toddler is ready to talk to their first “stranger” there is a large amount of “ground work” that has already been established for them to rely on. This is where I envision Teo to be at, except that the lack of “ground work” is limiting Teo’s potential to properly learn.

I sort of lost myself there for a moment. Did that make sense?

I don’t know.. Of all the bots I have tested, I have not been impressed by any to date.
Recognizing keywords and then jumping to a scripted context is not enough.

For example; if I was to say..

Human: I love apples.
Bot: Yes, apples are good.

How about this instead..

Human: I love apples.
Bot: Yes, apples are a good fruit and Apple is also an innovative company. Which do you like best?

This would require a high level Language Parser capable of not only identifying a type of word as grammar, but also the ability to associate certain words that may have multiple meanings. In addition, a hierarchy must be established for each subject noun in the sentence and this along with the comprehension of the pronouns, verbs, adverbs, prepositions and adjectives, a response can be calculated that is accurate and on topic.

This is where my research and development is focused and what my project will be capable of delivering. Guessing or approximating the reply is unexpectable to qualify as true AI.

Laura Patterson - Dec 5, 2011:

This would require a high level Language Parser capable of not only identifying a type of word as grammar, but also the ability to associate certain words that may have multiple meanings. In addition, a hierarchy must be established for each subject noun in the sentence and this along with the comprehension of the pronouns, verbs, adverbs, prepositions and adjectives, a response can be calculated that is accurate and on topic.

This is where my research and development is focused and what my project will be capable of delivering. Guessing or approximating the reply is unexpectable to qualify as true AI.

That’s the approach that I’m taking too. How far along would you say you are Laura?

For my part I devoted years to researching and developing parsing software. Now I’m constructing grammars that it can use, and the next step will be to assemble a knowledge base. Maybe after that, “learning” capabilities will become relevant.