Just have a look at this post

http://gizmodo.com/5910812/the-best-chatbot-in-the-world-is-still-awful

A product is tagged “the best chatbot in the world” and then is used as a good proof why chatbots do NOT work.

My idea is simple: A chatbot can be helpful to end uers even it doesn’t pass the Turing Test.  Turing Test has its role in research. However we who develop AI applications should create a new creteria that works better in real world and peole can easily understand. For example, where a chatbot created for a business, what is the percetage of questions the bot can correctly answer?

Further more, this post is not about Turing Test itself. It’s about client education and market cultivation. As American Milk Association publish many ads to educate peole to consume more milk, can we, the AI applications developers, join together to the education job?

As long as people believe chatbots are all lab toys that pretend to be real person but miserably fail, none of us can be very successful on this market. And I believe none of us along can change the impression or concepts that people have established for so many years.

Firstly peole have seen so many super-smart robots in sci-fis that they give unrealisticly high expectaion to the robot in reality. On the other hand they don’t belive a robot can help them because the media, either traditional ones or online ones, keep telling them robots, particularly chatbos, are just a joke!

Erwin and Dave have done much in educating the outside. I personaly appreciate their efforts greatly. However that’s not enough. I believe we need two major changes.

1) We should not promote our industry around something like Loebner Prize or Turing Test any more. We should bravely tell people our bots are not capable enough to act as human being. And we should clearly tell people the purpose of our bots is not to act human being. The bots, who work as bots, help people get information they need without needing to reading lenghty web pages, get problems resolved without wating for a human reply for hours…

2) It’s time to establish an independent and non-profit organization for our industry. Erwin has beening moving in this direction and I believe it’s time to make it happen this year. Only with such an organization, we can do something meaningful in educate the clients and change their perception. Only with a positive perception, we can make the whole industry, and each of us, very successful!

I’m aware most of players in our industry are custom solution providers. Many just want to educate one client each time. That’s not econocally saleble indeed. That’s partially why our trade hasn’t grow as it should be. Think about, what if most business owners believe chatchats can benefit them? What if everything website wants to add a chatbot to their web pages? Lt’s think big!

I just want to start a discussion here. So any comments or critiques are welcome.

http://gizmodo.com/5910812/the-best-chatbot-in-the-world-is-still-awful - Jun 4, 2012:

“I’m still dumb.”

I would argue that, in some respects computers are more intelligent than humans.  Can I write a program and give it to a human and have it play a better chess game than I can?  The human will make mistakes, but I can with my computer.  Can I give a human millions of instrutions and have them follow them flawlessly? nope, but a computer can keep virtually limiiless complexity ‘straight’.    Computers have intelligence.  They lack understanding.  That is what is being worked on.

so it should just say

“I’m intelligent, but I currently still have a lack of knowledge which prevents me from conversing via language”  and save “I’m still dumb” for the humans that don’t know this fact

http://gizmodo.com/5910812/the-best-chatbot-in-the-world-is-still-awful - Jun 4, 2012:

Actually, it’s kind of unfair to suggest that the Loebner competition represents the cutting-edge of AI, because a lot of academic researchers and large companies shun it in favor of more professional pursuits.

A full NLU system with the skill of a human wouldn’t be ‘professional’ ?!

Vic Duan - Jun 4, 2012:

On the other hand they don’t belive a robot can help them because the media, either traditional ones or online ones, keep telling them robots, particularly chatbos, are just a joke!

I’m sorry to say, but you arer showing your ignorance here. Your post clearly shows that you have a very limited view on the world of AI-research. For example, robots (as in robotics) have very little to do with chatbots (as in ‘internet bot’).

Maybe I can suggest some reading material for you:

http://en.wikipedia.org/wiki/Artificial_intelligence
http://en.wikipedia.org/wiki/AI-complete
http://en.wikipedia.org/wiki/Robotics
http://en.wikipedia.org/wiki/Artificial_general_intelligence
http://en.wikipedia.org/wiki/Philosophy_of_artificial_intelligence

Just to get you started, there’s more of course

Hans, he’s saying that seeing “super smart robots” on television gives people an inflated sense of what an artificial personality (like a chatbot) is really capable of. Just because he goes on to use one word (robot) in perhaps a broader semantic sense does not invalidate his overall point. No need to be so condescending.

Vic, I agree with you that the Loebner prize appears—to a lay audience—to represent the cutting edge in AI, and leaves those who hear about it unimpressed and sceptical about what chatbot technology is capable of. A refocus away from open-ended chat and towards specific domains would help those who’d like to turn chatbot development into a profitable business.

However, as a chatbot hobbiest, my primary interest isn’t in monetization but in forwarding generallized chatbot conversational abilities. And as long as many developers are in the same camp, I’m afraid you won’t see a consolidated push in the community to improve our PR. Perhaps I’m wrong. Certainly developers are shooting themselves in the foot by not presenting a clear and realistic message about what a chatbot should be capable of.

But if developers can succeed in bridging some of the gap between expectations and reality, I think they won’t have trouble finding an eager audience. That’s the hope anyway.

Victor Shulist - Jun 4, 2012:

so it should just say

“I’m intelligent, but I currently still have a lack of knowledge which prevents me from conversing via language”  and save “I’m still dumb” for the humans that don’t know this fact

I see what you’re trying to say about the capabilities of computers, and to some extent I agree. But there is another pervasive problem in the chatbot community: chatbots who have a ton of knowledge at their disposal, but can’t do anything intelligent with it. There exist large databases of semantic and world knowledge that most chatbots, even if they have access to it, can’t use it in meaningful ways. Can you think of one chatbot out there that can perform even the most basic logical functions, that can string together two facts to construct a third? I’m waiting for the day where this type of functionality is commonplace for chatbots.

Victor Shulist - Jun 4, 2012:

http://gizmodo.com/5910812/the-best-chatbot-in-the-world-is-still-awful - Jun 4, 2012:

Actually, it’s kind of unfair to suggest that the Loebner competition represents the cutting-edge of AI, because a lot of academic researchers and large companies shun it in favor of more professional pursuits.

A full NLU system with the skill of a human wouldn’t be ‘professional’ ?!

Pursuing full NLU as a research goal isn’t very professional. To broad and ill-defined an endeavor. But hey, that’s why we’re hobbiests.

C R Hunt - Jun 4, 2012:

date=“1338827792”]chatbots who have a ton of knowledge at their disposal, but can’t do anything intelligent with it.

Fair statement.  What I consider knowledge, though, is both ‘declarative’ and procedural—knowing and knowing what to do with that knowledge   They are very tightly connected in my design.

C R Hunt - Jun 4, 2012:

Pursuing full NLU as a research goal isn’t very professional. To broad and ill-defined an endeavor. But hey, that’s why we’re hobbiests.

eh…  seems reasonably well defined to me.  Specific applications can certainly be quite defined…. System requirements: ability to read all literature on a company’s products & services and answer potential customer’s questions.  Or—help customer’s with problems they have with our products, or, using naturual language, a system assists a human in debugging software.

C R Hunt - Jun 4, 2012:

Hans, he’s saying that seeing “super smart robots” on television gives people an inflated sense of what an artificial personality (like a chatbot) is really capable of. Just because he goes on to use one word (robot) in perhaps a broader semantic sense does not invalidate his overall point. No need to be so condescending.

CR, I don’t agree; if someone clearly sees no real distinction between a robot and a chatbot, I think it’s better to point out their ignorance then to be ‘just friendly’. People that are coming new into this field do think that ‘chatbots = AI’, and really can use some help to give them a better insight of the field. It’s better to be educated that to be left in the dark.

Besides that, misconceptions are among the biggest problems holding back AI research.

Hans, I agree that improper semantics can lead to miscommunication. But in this case, Vic hardly seems new to the field. Even from his post alone, that seemed clear. And “robot” is the source of the “bot” in chatbot after all. And his point was preceded by the example of chatbots united with robots in fiction. In view of all that, you came off as condescending. I do agree with your larger point, but not in this case.

Victor, I realize now that what I was talking about in terms of “intelligence” you call “understanding”. Oh, semantics strikes again. What I’m concerned with is exactly what you said: knowing and knowing what to do with that knowledge. Or at least having some ‘intellectual tools’ available for the bot to play around with that knowledge, however well-applied they may be.

And I agree that developing products for specific applications is defined well enough. But that’s not general NLU, is it.

C R Hunt - Jun 4, 2012:

Can you think of one chatbot out there that can perform even the most basic logical functions, that can string together two facts to construct a third? I’m waiting for the day where this type of functionality is commonplace for chatbots.

So just to clarify - the comment was regarding the extraordinary power and potential of computers for intelligence.  So I’m refering to specifically computer hardware and compilers ... not chatbots.    Chatbots, yes, most are just toys and just for fun. 

Your comment about no chatbots in existence that can string toether facts and produce a new one is correct of course, but I’m speaking of computers+compilers potential—- the current inability of chatbots to accomplish natural language understanding and moreover, natural language inference, is not, in my opinion a problem or limit of computers, but a limit of perhaps the human mind (so far) to figure out how to provide the necessary data+procedure to this most powerful of all machines.

I believe it is we, so far, that are limited, not the computer.  BUT… trust me, progress is being made

For the NLU/professional issue, to me it doesn’t matter if it is or isn’t, it’ll be the best thing since sliced bread.  If it is “ill defined” ..  let’s just ‘well define’ it

Hans Peter Willems - Jun 4, 2012:

I’m sorry to say, but you arer showing your ignorance here…

“People who live in glass houses shouldn’t throw stones.” They say.

Well, Hans, thanks for pointing out my ignorance. I will learn more. Before that let’s back to the how we could educate the clients and communicate with the public better.

C R Hunt - Jun 4, 2012:

...
However, as a chatbot hobbiest, my primary interest isn’t in monetization but in forwarding generallized chatbot conversational abilities. And as long as many developers are in the same camp, I’m afraid you won’t see a consolidated push in the community to improve our PR. Perhaps I’m wrong. ...

CR, perhaps you’re right, to the least I observed many individual developers think the same. However I believe a good PR would benefit the whold industry, if there is acutally such an industry, including all players from for-profit companies to for-pleasure hobbiests. The recognition of the public can attract more resources (people, money, exposures etc.) to flow into the field. That advances the technologies and then enables us to present better products to the public.

Hans, from your website I see you’re soliciting funding for your company. Don’t you think what I proposed would benefit you firstly? ;-)

Firstly peole have seen so many super-smart robots in sci-fis that they give unrealisticly high expectaion to the robot in reality. On the other hand they don’t belive a robot can help them because the media, either traditional ones or online ones, keep telling them robots, particularly chatbos, are just a joke!

I agree with this statement.  But I also think that perhaps this change has already (slowly) begun. We still have a lot of 50’s era sci-fi (or 80’s: just seen terminator 1 again, not much dialog for the bot though), but there’s also stuff like the movie ‘moon’, which I loved. The bot (it’s a physical moving machine, so it’s a bot), has a few opening statements that are continuously recycled, much like current chatbot technology.

Jan Bogaerts - Jun 5, 2012:

just seen terminator 1 again, not much dialog for the bot though)

I got such a kick out of when he selects randomly from a set of canned statements, and pics “f… you…. a….” lol.  He didn’t even synthesize a response, however it did consider the most ‘applicable’ statement for the landlord of a crappie apartment building lol

In order of coolness, I think it was Terminator 2, 4, 1 and lastly 3.  We’ll see what #5 gives us this summer.

I found #4, which I watched a couple of days ago again, made the robots look so ugly that it was like watching a horror flick lol.

Not a bad premise in the Terminator movies (same old , same old, robot programmed to survive at all costs, humans want to shut it down, thus conflicting with goal of staying alive….conclusion- remove the threat), but I don’t know, not as plausible of a story as “I Robot” (their job - keep humans alive, but humans self destruct, thus take over), which was, by far, more plausible of a story of ‘strong ai’ than terminator.

For TV shows, the (new - that is, 2008) Night Rider was a very ‘believable’ AI.

CR - yes, I strongly relate the power of understanding (making sense of the world) to be related to intelligence.

I did not know there was a 5 coming along, But it would explain the re-run of the first one.  I found nr 4 a big disappointment: they appeared to have gone all monster-truck like. What about nano/bio tech? (but perhaps then the film wouldn’t have worked, with all the humans gone)

Vic:

CR, perhaps you’re right, to the least I observed many individual developers think the same. However I believe a good PR would benefit the whold industry, if there is acutally such an industry, including all players from for-profit companies to for-pleasure hobbiests. The recognition of the public can attract more resources (people, money, exposures etc.) to flow into the field. That advances the technologies and then enables us to present better products to the public.

Yes, I agree that presenting a clear message about the extent to which chatbots should be expected to perform can only be beneficial for chatbot developers of all types. It allows companies to delineate and market products and it allows all developers to showcase when they’ve pushed the envelope in one area or another.

All that being said, I still don’t see the chatbot community uniting to forward a shift away from the notion that chatbots should aim to engage in general conversation. This idea is too fundamental to the original notions of a chatbot and too close to the hearts of many hobbiests. If anything, such a PR effort would have to be led by those involved in for-profit ventures.

Victor:

Your comment about no chatbots in existence that can string toether facts and produce a new one is correct of course, but I’m speaking of computers+compilers potential—- the current inability of chatbots to accomplish natural language understanding and moreover, natural language inference, is not, in my opinion a problem or limit of computers, but a limit of perhaps the human mind (so far) to figure out how to provide the necessary data+procedure to this most powerful of all machines.

Agreed!