Thoughts from the results of the Second Conversational Intelligence Challenge (a competition between chatbots) that took place during the NeurIPS artificial intelligence conference in Montreal in December. The Article also has some info on commercial state-of-the-art chatbots.

This reinforces my belief that using a neural network for a chatbot instead of being rule based is simply not an option. I have yet to see any neural net based bot that was any good.

Same here Steve.

I also agree with that, they’re barking up the wrong tree. The subject matter conveyed in conversation is too specific to just correlate words and hope that leads to an appropriate answer. Conversation is just not statistical beyond generic small talk about the weather. In addition, the problem of inconsistent personality, contradiction, and repetition, is created by the approach itself.

Having said that, I find this article opinionated and harsh. One example of what it calls “soul crushing dialogue” shows a program repeatedly saying “I don’t know what you mean”, but if you look closer, that seems to be a result of the user’s misspellings, omissions of subjects, and cryptic responses. I get it, Facebook is out to commercialise this so they want programs to handle all sorts of nonsense, but that hinders testing the programs’ technical capabilities.

Here is a paper describing the event with more detail: https://arxiv.org/pdf/1902.00098.pdf

Pattern-matching chatbots (AIML) or POS/pattern-matching (Chatscript) do appear superior, but I am still of the opinion that a combination of these methods with ML may be useful.

Not surprising, the “Mechanical Turk” method of training a bot seems to be the best (and is basically like AIML/Chatscript then). In other words, it still takes people to properly train a machine to respond as a human-  purely corpus-trained NN does not perform very well, even though the corpus may be actual human/human conversations.

The goal of being able to create chatbot a priori by an algorithm from a defined data set is a lofty goal to be sure, it is also an easy sell to Corp because “CHATBOTS” are all the rage these days; an easy way of maximizing profit by reducing cost through the elimination of human employees.

Let us be glad for now that chatbots are still not very human-like.  When they get too good to differentiate from a real human, we may regret it.