Some tricks used in LP, as I remembered remotely, are something like “which is bigger, Belgium or apple?” and “which is nearer, the Queen or the Queens street?” and “If we shake hands then what is in your right hand?”

These are good and reasonable tricks. Yet in my own design and development of chatbot I found Quotation is very difficult, with or without quotation marks. For example, Tom said Jack already told Sarah to call Peter…and two other most difficult issues to me are comma and “and”. These two little and common symbols are driving me crazy! Try parse “Tom knew Mary and Jack knew Sarah”

To me the only solution is a process of guess-evaluate-adjust but then there is no best answer but just valid options. Natual language is an art, anyway, esp. those stuff like flirt, joke, poem, political correct…

Or shall we design a lite version of English called “Concise English” and force everybody to use it with computer?

and, I hereby proudly annouce LQ now can handle composite noun like “the small red Japanese seafood restaurant”, and even the multiple pocessives like “Tom’s wife’s middle school classmates”, but it can not deal with the same seperated by dash, eg “we love to see the do-it-with-one-hand-and-two-legs trick”, which is actually a sentence. For dash LQ can handle a word with a relatively fixed pattern, eg “23-years-old-girl”.

Another difficult issue is to find which word in the context a pronoun refers to. The toughest one is “it”, which can be climate, time, a thing and even a whole sentence of a event or a thought. “She” can be a woman or a female animal or a boat or whatever the caller want to choose…

Good luck to me and everyone else who’s brave enough to challenge the heart of AI: natual language!

John Li - Sep 1, 2010:

Some tricks used in LP, as I remembered remotely, are something like “which is bigger, Belgium or apple?” and “which is nearer, the Queen or the Queens street?” and “If we shake hands then what is in your right hand?”

These are good and reasonable tricks.

reasonable tricks, yes absolutely.  The ones I posted are stupid .. I hope they don’t use those in the Loebner prize otherwise the thing is a waste of time and does not help to advance AI or chatbots.  However, I think they DO use these kinds of tricks.

John Li - Sep 1, 2010:

Yet in my own design and development of chatbot I found Quotation is very difficult, with or without quotation marks. For example, Tom said Jack already told Sarah to call Peter…and two other most difficult issues to me are comma and “and”. These two little and common symbols are driving me crazy! Try parse “Tom knew Mary and Jack knew Sarah”

Do not depend on the other to use proper punctuation.

John Li - Sep 1, 2010:

To me the only solution is a process of guess-evaluate-adjust but then there is no best answer but just valid options. Natual language is an art, anyway, esp. those stuff like flirt, joke, poem, political correct…

Human beings, a LOT of the time do not understand each other and have to ask for clarification !!  Also.. this ‘guess’  should be guided by conversation state.  That is, what has been said so far in the conversation, the topic, who you are taking to,etc.
In fact, I have had conversations with people, and listened to two people talk, and even after one has asked the other to clarify they are STILL talking about 2 different things and not knowing it!!    So it is absurd to think that a single LINE OF TEXT entered to a chatbot should be understood with no ambiguity!  Sometimes, it is simply impossible for man or machine to understand a single LINE OF TEXT without conversation context.

When you say all possible interpretations of the user input are guesses, you have hit upon the TRUE nature of A.I.,  N.L.P and thought…  the very essence of thought is guessing, and then seeking to prove or disprove your hypothesis. 

John Li - Sep 1, 2010:

Or shall we design a lite version of English called “Concise English” and force everybody to use it with computer?

Yes, at first absolutely.

John Li: Our approaches must be similar, my bot also hates conjunctions. Right now it can only understand complex sentences if it has learned a pattern for how to dissect them into simple sentences, using various special symbols to indicate how to group the simple sentences. Hopefully with enough training, it can apply one or more patterns to any complex sentence it encounters…

I haven’t even begun dealing with nouns that have multiple words. Right now my bot assumes all but the last word are adjectives. This might actually work out okay in most circumstances, but I’d like a more elegant solution.

The problem with pronouns is that they can only be interpreted with context, and an understanding of the various nouns that the pronoun could be. I’m trying to develop a module to handle pronouns. I think it’s going to have to associate each pronoun with a ranked list of possible nouns that gets modified and refined throughout a conversation. (Or throughout reading a text, depending on the situation.) We’ll see how it goes.

John Li - Sep 1, 2010:

Good luck to me and everyone else who’s brave enough to challenge the heart of AI: natual language!

Good luck indeed!

Victor Shulist - Sep 1, 2010:

John Li - Sep 1, 2010:

To me the only solution is a process of guess-evaluate-adjust but then there is no best answer but just valid options. Natual language is an art, anyway, esp. those stuff like flirt, joke, poem, political correct…

Human beings, a LOT of the time do not understand each other and have to ask for clarification !!  Also.. this ‘guess’  should be guided by conversation state.  That is, what has been said so far in the conversation, the topic, who you are taking to,etc.
In fact, I have had conversations with people, and listened to two people talk, and even after one has asked the other to clarify they are STILL talking about 2 different things and not knowing it!!    So it is absurd to think that a single LINE OF TEXT entered to a chatbot should be understood with no ambiguity!  Sometimes, it is simply impossible for man or machine to understand a single LINE OF TEXT without conversation context.

Whoops, what timing. I absolutely agree with Victor. Context (united with a good understanding of what nouns are capable of verbing) is absolutely necessary to determine pronouns. And ambiguity will always be there, no matter how good the chatbot. The key is to make sure your chatbot asks good questions to clarify! The old adage may be that there are no stupid questions, but that isn’t true in practice. I think a chatbot that can ask smart questions when it is confused shows more true intelligence than a bot that tries to evade or brazen its way through a complicated conversation.

My bot has no issue with conjunctions like ‘and’,  ‘or’,  ‘but’.  It can handle unlimited concatenations of nouns, verbs, adverbs into a list.    If you say “dog, cat, fish and horse” it knows that is a LIST of nouns, with ‘list-type’ of ‘and’.  and if you say ‘run, jump, spin, or twirl’ it knows that is a LIST of verbs with ‘list-type’ of ‘or’.

Of course it has no problem if that is embedded in a sentence like “dog, cat, fish and horse are on my farm”.

Also, it understands LISTS of predicates…

    She washed the clothes and folded the laundry.

2 predicates of ‘list-type’  ‘and’...  predicate 1 = ‘washed the clothes’,  predicate 2 = ‘folded the laundry’

C R Hunt - Sep 1, 2010:

Whoops, what timing. I absolutely agree with Victor. Context (united with a good understanding of what nouns are capable of verbing) is absolutely necessary to determine pronouns.

In stage 2 processing (parse tree generation), my bot first ‘brain storms’ to all the possible grammatical possibilities that a sentence could be.

Then, stage 3, it looks up in its huge database of, like you say, all verbs that a given noun can do, and assigns a kind of ‘merit point’.  That increases the likely hood of that specific parse tree as being the one the user   ‘really meant’.

Hi John, I don’t think we need a well defined “Concise English”. Actually, human have the ability to find a simple way to represent what they are talking about when they talk to a child or 2nd language speaker like me.

John Li - Sep 1, 2010:

Or shall we design a lite version of English called “Concise English” and force everybody to use it with computer?

C R Hunt - Sep 1, 2010:

The old adage may be that there are no stupid questions, but that isn’t true in practice. I think a chatbot that can ask smart questions when it is confused shows more true intelligence than a bot that tries to evade or brazen its way through a complicated conversation.

Yes, and this is exactly the way I have chosen to pursue my bot’s design.  Now in the Loebner prize it would be detected right away as being a bot.  This is why I suggest bot designers forget about the Loebner prize, it takes away from fruitful AI/NLP research.  We need a bot that will learn, ask questions, not another ‘Eliza clone’ that someone writes up hoping to win this prize. 

We need serious, dedicated, and realistic goals where we are ‘patted on the back’ for each milestone, and not expecting to go from a 1920 crank start car to a jaguar in the next 5 years.

I blame the apparent failure of AI and NLP on this type of thinking - expecting too much instead of aiming for each milestone and working our way up.

C R Hunt - Sep 1, 2010:

Victor Shulist - Sep 1, 2010:

John Li - Sep 1, 2010:

To me the only solution is a process of guess-evaluate-adjust but then there is no best answer but just valid options. Natual language is an art, anyway, esp. those stuff like flirt, joke, poem, political correct…

Human beings, a LOT of the time do not understand each other and have to ask for clarification !!  Also.. this ‘guess’  should be guided by conversation state.  That is, what has been said so far in the conversation, the topic, who you are taking to,etc.
In fact, I have had conversations with people, and listened to two people talk, and even after one has asked the other to clarify they are STILL talking about 2 different things and not knowing it!!    So it is absurd to think that a single LINE OF TEXT entered to a chatbot should be understood with no ambiguity!  Sometimes, it is simply impossible for man or machine to understand a single LINE OF TEXT without conversation context.

Whoops, what timing. I absolutely agree with Victor. Context (united with a good understanding of what nouns are capable of verbing) is absolutely necessary to determine pronouns. And ambiguity will always be there, no matter how good the chatbot. The key is to make sure your chatbot asks good questions to clarify! The old adage may be that there are no stupid questions, but that isn’t true in practice. I think a chatbot that can ask smart questions when it is confused shows more true intelligence than a bot that tries to evade or brazen its way through a complicated conversation.

Exactly, when a bot can ask smart questions to clarify the confusing, I believe it has true intelligence. The goal of natural language is communication, is to share idea between beings.

Victor Shulist - Sep 2, 2010:

C R Hunt - Sep 1, 2010:

The old adage may be that there are no stupid questions, but that isn’t true in practice. I think a chatbot that can ask smart questions when it is confused shows more true intelligence than a bot that tries to evade or brazen its way through a complicated conversation.

Yes, and this is exactly the way I have chosen to pursue my bot’s design.  Now in the Loebner prize it would be detected right away as being a bot.  This is why I suggest bot designers forget about the Loebner prize, it takes away from fruitful AI/NLP research.  We need a bot that will learn, ask questions, not another ‘Eliza clone’ that someone writes up hoping to win this prize. 

We need serious, dedicated, and realistic goals where we are ‘patted on the back’ for each milestone, and not expecting to go from a 1920 crank start car to a jaguar in the next 5 years.

I blame the apparent failure of AI and NLP on this type of thinking - expecting too much instead of aiming for each milestone and working our way up.

I agree with you. I hope the Loebner contest can seriously consider this problem. That’s why I ask how can they evaluate which chatbot is better when all of them can not pass the Turing test. We need a realistic standard to evaluate the chatbots.

Actually, I believe the goal in this stage should be resolving problems by chatting in natural language, instead of chatting like a human.

@everyone

This is a wonderful discussion that I’m sorry to say that I haven’t had sufficient time to participate in, due to family issues. I’ve only barely scratched the surface of reading it, but I hope to soon be able to read it in greater depth.

@Victor

How well would CLUES handle the following sentence?

You have a choice of pork and beans, macaroni and cheese, or liver and onions.

I would imagine that this sort of structure provides more of a challenge.

Thanks everybody. Allow me to response in one post, please.

Victor, I totally agree with you on the thoughts on the natural languages. And I admire your work on the “and” and comma solution. But please try the good example provided by Dave, ie “You have a choice of pork and beans, macaroni and cheese, or liver and onions.” And it is fun to give you a weird one here, “Tom and Sarah, the couple in charge, called Sam, who’s the manager, the plumber, and Peter the husband of Jenny, to turn off the lights, close the window, and stay awake.”

C R, conjunction like “or”, “if”, “since”, “because” etc are fine, but “and” is really a big problem to my bot, esp when it comes with comma. For complex sentences, another issue is omission, something like “so do I”, “the same applies to”, “ditto”, and the omitted “that”, are all tough issues. Ranked list sounds like a good solution to me, I’m using a score system to evaluate each option, must be similar to yours.

Nathan, just think about it, if we expand the current command set of a certain OS to support all simple declarative, imperative and interrogative sentences in English, and some kinds of complex sentences with fixed-pattern, would it be easier to apply a voice-driven UI and move the OS to next generation? This expanded command set can also associate with SQL, something similar to Microsoft’s English Query. That’s the idea of the version one of “Concise English” in my mind. I thinks it’s a better start point than children stories since they’re still very difficult—think about the ideas that pig can fly and monkey can read a map… I agree with you that chatbot 1st goal should be resolving problems in NL, not chatting like a human, but the irony is that chatbot’s ability/performance of problem-resolving actually rely on its ability/performance of chatting like a human. Two most important applications of chatbot, in my opinion, are machine translation and information mining, both require the bot to deal with tons of sentences and paragraphs in NL with all odd symbols like comma, dash, hyphen, single/double quotations marks, square/round brackets, etc., so if the bot could really do the 1st step of resolving problem, then it would already be qualified for the ultimate goal: chatting like a human.

Dave, nice example.

Hi John, I believe we are talking about different problems. When I consider chatbot, i’m considering a true AI system, not a natural language application. A commands based natural language interface may easy to practise, but it can never lead to a true AI system.

A child-like chatbot may be hard to practise and can not support a UI interface, but it can validate a knowledge representation model which has the basic ability of induction and reasoning.

I agree that machine translation and information mining are also important. But before we have a full functionality knowledge system which has the understanding ability like a human adult, they can not get practical. Especially the machine translation.

Chatbot has an advantage that it can ask question when it is confused. Asking question make chatbot different from human but it can help the chatbot to resolve problems. So I do not agree that chatting like a human is so important for resolving problems.

Actually, when we considering to develop a chatbot, there are two different problems.
The first is a knowledge model which supports understanding, induction and reasoning.
The second is a educational problem. Even we have a knowledge model which is powerful like true human , we still need a long time to teach it. A human child need to be teach more than 10 years. 

A child-like chatbot can focus on the first problem.

I believe the most important standard for evaluating chatbots is the ability of induction and reasoning in chatting.

Dave Morton - Sep 2, 2010:

@Victor

How well would CLUES handle the following sentence?
You have a choice of pork and beans, macaroni and cheese, or liver and onions.

I have designed the CLUES engine from the ground up with these kinds of complex structures in mind! 

This is a very good example.  What I will do is have CLUES think of this as a list-of-lists

top level list = A, B and C, list-type = ‘or’

list A = pork and beans , list-type = ‘and’
list B = macaroni and cheese, list-type = ‘and’
list C = livere and onions, list-type = ‘and’

right now I have list types of:

and ( ‘dog and cat and mouse’  or ‘dog, cat and mouse’)
or (‘dog or cat or mouse’ or ‘dog, cat, or mouse’)
space (dog cat mouse)
comma (dog, cat, mouse)
complex (dog or cat and mouse)

The ‘complex’ list-type means the bot is not sure, but it knows it has to ask a question !
for the example above, it will ask:

          Do you mean:

                    1: (dog or cat) and mouse
                    2. dog or (cat and mouse)

And , no, before you ask, I don’t think I am going to bother supporting brackets… people usually don’t use brackets and write complex boolean expressions in ordinary text.  Later on perhaps I will, but that is not the core functioanlity of NLP.

I think I will have a new list type of ‘list-of-lists’

Although I have dozens of things in my list to work on next, I will ‘up’ the priority on this one and post the results in the CLUES thread.