Today I was thinking about intonation and meaning in sentences (from wikipedia):

“I never said she stole my money” - Someone else said it, but I
didn’t.
“I never said she stole my money” - I simply didn’t ever say it.
“I never said she stole my money” - I might have implied it in some
way, but I never explicitly said it.
“I never said she stole my money” - I said someone took it; I didn’t
say it was she.
“I never said she stole my money” - I just said she probably borrowed
it.
“I never said she stole my money” - I said she stole someone else’s
money.
“I never said she stole my money” - I said she stole something, but
not my money.

And I was wondering myself: how can we ever creating intelligent text based chatbots without using voice?

Well, I didn’t use my ears to receive those messages, but I understood.

My bot currently knows when you are putting stress on a specific word if you surround it with asterisks.  Later, when I have a coloured GUI for my bot, perhaps I will allow you to change color for word, simply by double clicking it , that will cause the word to toggle between black text (no stress) and red (stress on the word).

As long as the user knows how to indicate stress on a word, there is no problem.  Later, I am going to put in rules and logic that, taking which word you put stress on, will change the meaning.  Based on the word you specified to have stress, and given the parse tree, I don’t think it will be too hard to alter the internal representation of the intended meaning. 

So,  “I didn’t say she stole *my* money”...  the bot will notice that you are putting stress on the adjective ‘my’ which modifies ‘money’... thus it will consider that word “questionable” and consider other modifiers.. perhaps you meant ‘his’ money, or ‘your’ money.  Same concept as the rest of the words… the bot will see stress on a specific word, find out what role it plays in the sentence, and find and consider alternatives.

I’m working hard on this functionality . .. Stay tuned !!

I’m staying tuned. I’ll definitely stay tuned! It’s all so exciting what you guys are producing!!!

Hi Erwin,
I’ve used an example similar to yours in a communication exercise I taught to employees at the company I work.  It illustrates a weakness in written correspondence and the problem with verbal comms.

I intend, eventually, to deal with this specific issue by simulating it in text only chat mode similar to Vic.

I NEVER said she stole my money.
I never SAID she stole my money.

I would like my parser to allow for upper case words and to assign the appropriate emphasis. In addition, the chat bot can communicate emotion in a similar way by changing the case of a word or two. 

Vic,
That’s cool that you can communicate stress to your app.

Regards,
Chuck

what if we use font size differences per character?

Heck, just about anything that alters the appearance of the stressed word could be used, I’m certain, although my favorite is SHOUTING.

we__ cou________ld a_lso____ u_______________se_ u_nderli_ne_s

to indicate the length (duration) of vocals

Yes, the format used is irrelevant.  I think just pick one and let the user know what it is !  No sense having 25 different ones     Since, after one would acheive creating a bot that could pass the turing test, you would want to go to audio anyway, in which case those 25 different formats would not matter.  You would be dealing with audio frequency wave form analysis algorithms to determine stress, intonation, etc.

Is there no software out there available that translates audio input into transscripts + intonation? That would be a nice project on itself!

There probably is.  For me though, this is SO far in the future.  I won’t take on that project, I would rather buy the functionality and put it as a front end to my bot.  We’ll keep putting all these building blocks together until we end up with ‘Mr. Data’ on Star Trek !!

It’s actually not far in the future. It’s a trend right now. Speech Analysis companies, traditionally working for call centers, are starting to co-operate with professional chatbot developers, who are developing virtual agents (or how you’d like to call them) for ..call centers. Speech companies claim that they can better predict what is actually said, when they understood what is being said.

To mention an example:
What can I do this weekend in Am*********m (where ******* is noise).
Speech analysis software can’t predict what’s between Am and ...m
while smart listening chatbot can predict it’s probably Amsterdam, because this weekend in <city> is a quite common phrase.

Awesome stuff !!

So it will be the combined strengths of both audio waveform analysis and contextual/knowledge based inference !  Now that is truly fascinating !!.  When I’m completed my core NLP engine, I will want to look into that, putting that ‘front end’ for the bot.

This is why I am focusing on grammar so much.  The grammatical construct, coupled with hints about the properties the words have, combines to give a powerful mechnism to give ‘best guess’ hints to clarify to the audio waveform analzyer.

Erwin Van Lun - Aug 16, 2010:

Today I was thinking about intonation and meaning in sentences (from wikipedia):
...
And I was wondering myself: how can we ever creating intelligent text based chatbots without using voice?

Now that is a great example to show that language and spoken communication is more than just the characters in our emails. However it also makes clear, that several communication channels simply do not easily transfer all of the nuances of spoken communication.

Many misunderstandings in email - or in a forum - result from things like this. Missing intonation, which leads to a wrong interpretation/disambiguation and voila: Let the flamewars begin. Email is not the only domain where intonation causes problems. See sign language and deaf people. There are concepts for putting something in focus, but it still does not resemble the possibilities of the audio channel.

The fact that even people get interpretations wrong if the intonation is missing leads me to the conclusion, that this NLP-problem is unsolvable in the general case if your communication channel does not provide a carrier for that information.

You MIGHT get it right in some cases or at least in some direction. Imagine the bot answering with some words emphasized (all caps, italics, bold,...) which could help the human to understand an intention better.

Richard

I believe it’s not the main problem we are facing. When human chat in text, they get different information from voice chatting. As Erwin’s example,  in all the sentences, the basic idea are the same. The only difference is that besides the basic idea the listener may guess some additional information. In text chatting, the reader won’t guess. Actually, when a human chat in text with another human. He may use two sentences to represent the full information instead of only one sentence with intonation.

Nathan, that depends on the person passing the information text, and the specific media being used. For example:

When I’m typing in a forum thread, such as this one, I have at my disposal various text formatting options that I frequently make use of. If I want to place stress or emphasis on a word, I can use bold, italics, or font size formatting, and I even have the use of color. Failing these conveniences, I still have UPPER CASE and (parentheses) at my disposal to use to add stress and emphasis to my text. Granted, this is more work, but it CAN save time, by minimizing the need for clarifying sentences. Not to mention the use of smilies.

Sure, Dave. Human can chat in text with additional information. But human can chat in text without additional information too. It’s an additional functionality for a chatbot, not a necessary one. Actually, many chatting interface only support text without any tags. There is not any problem that human communicate to each others in pure text.

You are right. These intonations are for saving time. But it’s not necessary for communication. The intonations can be replaced by more explanation sentances.