Hello everyone!

I’m currently working on some AI that offers a number of possibilities including a “chat bot”  technology.

As AI is an interest I like to keep up to speed on whats going on, especially in terms of Human - Computer interaction.

I’ve come across quite a few bots that use AIML, pattern matching via regex etc, but I was wondering how many of you guys active in this field have looked at, or tried using, true NLP when it comes to processing human input and generating output?

I’m currently implementing this with true grammatical and semantical tree representations, and its going pretty well.  Just curious of anyone else had attempted it and how you found it?

Any heads up on problems you might of hit too would be nice

Thanks

Dan

Hi Dan, welcome to chatbots.org

I know of three other regular contributors, including myself, who are taking a robust approach to natural language processing. The other two are AndyHo and CRHunt, using C# and Python respectively. I am using CommonLisp, C and PostgreSQL on Linux. (Apologies if I missed anyone else who is doing NLP—please speak up if I did.)

Currently I am developing a Tomita GLR parser (the oracle generator is in CommonLisp, the run-time is in C) and context free grammar definitions to perform a variety of tasks. Last week I was able to perform discourse analysis and sentence splitting with this (e.g. http://asmith.id.au/files/the-country-of-the-blind.xml) You might also be interested in the natural language resources that I have been gathering at http://wixml.net

Looking forward to hearing more about your work.

Cheers,
Andrew Smith

Hi Andrew,

Some good stuff on the link you’ve posted.  Will have a good look through for anything of use   Thanks.

I’m still very early on in development (about a week), but Ive been facinated by AI since I was a kid (I built robots and stuff in spare time hehe).  Unfortunatley things get sidetracked and I moved into on other fields in tech/telecomms and IT.

Thankfully I’m in a lucky position to be able to get back to my early fascinations and can pretty much work on this full time.  I’ve have spent quite some time researching various theories of intelligence, language processing and the like before even putting fingers to keyboard.

Ive come to some conclusions and have theories that may go against the grain with regard to some topics, but, stepping outside the box has always proven beneficial before

At present Im mainly concentrating on NLP and structuring the language into common (and hopefully elegant) forms that I can use in a general fashion.

For example:  Dan and Jenna are excited, they are going to the park.  Is structured as below.

{
"actions": {"going-9": {
"action": "going",
"tense": "present"
}},
  “copulas”: {"are-4": ["excited-5"]},
  “elements”: {"excited-5": {
"tense": "present",
"weight": 1,
"word": "excited"
}},
  “objects”: {"park-12": {
"classification": "O",
"det": "the",
"name": "park"
}},
  “subjects”: {
"Dan-1": {
"classification": "O",
"name": "Dan",
"subject_elements": {"excited-5": {
"tense": "present",
"weight": 1,
"word": "excited"
}}
      },
      “Jenna-3”: {
"classification": "PERSON",
"name": "Jenna",
"subject_elements": {"excited-5": {
"tense": "present",
"weight": 1,
"word": "excited"
}}
      },
      “they-7”: {
"classification": "O",
"name": "they",
"subject_actions": {"going-9": {
"action": "going",
"tense": "present"
}}
      }
  }
}

BTW very interesting that you mention discourse analysis, this is something Ive researched, but still pondering the best way to go about it.

Thanks

Dan

There is nothing wrong with having theories that go “against the grain” so long as you’ve done your homework and know what “grain” it is that they’re going against. Far too often we get nitwits storming through the forum with some half-thought out idea that they claim will change the world and oblivious to the fact that’s its been tried (and failed) dozens of times already. For some reason they get very upset and offended when this is pointed out to them and then we never hear from them again.

Can you recommend any good books, papers or software? What are you using to create the example output that you posted?

Heh well there are nitwits in every field, nothing new there

As for books, I can recommed “Critical Thinking Skills” by Stella Cottrell which looks at analysis, arguments, opinions etc, “The Archaeology of Knowledge” by Foucault which looks at formation of concepts, objects, consequences, and Piaget’s “The Language and Thought of the Child”

The last one was useful in the way that children think about things a lot differently than adults, for example, “Are you present?” asked to a child, will most likely result in the child thinking that they are going to get a “present”.  That book was useful for researching where the context can go wrong and WHY

For the output above I am currently using the Stanford NLP Java library.  On top of this sits a bunch of other functions, the most important of which is what I can the Rationalizer.  The Rationalizer basically looks at all the elements of the sentence, clauses, phrases, word type (noun, verb adjective), processes grammar, and looks for semantic relations within the context of the sentence.

It then breaks that down into essentially, subjects, objects, elements, and actions, which from my research are what form 95% of sentences.

The above structure is then very easy to apply logic, functionality and processing.

Interesting that you seem to be approaching the problem from a psychological perspective.

I’d characterise my own approach as purely mechanical. I draw on computational linguistics a great deal, but have been ignoring the current “statistical analysis” fad.

Have been finding a great deal of inspiration here http://www.cs.rochester.edu/~james/ and have the book “Natural Language Understanding” which is probably the single greatest book on the subject. If anyone has accomplished more than the team at Rochester University I have yet to hear about it. (I’m sure somebody has, but it’s probably a top secret project buried in an underground bunker somewhere.) Make sure you take a look at PLOW http://www.cs.rochester.edu/research/cisd/projects/plow/

When I’m not doing NLP I’m doing robotics too. I have another website for that at http://infurl.net and also have a lot of older stuff at http://asmith.id.au

More things to read :D Thanks for the links.

You are indeed correct, I am trying to approach from a psychological angle as much as possible.  There are mechanical aspects to it of course but the “guts” will attempt take reference from the abstract as much as possible.

Aside from the psychological angle there is another aspect that I kind of stumbled across when I was attempted to define influences upon “intelligence” on the white board.

It seems that “cause and effect” have a great deal to do with understanding what has happened and what may happen.  Everything in the universe is driven by cause and effect.  You can not have an effect without an action, and you can not have an action without a cause.  To really throw a spanner in there, you also can not have a cause without an effect, its completely circular in its nature.

With that in mind, if you know the cause and the action, you can determine the effect.  Similarly if you have the cause and the effect, you can attempt to define the action.  What is very interesting here is that if you know the effect (future) and even some of the action, you can predict the cause.

Cause and effect play a large part in the model that I am working towards implementing, and should (I hope) allow a more abstract, and human like method of learning.

ChatScript has NLP technology built in. Currently built in is a POS-tagger, and I’m working on full shallow parsing at present. This supports keyword use in topics where the keyword is correctly tied to part of speech, so buy~v only refers to buy when used as a verb.  It also supports storing phrase data from sentences for representing information the user has volunteered. It can correctly tag and phrase things like: “The baseball player’s malice toward the referee was revealed in his spiteful remarks to the media, which almost ruined the referee’s career.” though it doesnt currently handle “Buffalo buffalo buffalo Buffalo buffalo”, nor “This is the cheese that the rat that the cat that the dog chased bit ate” though I expect it will handle this latter one shortly.

Bruce, I couldn’t help comparing your path to Andy’s just now. From the discussion the other day, Andy started out building NLP software and, faced with the computational constraints that presents, has moved on to a more relaxed pattern matching strategy. You seem to be following the same path in reverse. There never does seem to be a “magic bullet” does there.

I’m still pinning my hopes on strict algorithms. A while back I made the observation that computer chess is dominated by machines and algorithms that take the brute force approach. That is, the ability to evaluate the largest number of possible moves tends to beat the ability to make the smartest moves. (A more recent study which started appearing on the internet in the last few weeks indicates that a combination of brute force and smart moves is even better. If true, it would mirror the real life winning strategy of “talk softly and carry a big stick” and it would certainly seem to be intuitively correct.)

As a result I’m focusing my efforts first on developing the most powerful parsing software that my skills, current research, and available hardware will allow. Andy said that he was using about 600 rules with his GLR parser, but grammars like the English Resource Grammar have 47,000 and XTAG is similar. I’ve observed that having a larger number of more specific rules makes parsers run faster, and so one of the requirements of my software is to be able to support extremely large rule sets efficiently.

Actually, I wouldn’t describe me as doing it in reverse. More simultaneous.  I started with Chat-L, which used the CMU linkparser as parser support. ChatScript dropped that because it was not fast enough for my purposes, and I had to suppress it entirely for sentences of 18 words or longer. I have a definite focus on both capability and performance. I definitely believe you need NLP to properly handle approximations of meaning in a chatbot. Also I feel one needs a proper word ontology. Wordnet only really does nouns passibly (I disagree with bunches of it). SUMO does nouns and verbs passibly (again I have violent disagreements with it), so I have my own noun,verb,adjective,adverb,preposition ontology world.

Hi, Dan, and welcome to chatbots.org!

There are quite a few members here that are working on various aspects of NLP, and there are numerous threads here that deal with the subject. Be advised, however, that while the community at large is both friendly and helpful, we’re also a somewhat passionate lot, so open discussions can sometimes get rather “lively”, at times. Feel free to browse the various topics of discussion, and if you see anything that draws your attention, you are most certainly welcome and encouraged to voice your opinion. We’d love to hear from you.

[edit]This was actually meant to be the “first reply”, but before I could post it, I had to deal with a plumbing emergency, and only just got back to it. Drat!
Anyway, as you can see, many of my statements about the community (friendly, helpful, passionate) have been fairly well born out. :D

Hi Dan,
I also did deep NLP using my neural net. The major drawback of this approach, I found, is the fact that unoptimized path-ways can take an eternity to calculate (but are always optimizable) + rendering output nicely (like patternbased systems do), is hard. So I am now doing a combination of the 2 (which appears to be working better).

It then breaks that down into essentially, subjects, objects, elements, and actions, which from my research are what form 95% of sentences.

This is indeed the important bit.

I have used the Stanford NLP suite (in Java) and found it very useful when I want to parse questions or candidate answers found on the net.  You can find lots of examples of this approach in the OpenEphyra code - they use both Standford NLP and Open NLP for sentence splitting, tagging, parsing and named entity recognition.  While not perfect, I would have thought that your time is better spent working with the output of these already highly-developed libraries to try your new ideas.

I found Mr Smith’s comments interesting… some comments, if I may…

“.....with some half-thought out idea…..”

—That is the nature of research, you don’t know all the answers from the start, if you did, it wouldn’t be research.  If anyone wants to go about their research in an experimental fashion, do they need your permission?  Are you some kind of self appointed authority?  I see, Mr Smith wants to have a FULL ACCOUNT of all your plans for all your subsystems before you are allowed to discuss any single one of them on this forum.  Yes SIR!

“....they claim will change the world…” 

— I haven’t seen any statement like that in the forums.

“...oblivious to the fact that’s its been tried (and failed).” 

— 1. How do you know what others are oblivious to?  I see, you keep records, being the authority, of exactly who did what homework.  2.  You are arrogantly assuming you know what “it” is, and that you know everything about another member’s project to make an educated evaluation, which I doubt is the case.    I think you over-simplify it (easy to see why) and equate it to a previous attempt, and quickly write it off. 

“... dozens of times already….”—I think perhaps, -you- have failed dozens of times already.

“For some reason they get very upset and offended when this is pointed out to them .....”    -  aw, poor baby.

“..and then we never hear from them again..” —-  I see, so you’re assuming the forum member actually –bought- your over-simplified, simple-minded evaluation of their approach and got discouraged?!  I very much doubt that is the case.  Perhaps it hasn’t occurred to you that those members may simply have got enough out of the forums for the time being (and especially from you) and are busy on their project, busy with work, or just enjoying life.  We don’t –have to- log a certain minimum hours on chatbots.org per week/month to keep membership.

oh… last but not least…

“...we get nitwits storming…” ——nice (that’s probably a much higher opinion than others have of you )

Anyway, good luck with y our project Dan, and don’t let the nitwits write your ideas off as “its been tried (and failed)” 

Victor! Great to “see” you!

Of course, it would have been better if your comments weren’t so seemingly defensive. I’m certain that those comments weren’t directed toward you, so I don’t think that getting bothered about them is going to be productive. (Personally, though, I agree with the premises that many of your comments engendered - I would have probably couched them in a bit more diplomatic style, is all. ) Anyway, I’m rather keen to learn how things are going with your project, so if you could update us at your convenience, I would appreciate it.

Dave Morton - Aug 8, 2011:

Victor! Great to “see” you!
[...[Anyway, I’m rather keen to learn how things are going with your project, so if you could update us at your convenience, I would appreciate it.

Seconded!