Eulalio: On the first page of this thread I show some examples of the complex sentence splitter (CSS) at work. In general, the program creates a pattern using two input strings, (1) the complex sentence and (2) a string containing all the simple sentences that the complex sentence conveys. It does this by the following method:

a) Identify words common to both strings.
b) Word groups that always appear together in both strings may be grouped together as a single tag. They are only grouped together if they are also either
- a noun phrase (containing adverbs, adjectives, and nouns)
- a verb phrase (containing adverbs and verbs)
- an adjective “phrase” (containing adverbs and adjectives)
- all of one type (all adverbs, all adjectives, etc.)
c) The tag identifier of the word/word group indicates the principle part of speech (in rank order: noun, verb, adjective, adverb) and a unique number, ordered by occurance in the complex sentence.
d) All occurrances of each common word/word group are replaced in each string by the tag. The new strings comprise the pattern.

In order to match a new sentence against the resulting pattern, a combination of reg ex and pos identification is used to try to map the new sentence to the pattern. If the sentence can be mapped, then each tag in the pattern is assigned to a word/word group in the new sentence. The simple sentence pattern is then filled with these words/word groups and the resulting simple sentences are returned.

As you can see, new patterns must be taught explicitly (by inputing the simple sentences). But the system is fairly robust and extendable to applications beyond complex sentence mapping. For example, I intend to build tools to use the CSS to identify not only entire complex sentences, but word groupings within a sentence that convey an independent idea. So, for instance, if the user inputs the sentence,

“The robot’s body is made of metal,”

the bot could use a learned pattern,

“%%n0 ‘s %%n1”—> “%%n0 has a %%n1 .”

to learn the sentence “The robot has a body.”

Bam, possessives learned.

As for the speed of the CSS, I can’t make any definitive comments at this point. The largest list of patterns the bot has tried to search through was no more than 10. Speed was negligible, but this could change once the pattern list grows relatively large. I’ve tried to design the CSS to identify incorrect patterns as quickly as possible to spare computation time, but I’ve yet to really tax the system.

Victor: The quotes were for emphasis, since I couldn’t remember the italics tags off the top of my head. But now I won’t forget them.

Thanks, CR. Some good ideas there. I had more or less come to the same general approach, but wanted to see if any of your specifics were different. I may table conjunctions for now to focus on developing my “topics of conversation” module, whose focus is on finding keywords and specific input patterns that reveal what the conversation is about and what the user’s views on that topic are.

e.g., USER: “I think kant’s views on time and space are ridiculous.” This sentence would match the very common “I THINK (or synonym) [that] X” pattern. It indicates that what follows is a belief of the user. Keywords like Kant, time, and space are also useful for helping STU stay on topic.

Also, good idea for using possessives to draw info from as well. It’s amazing how much knowledge can be gleaned from grammar once you really start thinking outside (or inside) the box defined by a given sentence.

CR: your turn for an update   Or perhaps you have been too busy for ALEX?

Thanks for the interest Victor! I’d like to think I’ve gotten a lot accomplished since updating, but looking at the dates on these posts has reminded me just how fast time flies! It’s been a busy year and I wish I’d had more time for this project. I’ll try to put together an update soon—I’ve got some neat things to share. Mostly I’ve been working on two features:

1) Parser flexibility. Trying to make my parser more robust and capable of handling a wider range of input. Also trying to improve clause and phrase assignation using the knowledge base. However, this got moved to the back burner as I’ve changed and expanded the formatting the knowledge base itself.

2) Rule design. I’ve got a mechanism in place that let’s ALEX test different “actions” that it can take on a sentence to map it to other sentences. When a group of actions works, it is saved as a rule to be tried on similarly constructed sentences.

For example, say I tell ALEX that when I say “The dog ran quickly down the lane.” it should learn that “Dogs can be quick.” The words “quick” and “quickly” constitute different parts of speech and “dog” is singular whereas “dogs” is plural. Even still, ALEX will try the various actions it knows on chunks of the sentence and come up with a “rule” that lets it transform one sentence to the other. So if I later tell ALEX “A man slipped quietly into the night.” then ALEX would guess that “Men can be quiet.”

In the next week (I’ll be on vacation, so my schedule is scattered) I’ll try to generate some actual I/O to post.

Very interesting approach, looking forward to the posts.

Since September last year I have been really focusing on flexiblity as well.  The main thing was handling many modifiers, not just simple terms, but entire phrases.  For example, any number of phrasal modifiers to a noun linked by contractions.

The last 2 months I’ve been working on what I call imagination.  This lets the bot try on its own to expand the rulese ... but with limitations… it could otherwise go right out of hand!!