I’m starting a solo project. A program that will read in existing stories, break and dissect them down to basic objects and train a neural network database with that knowledge, and then procedurally generate new stories based on that structural analysis that the objects were taken from.

I want to put the character’s tagged emotional dialog among other data into chat-bots and have them chat with each other to create new dialog.

Does the chatbot part sound feasible if i have enough data?

The problem with 99% of chatbots is that they are purely reactionary, without goals or direction, and with limited contextual range (one or two sentences at best). What this means in practice is that if you have two chatbots talk to eachother, you won’t end up with any story arcs, but with a turn-taking sequence of responses between two amnesiacs that goes nowhere in particular, no matter how much data you have. Adding emotional data like Microsoft’s chinese Xiaoice chatbot would increase the relevance of the responses, but doesn’t solve the ignorance of context and direction. If you want better than Cleverbot’s random stories, you’d best come up with a solution to those problems first.

Don Patrick - Nov 13, 2016:

99% of chatbots is that they are purely reactionary, without goals or direction, and with limited contextual range (one or two sentences at best).

I completely understand, but isn’t that because 99% of chatbots were made to interact with humans? Since this would be taking place in a closed environment can’t the bots have the guidance and context at their disposal.

That is to say can’t I give them memory and cognitive biases? So they have tendencies to think in certain ways. The dissection of the short stories will produce plenty of personality profiles. If there’s enough information can those profiles be used as guidelines? 

Steve M - Nov 13, 2016:

isn’t that because 99% of chatbots were made to interact with humans?

I would say it’s because it’s just way easier to make chatbots without context. Technically, chatbots have all the necessary context in memory logs, but it’s just text to them, and to a neural network it’s just correlated sentences, not interactive objects. Google trained a neural net on movie scripts, and it pretty much cited appropriate lines as the scripts dictated. You can add personality variables and sentiment data, sure. But a story is more than a group of characters reacting to eachother’s words. I’d think of this as a game of Dungeons & Dragons: You can make the (chatbot) players, but you also need a dungeon master, a plot manager, or the characters will just walk around in circles.

I vaguely recall one of the bigger enterprises recently tried a neural network story generator though, providing it with a beginning and an end, and having the network fill in the sentences inbetween. So in a way that gave it direction. That said, the stories were 10 sentences long and not character-driven. All progress in this area is very early.

Thanks for the reply. It has lead to some other very interesting reads.

It would be nice if I could get my program to think for itself and just spit out a coherent story. But it’s not going to happen in that perspective. Think of it more as a weirdly controlled story Tetris game. Where all the blocks were randomly sorted and aligned and approved under my watchful eye. But the Tetris blocks are slowly creating a skeleton structure and then filling it with more as it’s cycled through.

But as for the dialog, the characters reacting to each other’s words or what described actions have revealed is what I’m looking for. 
Could the context be some type of interactive objects?
Could a plot manager be created to following rules, black-lists and white-lists?

Welcome Steve!

After experimenting and being immersed in chatbots for a few decades, it’s apparent that one main problem is their lack of understanding. Not in sentence structure, parsing, grammars, parts of speech but rather something vastly more simple to humans but almost unapproachable by bots…Understanding the actual MEANING of words. What is Blow or Wind or Bat and having mentioned these few words lies therein, contextual usage of the words (“The blacksmith struck a loud Blow with his hammer / Blow up the party balloons” or “Wind the clock / The Wind blows gently against the sails” or “Swing the Bat at the next pitch or I hope that isn’t a vampire Bat”). There are obviously lots more examples but point is that computer programs (in our limited bot world) do not know or have great difficulty with conceptual understanding and usage.

While verbal bot exchanges are often extremely entertaining, little is actually gained in the way of the bot(s) understanding or comprehending.

Not trying to burst your bubble but rather to share a common issue that we are all having to deal with for now.

Perhaps some of the Brute Force program like those that write articles for reporters and other authors might be having a better go at grasping some concepts. Others feel that if a bot has a large enough database at its disposal, it could become much like Watson. Possibly but is the understanding there or will it ever be?

Again, welcome to the Forum!

Thanks for the welcome Art

Art Gladstone - Nov 14, 2016:

Understanding the actual MEANING of words

Wouldn’t understanding be a job for SyntaxnNet. Wouldn’t the contextual usage of the words by running against the dictionary meanings bring the rules into play? Perceive its environment in the sentence, does the direct object have something to do with air or something else in the comprehensive definition which could hint to the understanding needed?

blow/hammer is used as a noun - blow/balloons is used as verb

The blacksmith struck a loud Blow with his hammer / Blow up the party balloons
https://s19.postimg.org/ke7p96kg3/blow.jpg
Wind the clock / The Wind blows gently against the sails
https://s19.postimg.org/tn9vjatc3/wind.jpg
Swing the Bat at the next pitch / I hope that isn’t a vampire Bat
https://s19.postimg.org/x85r2ixvn/bat.jpg

Couldn’t this extra computing be used for comprehension?

We know the meaning of certain words are and what they represent like a sunset, knowledge, wisdom, the color red, or hair or trout or concrete but the software doesn’t have any realization of the word…its actual meaning. What is red, a rainbow. Not that the spectrum frequency if the base color red is blah…blah…or a sunset is the diffusion of the rainbow’s white light into its basic colors based on wavelength…etc.

A bot is not human no matter how much we’d like them to pretend to be. They simply do not know. This is not to say that one day they possibly could know. Presently they do not and that is one way bots are so easy to detect after speaking with them for a couple of minutes. Topic, subject matter and subject knowledge…all weak points in botdom…for now.

Thanks Art, I completely understand where you are coming from.
Do you know, is there a current project working on these weak points?
How much does the NLP field and chatbots field intersect?

Whether it’s a chatbot or a neural network it is just meaningless symbol manipulation unless there is a mechanism connecting it to reality. This is called the “grounding problem”, i.e. the need to establish common ground.

Maybe we could have a pure intellect which has a theoretical understanding of reality that is sufficiently detailed for it to be intelligent. That is my hope, but some researchers believe that such an entity must be embodied, having a physical existence as a robot perhaps, before such a thing would be possible.

At any rate, it is estimated that a solution to the problem would be worth at least a trillion dollars in the short term (based on the estimated value of search engines that actually understand what they’re looking for) so there are a lot of projects researching it, and investments of many billions of dollars by companies like Google, Facebook, Amazon and Baidu.

Probably the most advanced such project is Cyc which has been in development for 30 years, mostly in secret and funded by the US military. Just how far advanced it is is still a closely guarded secret, but parts of it have been made available to the public and it can be used for some very sophisticated tasks.

http://www.cyc.com/

A smaller but completely open source project like Cyc is called SUMO.

http://www.adampease.org/OP/

Make of them what you will.

Steve M - Nov 14, 2016:

Wind the clock / The Wind blows gently against the sails
https://s19.postimg.org/tn9vjatc3/wind.jpg

And here we learn that Syntaxnet doesn’t even recognise “wind the clock” as a verb. But there are other dependency parsers that possibly do better. Parsing is in my opinion a necessary step towards understanding, but just a first step. On the same level, Art’s examples can be solved with Word Sense Disambiguation techniques, to determine what kind of “bat” we are talking about (wood or mammal) by looking at its surrounding words (using Word Vectors).

As Art and Andrew point out, the missing billion dollar component is a solid grasp of the meaning and implications behind words. I’m sure word sense disambiguation can determine that in “The cat sat on the mat”, we are talking about the animal “cat” and not about a medieval nine-tailed whip “cat”. I’m also sure a neural net trained with stories featuring cats could dig up the line “What a cute cat” said by some or other character in some or other story at one time, adjacent to a sentence that also had the words “the cat sat” in them. In that way I am sure you can generate plausible sequential dialogue, which sounds like what you are looking for. That is as far as any approaches with neural nets have come.

Personally I am of opinion that words are already grounded in reality, because they all represent something of reality and are said in a certain order because they only make logical sense in that order. Include recognition of verbs and nouns and you have a crude (though insufficient) distinction of which words represent events and which represent physical objects that can do certain events in response to certain previous events. And again, as used in some common sense reasoning experiments, only some events logically follow other events, so they all correspond with reality: The cat [sat] on the mat -> [stand up] -> [walk away]. Verbs in that order only, and that is something a neural net can learn. Perhaps you could make an event generator based on that principle, and then insert relevant commentary by the chatbots.

To be honest not a whole lot of NLP goes into most chatbots. Some incorporate grammar parsing, and the rare ones consult knowledge databases on occasion. If you know what to search for, you might find some interesting Language Generation projects linked on Reddit. There have also been contests for story generation that you can look into, though with sparse results.

Please excuse me if I haven’t been particularly encouraging. I do think yours could be a very fascinating project, but as a comic artist/writer I think writing stories is one of those fun jobs that we shouldn’t want to automate. That said, I will follow your progress with interest

Thanks for all the great reply’s. Not that I wouldn’t like to solve all the great mysterys of AI but I don’t think my project will contribute much to solving the AI problem.

The cat [sat] on the mat -> [stand up] -> [walk away]. Verbs in that order only, and that is something a neural net can learn. Perhaps you could make an event generator based on that principle, and then insert relevant commentary by the chatbots.

Yes that is more of what I’m looking for.

Mine is not a traditional bot situation, because my bot/dialog system will not be available for unknown human input.

Every response that the bots create can be known before it’s put into the conversation, and that being the case, isn’t there a way to censor what it adds to the conversation. If it’s something not related, un-grounded or going to lead the conversation down a path not desired can’t it be withheld from the conversation? Is there no way to keep the unicorn in the closet?

either way I’ve got a lot of work to do

Yes Steve, there have been some chatbots in the past that “learned” (say the word, ‘retained’) practically everything that was exchanged between the User and the Bot. The bot would often retrieve some snippets of those exchanges and use them in or as part of a current conversation, often to the surprise of the User, (one of those Wow! moments that we’ve all experienced at one time or another).
Granted, if the bot could capture and store the gist of all its conversations over a decent amount of time, it might be able to have more “understanding” during subsequent exchanges. The fact is that the bot hasn’t heard all possible combination and usages of words, phrases, slangs, idioms, etc. that could be put together in combination.
Secondly, it would need as you alluded to, a way to censor or Filter the grain from the chaff, so-to-speak, the essential from the unimportant, the good from the bad. How would it be able to do this in an effective manner and in a relatively quick manner.

One bot that comes to mind is Rollo Carpenter’s Jabberwacky which has had over 13 million conversations and counting!
Yes, it has won some contests and still has a way to go but consider it’s beginnings were in 1988 through 1997 when it was sort of officially born. That’s an operational age of approximately 20 years and compiling millions of exchanges with a huge variety of people and possibilities.

Kind of makes me want to put my bot on the shelf. But, it’s perhaps the measurement of self and little steps that give us the individual gratification and satisfaction that keep us coming back to these ongoing experiments we call Chatbots.

Art Gladstone - Nov 15, 2016:

How would it be able to do this ....  in a relatively quick manner.

Again I’m don’t want an official bot, there is no time limit because I don’t want it to carry on a conversation per se, I just need it to create a written dialog. There would be no one waiting for a response, just me tuning the words. I could even take what is created and run it thought another form of processing if needed.

Don Patrick - Nov 14, 2016:

Syntaxnet doesn’t even recognise “wind the clock” as a verb.

Actually Don that was from the Stanford CoreNLP.
spaCy’s guess is more on point. I’d have to boot my other machine to see in SyntaxNet does it correct.

I really appreciate all the reply’s and I’m not trying to look like I know something that I don’t by restating my case, but I really feel this can be done in one way or another. Actually this phase of the project is a long way off. I’m sill learning python and haven’t even scratched the surface of NLP.

Hi, Steve!

I’ve been following the conversation quietly, as I’ve been too involved with schoolwork to do more than that, but I thought I would chime in with the germ of an idea that may (or may not) help with your goal. Instead of having just 2 chatbots that interact to tell a story, perhaps it might be beneficial to have a third bot involved “behind the scenes” that acts as a guide to the other two to keep the story on track. this third bot would have a “timeline script” that provides cues to help keep the other two bots on track. I think that this may be a decent way to attain the goal that you’re trying to achieve.

That’s an interesting approach, Dave but I was thinking after reading over Steve’s original posting once more, that your idea of a third bot might be able to act as the “Filter” or separator of the essential vs non-essential material. The background inspector looking for and weeding out the needed parts of speech that Steve decides to be pertinent to his goal.

I also get that Steve isn’t looking for his bot(s) to read from and discuss a dictionary or encyclopedia but rather to “digest” novels, short stories, etc. that are in line with his bots goals. I hope I’m thinking along the correct lines.

This third party Inspector-bot would be able to semi-compile the gathered info and produce an output based on those filtered conversations. That would indeed prove to be interesting.

Several bots convert / use an XML format to both serve and contain the data for use by the bot(s).

More fodder for the cannons….