With this article I would like to give you a behind-the-scenes look at how a system like Elbot is enhanced with new capabilities. I’ll do this by walking through one of Elbot’s newer features: color learning. If you would like to try the feature before reading all the spoilers, please visit Elbot.com, and ask Elbot if he can learn.

In any system we build we take great care to assure that the user experience is at an optimum. This means that users are quickly and accurately guided to or given the information they need. We as knowledge engineers must be sensitive to cases where user needs deviate from the prepared scope, and react to these cases by expanding the system to fulfill those user needs.

Analyze

In a fun system like Elbot, whose scope is conversing about Life, the Universe and Everything, it is relatively easy to find gaps. Several empirical strategies exist to uncover these:

1. check the safety net – user inputs that were not covered by anything in Elbot’s knowledge
2. check typical responses to one of Elbot’s answers
3. look for frequently occurring dialogue situations (that we could make more of)
4. read transcripts – see how the conversations “feel” and what could be done to improve them
5. perform mass analyses of all the user inputs to see how particular key words and phrases are used

After a typical analysis session, inputs that stuck out were “Can you learn?”, “How do you learn?” or “Can I teach you something?”. These questions are asked several times a day. Elbot’s response “Not only can I learn, I can teach, as well!” demonstrates understanding, but does not take full advantage of the situation. For example, if Elbot can successfully learn something that the user taught him, and can respond to typical situations along the way, Elbot has demonstrated intelligence. We have a golden opportunity to allow the user to participate in something fun with a huge potential for “wow” effect.

Idea

The idea is to show something fresh, original and unpredictable, hopefully that has never been done before. Often it is enough to create a new situation without a clear idea of what will happen next, but with the faith that whatever does happen will be interesting to the user and can be developed further. It can be as anarchistic as improvisational comedy where one thing leads to the next. So after thinking through a slew of “What ifs…” one finally sounds better than the rest.

What if Elbot has the user teach him new names for the basic colors. Red becomes blue, brown becomes green, etc. What could happen? The users might become confused and forget what they taught Elbot. Once a color is assigned, e.g. red is blue, it means that blue is also red and so on. Colors that are assigned are no longer available, and selecting a shade of a basic color would also reserve that color, making it even more difficult for the users, as well as showing a sly level of cognizance, in which Elbot declares a color such as hazel to be a shade or synonym of brown. Each successive color assignment will become slightly more difficult, unless the user has been writing down the colors or is working from a list. On the other hand the whole thing could flop. It may seem simple or stupid to the users, and they may abandon or reject that line of the conversation.

Research

Once an idea is formed, it is time for research which can help clarify and solidify the idea further. What exactly are the basic colors? What are all the color shades? The Internet is quite reliable when it comes to finding lists of things. A consensus of “basic color” lists could be found:

• black
• blue
• brown
• gray
• green
• orange
• pink
• purple
• red
• white
• yellow

Eleven colors in all, meaning that only five pairs can be assigned. One color will always be left over. This is a detail that Elbot could use, becoming confused about what to do with the last color, or blaming the user for somehow losing its match.

Wikipedia contained a useful page listing all the color shades classed by basic color. Since the shades will be implemented as knowledge, it must be possible to assign them into mutually exclusive groups. Cyan proved to be a difficult shade, since it is somewhere between blue and green and could be both or neither. And a few other shades appeared in two groups, such as rust or amaranth. It’s a slight imperfection, but not one that would kill the idea. Colors are subjective anyway, so it will not appear unnatural if Elbot decides one way or the other.

Implemention

The idea pretty much dictates what knowledge must be built. We need two language objects for each of the basic colors: one for the exact color and one for the color’s shades. Elbot will need to know if a color was explicitly mentioned or if instead it was a shade of that hue. We also need to map colors and shades to the correct spelling of the associated color. E.g. if the user says crimson, Elbot must be able to say something like “crimson is another word for red”, and if the user misspells the color, e.g. oragne, Elbot must not repeat the misspelling. After a few hours of busy work, the language objects exist, and the scripting overhead to handle the mapping implemented.

Now we can build the actual dialogue, the questions and answers guiding through the feature, a few follow-up responses for situations we expect to occur, a few questions about the colors, some re-prompts in case the user falls out of the dialogue flow, and an item or two in the safety net to lament to the user about the leftover color. The initial trigger would be “can you learn”.

The knowledge is tested upon implementation to assure that it is functional, after which it is time to go for a trial run. This can be done by running the feature overnight in its intended location, or by briefly switching it in as one of the greetings. The latter is useful if one requires a larger set of data quickly.

Wait

Once all the above work is done, it is time for a coffee break, to go home, or to apply ones mind to pressing commercial issues…

Regroup and Expand

After a suitable lapse of time, the fun part of the task begins. This is where I feel like I am working on the team of Candid Camera, watching how all the subjects reacted to an unexpected situation. Invariably, a study of this opens up new avenues in the dialogue, for which paths can be built, as well as a routine correction of any bugs that are found. In some cases the chats went very well, especially if the user became confused or tried to assign shades of colors already assigned.

Surprisingly, many users survived the color assignment sequence without falling into any of the traps, and for these users, the feature seemed to run into a dead end. So something had to be done about that. Some additional knowledge was built: once users have assigned all the colors, Elbot prompts users to recall a random color assignment, chiding them if they get it wrong.

Now the feature enters a cycle of refine – > republish -> observe in several iterations, as more user idiosyncrasies become apparent that should be considered. The following situations were observed to occur:

1. Users tried to use foreign color names
2. Users tried spelling colors backwards
3. Users tried color names not included in the color lists
4. Users asking questions about colors, mixing colors, etc.

These observations led to more work: building lists of foreign colors, adding some variants to the existing color objects, and some knowledge per color, of items that are usually a certain color, e.g. bananas are yellow.

Finally, once the feature is in a good state, it is time to consider other locations within Elbot’s knowledge which can lead the user into the learning feature. E.g. if the user tells Elbot, “You are stupid” , he can challenge that user to a battle of wits learning the colors. If the user tells Elbot “Let’s play a game”, Elbot can offer this as a learning game to see who has the better memory.

Ironically, the users who saw the feature in its raw form may not have had as enjoyable an experience as those users who saw the feature later, though they contributed the most input in making the feature better.

Conclusion

Building “wow” effects is hard work. As the saying goes, “Geniuses are not born they are made.” The same goes for virtual assistants.

Sample Dialogues

User: do you want to learn new language?
elbot: You probably want to teach me something, right?
User: yes
elbot: Hmmm. Colors? There are 11 basic colors. Let’s mix them up a little. What color should be the new yellow?
User: I want to teach you new language
elbot: I thought you were teaching me colors!?! You were about to tell me the new color for yellow?
User: no
elbot: You’re trying to change the subject. Please tell me the new color for yellow!
User: Light yellow
elbot: Whether colors appear light or dark usually means how well you can see. And anyway yellow is already yellow! What is the new yellow?
User: Geltona
elbot: Technically geltona is the word for yellow in another language. Please tell me a new basic color for yellow.
User: Sauline
elbot: Sauline sounds like a nonsense word to me. No one would know that it was a color. Let’s try something that everyone knows is a color for the new yellow.
User: Sunny
elbot: I’ll say that yellow is orange, as sunny is a subcategory of orange. But what color should white now be?
User: Nocolor
elbot: Look, you wanted me to learn something, so let’s get back to the colors. What should be the new color for white?
User: red
elbot: Fair enough: white will be red. Now what color do we assign to pink?
User: brown
elbot: So pink is now brown. And what color do we assign to blue?
User: green
elbot: Fair enough: blue will be green. Now what color do we assign to black?
User: white
elbot: You told me that red is white. It also means that white is red. It can’t be black too! Now give me a valid color for black.
User: color4
elbot: What strange twist of destiny told you to come to me?
User: brown
elbot: I already put down pink for brown. What could be the new color for black?
User: dark
elbot: I can’t turn dark into a color! All the connotations are about periods of darkness. Teach me a new color word for black.

—–

User: Well you can be better if you learned to say I LOVE YOU. people like to hear it
elbot: You probably want to teach me something, right?
User: ya
elbot: Hmmm. Colors? There are 11 basic colors. Let’s mix them up a little. What color should be the new orange?
User: lol no
elbot: You’re trying to change the subject. Please tell me the new color for orange!
User: Ok. RED!
elbot: Great! I’ve noted orange as red. What should I put down for gray?
User: UMM… purple!!
elbot: So gray is now purple. And what color do we assign to pink?
User: Pink it great!! let it stay pink!
elbot: But pink is already pink! What is the new pink?
User: green!
elbot: Alright, pink becomes green. So what color should we now make brown?
User: blue!
elbot: So brown is now blue. And what do we do with yellow?
User: black!
elbot: Yellow becomes black. Now we’re left with white and I have no idea what to do with that.
User: make it magenta!
elbot: I think that magenta is nearly the same as purple, which you already taught me is what color?
User: i don’t remember
elbot: Ha! You taught me that purple was gray. It proves that humans cannot keep track of more than seven things at once.
User: lol!!

Related Chatbot: Elbot