Hi,

I am wondering how much time it takes to make a bot - from zero to hero using AIML, RiveScript, ChatScript and any other non-cloud based platforms (not personalityforge or similiar)..?

Let’s say a person with expertice in the various platforms were to make a complete chatbot with the same functionality.. How long will it take for each?

The reason I ask is because I’m checking out various platforms and need to set a realistic timeframe for the project.

I really doubt that the differences between platforms will be material, compared to the time it takes to learn one and develop a bot, but I will let the advocates of different platforms make their arguments.  I think you’ll need to define “hero” if you want realistic estimates of how long your project could take.  Can you set out the goal / domain of the chatbot?  Are you answering questions about a new product, providing stimulating discussion about the works of Shakespeare or passing the Turing test?

It doesn’t matter which platform you use, really. For someone who is a “master” with the various platforms, creating a great chatbot will be the work of years. Case in point: Mitsuku (which is still, to this day, a work in progress). I’m sure that Noah Petherbridge and Bruce Wilcox (creators of Rivescript and ChatScript, respectively) will tell you the same thing. The “difference” with these three platforms is that with AIML, you can obtain free, pre-made AIML sets that allow you to create a functional, though mediocre (sorry, Rich) chatbot in minutes, whereas with Rivescript and ChatScript, you’re more or less starting from scratch. Given time, I’m sure that there will probably become available some sort of publicly available “core” chatbot framework sets for each of the other platforms, so at that point the playing field will be more level; but for now, true beginners only really have AIML to get them started with a relatively shallow learning curve.

Thank you very much for your inputs!

I get the understanding that there is a lot of work behind building a complete chatbot, and you think that by starting without a “core” framework like the ones of AIML, it would be just as much work with AIML as with RiveScript or ChatScript?

Let’s say my definition of a “hero” is to build a bot that can answer/explain everything about Microsoft Excel. That’s a pretty well-defined goal, and also needs a lot of content, facts, step-by-step descriptions and howto’s.. Would you experts be able to figure out what amount of work that would need to be necessary to make that happen? A “core” framework would not be much help, as the content is something else than the “traditional” chatbots..

There are also a lot of commercial products, like Artificial Solutions that has the IKEA bot “Anna”, and many others.. Anyone know how these “engines” work compared to the platforms mentioned above? Anyone know what they can do that is not possible with the open platforms? Or how much time it takes to make solutions with these compared to the open platforms like AIML, chatscript etc..?

@Dave: I’ve read a lot of your posts on this site, and I really like the way you always give a lot of helpful insights and always keep a good tone with everyone. Just wanted to say you’re doing an amazing job as moderator..

The amount of work involved in completing the scenario you described will vary, based on several factors:

1.) The overall complexity and completeness of the answers you want to provide. After
    all, it’s one thing to be able to rattle off a bunch of terse, factually true but unhelpful
    responses, and something else entirely to provide in-depth explanations of various
    processes.

2.) The extent and variety of “blank” response templates that you have at your
    disposal. To use AIML as an example, without a wide selection of empty code
    snippets (a brief example is at the end of this post), you end up having to
    re-write the same code over and over again. If you’re planning on creating a
    chatbot from scratch, no matter which platform you want to use, first take the
    time to compile a library of code snippets, to make the task easier. The time
    spent with this will be rewarded with interest after only a few uses, believe me.

3.) The usability and overall “user friendliness” of the development environment you
    choose will have a huge impact on the amount of work required to complete your
    project, as will the level of familiarity you have with said environment. If you’re
    most comfortable using Notepad (don’t laugh; I know some masters of the craft
    that use Notepad, and do so quite well), then go for it. Personally, I like my tools
    and snippet libraries within “easy reach”, so I use Bluemental’s WeBuilder to write
    AIML files. When comapred to using Notepad to perform the same coding tasks, I
    probably save at least 40% by using WeBuilder, and maybe even significantly more
    than that.

So as you can see, there’s no “carved in stone” answer for you. you should also bear in mind that planning is easily half of the process, and requires a lot of thought. Good organization skills don’t hurt, either.

Example Blank AIML Snippets:

Robert Jones(#2) - Jan 17, 2013:

I get the understanding that there is a lot of work behind building a complete chatbot, and you think that by starting without a “core” framework like the ones of AIML, it would be just as much work with AIML as with RiveScript or ChatScript?

Each environment has its pros and cons. A veteran botmaster intimately familiar with the platform will produce faster and be more productive than a rookie. There are some things that are possible in one environment that are much more difficult or impossible in another environment. The base AIML set is an advantage and Steve provides a number of AIML sets. But, AIML can also be ported into other environments and back (ie JAN’s AIML Importer, RiveScript to AIML Converter)

Many botmasters use a text editor to create the patterns and responses. There are few real “bot editors”, but many tools built for XML/HTML can be adapted to aid the process.

Some languages feel more “programmatic”, more “XML like” or are more concise. Depending on who is going to do the ongoing maintenance of the bot, this may make a difference to you.

For example, this pattern in one language:
what is my (home|office|cell) [phone] number

Would need to be expanded to:
what is my home phone number
what is my home number
what is my office phone number
what is my office number
what is my cell phone number
what is my cell number
in another language. If you are typing this in manually it takes longer, to input and it is more difficult to maintain.

Part of the decision will be on where/how you want your chatbot to run. Is it for your own desktop use or do you want to deploy it to others? Some of the user interface issues may push you in a specific direction. Do you need simple a “stimulus/response” bot or do you want it to be able to call external functions and do real “AI” like actions.

Robert Jones(#2) - Jan 17, 2013:

Let’s say my definition of a “hero” is to build a bot that can answer/explain everything about Microsoft Excel. That’s a pretty well-defined goal, and also needs a lot of content, facts, step-by-step descriptions and howto’s.. Would you experts be able to figure out what amount of work that would need to be necessary to make that happen? A “core” framework would not be much help, as the content is something else than the “traditional” chatbots..

Yes, it is possible to “quote” the commercial development of a chatbot. But this requires rigid determination of the specifications. There are really 2 components of a chatbot.
- The Chatbot/AI framework (programming)
- The Personality (patterns/responses)

These each require a different skill set.

Robert Jones(#2) - Jan 17, 2013:

There are also a lot of commercial products, like Artificial Solutions that has the IKEA bot “Anna”, and many others.. Anyone know how these “engines” work compared to the platforms mentioned above? Anyone know what they can do that is not possible with the open platforms? Or how much time it takes to make solutions with these compared to the open platforms like AIML, chatscript etc..?

Some of the commercial solutions use the other platforms. Some offer support and hosting. Part of the decision involves time to learn the tools and if you want to do it all yourself. Some of us have built a number of tools to aid in the creation and “data mining” of patterns and responses.

One of these days, when (IF) I get the time, I’d love to combine all of the information here and turn it into an article. But given my current schedule, that’s going to be a ways off. If there are no objections, and if someone else has the time to do so, I think that it should be done, regardless of who actually does the writing.

A few comments on this discussion.

1)  It’s not entirely an “either/or” situation.  I wrote the public interface for my chatbot that looks to the outside world like a Pandorabot (AIML),  but under the covers it sends the user’s response to BOTH an AIML pandorabot and ChatScript.  If Chatscript says it has no good reply,  the interface sends the Pandorabot response back to the user.

That won’t scale up to thousands of users, but I only have a few users so it works just fine.  That way I have the whole AIML basic dialog AND a maintainable ChatScript knowledge-base about the domain I care about.  It “works for now.”

2)  The example question about “knowing everything about Excel” seems to me both ill-formed and misleading.

A question back:  How long would it take someone to simply type out “everything about Excel”,  with no supporting dialog infrastructure at all?   

How long would it take just to type “The Excel Bible” or whatever one of those 1400 page tomes is called?

The problem is somewhat fractal and depends on the user’s state and mental capacity.  How long it would take if every fact had to stand alone on less than a page of text is different from how long it would take if one can determine some expertise and shared context.

So,  if a Chatbot knew exactly what the user already knew, how proficient they were,  what level language they could understand,  what other languages they knew,  etc.,  the answer to a question might only require two sentences, or a link to a web-page.

In other cases, the Chatbot or a wise person might reply ” That very concept is so far beyond you I won’t even begin. It would take two years for you to understand enough to know what the answer entails.”

3)  My programming team documented some legacy systems using a wiki.  Every time we came to a new area, we looked to see what we had down in the wiki, and added a little if there wasn’t enough.  Over a year we developed pretty good documentation for the PART OF THE WORLD WE CARED ABOUT,  in the ORDER WE CARED ABOUT IT.

I can imagine creating a Chatbot similarly,  starting with the most frequently asked questions and working back slowly towards the less frequently asked ones,  at each point slowly but inexorably shifting the technical support burden from human beings on the “help desk” to the automated system. 

Such an approach would never actually “finish”, but be a member of the “Church of our Lady of Perpetual Beta”.
But no matter, it would be very helpful,  not waste time on minor issues,  not waste time on very complex and hard to program dialogs,  and all of it would be somewhat irrelevant within a year when the NEW VERSION of the software came out and all the rules changed.

4)  For something like that, it would seem that a chatbot-assistant who could help a person navigate the existing knowledge bases would be a better plan.  The assistant might not need to understand the details of what is being searched, if what’s stored has good structure.  The assistant might only need to know how to ask: “What part of what you just read was useful?  Highlight the part that was NOT desired.  Give me a few more clues and I’ll write a much better query for you that will zero in on what you want.”

So now the Chatbot isn’t an expert in Excel—it’s an expert in navigating whatever web pages exist at this moment related to Excel, and helping humans create better queries. 

Which, perhaps, is a better approach than the original question.

Anyway, fodder for thought.

Wade

One benchmark we’ve used is that a high quality chatbot requires about 10,000 AIML categories (or the equivalent number of patterns and responses in another scripting language).

A skilled AIML botmaster can reach a productivity rate of about 1 category per minute.  This is not to say that every category takes one minute: some categories may require 20 minutes of thought, other times you can write 20 categories in one minute by cut & paste operations.  But on average, the botmaster might reach a productivity rate of one per minute. 

10,000 minutes = about 1 week of 24x7 or about three 40-hour work weeks.  Adding a fudge factor and time to get up to speed on the platform/language of choice, you should probably plan for one to two months of full time work by one or two creative people.

We’ve been highly focused on creating tools to make “scaling the content wall” more efficient.

* The Superbot Development Kit provides a set of prewritten AIML patterns and allows you to “fill in the blanks” to create responses. 

* The Spellbinder program can generate a good first draft of a bot from conversation transcripts when these are available.

* The new Program AB includes a feature called “Pattern Suggestor/Category Browser” that automatically generates patterns and displays sample inputs with these patterns.  In tests I’ve reached a productivity rate of 6 categories per minute with Program AB.