hello everyone,I have studied Dr Wallaces “Elements of AIML Style” and in the beginning of the book Dr Wallace talks about Zipfs Law.
can anybody explain Simply what is Zipfs Law?

any answers would be appriciated

Zipf’s law states that given some corpus of natural language utterances, the frequency of any word is inversely proportional to its rank in the frequency table.

Dave Morton - Oct 3, 2014:

Zipf’s law states that given some corpus of natural language utterances, the frequency of any word is inversely proportional to its rank in the frequency table.

thank you for your reply Mr Morton.

xxxxxxxxxxxxxxxxxxxxxxxxxxxWHERE
xxxxxxxxxxxxxxxxxxxxxARE
xxxxxxxxxxxxxxxxxYOU
xxxxxxxxxxxIN
xxxxxxxxxxTHE
xxxGRAPH

The above example attempts to illustrate the frequency of words that might be said to a bot. The word at the top, with the highest frequency, is ranked with the smallest number, 1.

The primary use of Zipf’s law with regard to AIML authoring, is to collect the things that people say to your chatbot, and then analyze the frequency of whole sentences. So you do this with whole sentences, and you begin to see what people say most often to the chatbot. This gives you a priority of what to build replies for. You start coding with the most frequently said sentence, and follow down the line until you get tired of typing. It is a very laborious way to program a chatbot.

Thiis was a popular technique in the mid 1990’s, but some people may still adhere to it, or use it in combination with other authoring techniques.

An alternative is subject authoring, where you begin with a topic that you want your chatbot to be able to discuss, and branch outward, adding questions and statements in an attempt to anticipate what other people will ask your chatbot about its subject of interest, building an intention for your chatbot that people will talk about that subject.

Robby.

Thank You Mr Garner.

Additionally Zipf’s Law as applied to the corpus of “Things people say to chatbots” can be used separately in layers of bot development - handling:[ol]
[li]-linquistics: how people communicate with chatbots (question structure, statement structure, relations, goals, specification, generalization, reference, context)[/li]
[li]-domain: what people communicate about (topics, properties and values, methods, time)[/li]
[/ol]

Personally, I have found AIML v1 to quite restrictive for representing linguistic features, as well as restrictive and obfuscating for representing domain knowledge.  It was not easy to separate input understanding, from knowledge representation, and not easy to separate those two from response generation.

AIML v2 with the addition of sets and maps will allow further isolation between linguistics and domain knowledge within AIML. 

It should even be possible to implement a set of AIML CRU(D) statements that use maps to totally isolate knowledge into a database of maps.

Then a zipf topic analysis can guide domain knowledge map writing, and a zipf linguistic analysis can guide input reduction.  We’re on our own for the response generation techniques though.

Alan McDonley - Oct 4, 2014:

Additionally Zipf’s Law as applied to the corpus of “Things people say to chatbots” can be used separately in layers of bot development - handling:[ol]
[li]-linquistics: how people communicate with chatbots (question structure, statement structure, relations, goals, specification, generalization, reference, context)[/li]
[li]-domain: what people communicate about (topics, properties and values, methods, time)[/li]
[/ol]

Personally, I have found AIML v1 to quite restrictive for representing linguistic features, as well as restrictive and obfuscating for representing domain knowledge.  It was not easy to separate input understanding, from knowledge representation, and not easy to separate those two from response generation.

AIML v2 with the addition of sets and maps will allow further isolation between linguistics and domain knowledge within AIML. 

It should even be possible to implement a set of AIML CRU(D) statements that use maps to totally isolate knowledge into a database of maps.

Then a zipf topic analysis can guide domain knowledge map writing, and a zipf linguistic analysis can guide input reduction.  We’re on our own for the response generation techniques though.

thank you Mr McDonley