Hi all,

I’ve almost completed a chatbot in C# by using AIML. Now I am going to employ some sort of dictionary (like WordNet) so as to make the learning process a bit easier. The learning process is still, however, difficult so I decided to ‘extract’ some knowledge from somewhere. My choice is Wikipedia.

So what I need to know is your thoughts about using information from Wikipedia in NLP and finally in my chatbot. I mean I can fetch a page from there (lets suppose its about ‘Sodium’). I get the text, remove HTML from it and can have the pure text. Now the problem is exactly how do I USE it…

Any ideas?

Regards,
Yasir

One good place to look for ideas on how to do this is the YAGO-NAGA project which extracts useful facts from Wikipedia in a format that is very easy to use in other programs.

“YAGO is a huge semantic knowledge base. Currently, YAGO knows more than 2 million entities (like persons, organizations, cities, etc.). It knows 20 million facts about these entities. Unlike many other automatically assembled knowledge bases, YAGO has a manually confirmed accuracy of 95%. YAGO is part of the YAGO-NAGA project at the Max-Planck Institute for Informatics in Saarbrücken/Germany.”

http://www.mpi-inf.mpg.de/yago-naga/yago/

Thanks Adrew. I’ll see this.

@Muhammad: welcome to the Chatbots.org AI Zone forums!

Many of the Loebner Prize winners and participants like Cleverbot and Chip Vivant make use of WikiPedia. Simply because a question like: ‘who was the lead singer of the Beatles’, should be answered right away, it is considered common sense knowledge.

I don’t know precise how it works, but Wikipedia pages are quite standarized so it shouldn’t be that difficult. To handle large volumes like Cleverbot, that’s the main challenge.

I can ask the creators if you like. A complete profile with picture will definitely help.

@Andrew: cool link! We definitely should use these kind of knowledge when we’re setting up our Chatbots.org - chatbot revealing all information about chatbots.

Erwin Van Lun - Dec 25, 2010:

@Muhammad: welcome to the Chatbots.org AI Zone forums!

Thanks! I hope I’ll get great help for my chat bot from here.

Erwin Van Lun - Dec 25, 2010:

I can ask the creators if you like. A complete profile with picture will definitely help.

Yes, please. Nothing can help me better than this.

The Loebner Prize contest does not allow connection to the internet. You need to extract the conversational aspects from Wikipedia into your chat bot’s database before entering that contest.

Wikipedia contains encyclopedia data, not common sense knowledge. Knowing who is the lead singer of the Beatles is not common sense. Eating when you are hungry, sleeping when you are tired, being able to tie your shoes is common sense. Knowing when to look something up using Wikipedia is like common sense. Knowing that George Washington was the first president of the United States of America is general knowledge, not common sense.

Instead of crawling Wikipedia, you might try http://wiki.dbpedia.org/About for the extracted data.

you’re right Gary, I was using ‘common sense knowledge’ in the wrong way. Thanks for the correction! (learning all the time…. )

Muhammad,
My bot Skynet-AI) has interfaces to Wikipedia and a variety of search engines. Part of its core design was to answer factual questions. It might give you some ideas of how a chatbot can be used with a wiki or search engine.

I agree with Gary that either Dbpedia or Freebase (http://www.freebase.com/) might be a better source for basic facts.

For common sense knowledge you may want to look at CYC (http://www.cyc.com/cyc/opencyc).

I have posted a post in another thread regarding to a FYP. Actually I wanted a more general bot.But my supervisor advised me not to go for it.He asked me to select a particular domain.
I have two ideas.
      1.Creating a bot for java question answering.
      2.Question answering bot.

        I went through above knowledge bases and they consists with APIs. Those API’s easier NLP. Isn’t it?.My main goal should be go some extend deep it into NLP. So how I should do this.I wasted few days searching for a solution but here I don’t have time now.I should report him telling what is my final decision and project proposal.

I would like to know a way to do one task from above with learning NLP to some deep.How find a knowledge to my bot.

@Charith: would you be interested to work with me on a chatbots-chatbot > This chatbot should know everything about AI, NLP, virtual agents/converstional agents, virtual humans (animation) etc. We can host the chatbot on Chatbots.org and ask people to contribute content.
What do you think?

Thanks a lot about it Mr.Erwin.There is nothing more than to a beginner.And thanks about the idea.I am continuing with this . As I told earlier the only problem is about knowledge sources.May be some one have a good idea with knowledge sources for each.Thanks.

I ran across this (1996) book in the library: Developing Natural Language Interfaces, by Russell Suereth. It has a lot of example dialogs that I would like to get my bot to do