AI Zone Admin Forum Add your forum

NEWS: Chatbots.org survey on 3000 US and UK consumers shows it is time for chatbot integration in customer service!read more..

RSS AI Zone  Forum Home  >  (Turing) Tests, contests and awards  >  Other tests and contests  >  Thread
78% “pass” rate in a modified version of the Turing Test in a Virtual World
  
 
Posted: Jan 29, 2015

Thought you guys might like this paper on a recent modified Turing Test run in Second Life with a 78% (!) pass rate.

http://www.sciencedirect.com/science/article/pii/S1071581914001049

Our blog post gives more background, context and a link to a possibly free copy of the article.

http://daden.co.uk/conc/blog/2015/01/daden-chatbot-significant-passes-modified-turing-test-virtual-world

Disclosure: We (Daden) came up with the idea of the test (in 2009), and supplied the chatbot engine, but Loyola Marymount University designed the test, authored the bot, ran the test and analysed and published the results.

David
 

 
Posted: Jan 29, 2015
   [ # 1 ]

Nice work. Do you guys still use AIML for your Discourse engine?
 

 
Posted: Jan 30, 2015
   [ # 2 ]

A heavily modified version of it. Not checked out AIML2 yet to see if its worth switching, ditto Chatscript, but we have done some work a more semantic/grammatic parser based engine. Realistically though its probably horses for courses, Discourse version of AIML being great for very fixed domain bots, but something else being needed for a “generalist”, and learning, chatbot.

David
 

 
Posted: Jan 31, 2015
   [ # 3 ]

Not bad. I think this percentage should always go along with mention that people weren’t aware of the possibility that there might be a bot. It disables the “interrogation” aspect, so it’s difficult to compare to other Turing Tests or the benchmark % that Turing described under different conditions. “Covert Turing Test” sounds good to me.
 

 
  login or register to react
‹‹ Charlie Chiemo Awards      Intelligent Assistants Conference 2015 ››