Widening the Evaluation Net
5071
Paper
published in 2009
by Brian Mac Namee and Brian Mac Namee
Intelligent
Virtual Agent (IVA) systems are notoriously difficult to evaluate, particularly due to the subjectivity involved. From the various efforts to develop standard evaluation schemes for IVA systems the scheme proposed by Isbister & Doyle, which evaluates systems across five categories, seems particularly appropriate. To examine how these categories are being used, the evaluations presented in the proceedings of IVA '07 and IVA '08 are summarised and the extent to which the five categories in the Isbister & Doyle scheme are used is highlighted.