Evaluating Categorisation in Real Life – an argument against simple but impractical metrics

Vide Karlsson, Jussi Karlgren, and Pawel Herman

Text categorisation in commercial application poses several limiting constraints on the technology solutions to be employed. This paper describes how a method with some potential improvements is evaluated for practical purposes and argues for a richer and more expressive evaluation procedure. In this paper one such method is exemplified by a precision-recall matrix which exchanges convenience for usefulness.

Presented at the 7th CLEF 2016 Conference and Labs of the Evaluation Forum, 5-8 September 2016, Évora, Portugal.

Paper is here