School of Computer Science and Information Technology,
GPO Box 2476, Melbourne 3001
Tel: +61 3 992 59675
Fax: +61 3 9925 3748
In 2010, I published an extensive review on IR evaluation, focusing on test collection based evaluation. Here you can find a copy of the full paper; the publishers of Foundations and Trends® in Information Retrieval allow authors to put the full PDF of their paper up on their web site for free. The citation for the paper is
Sanderson, M. (2010). Test Collection Based Evaluation of Information Retrieval Systems. Foundations and Trends® in Information Retrieval, 4(4), 247-375. doi:10.1561/1500000009
I created a set of slides that largely follow the order of the article. They were presented at the Yahoo! IR Summer School in Bangalore in 2011. They were split into part one (PPT) (PDF) and part two (PPT) (PDF). If you do use some or all of the slides, please reference the paper.