Relevance

The key measurement in information retrieval is relevance, and the criteria by which relevance is measured are recall and precision. Recall is the percentage of relevant documents retrieved from all documents in the collection. Total recall would locate every document in the collection that matched the search criteria. Precision is the percentage of retrieved documents that are relevant. High precision means that all of the documents retrieved are relevant to the query. And there’s the rub. Who determines what is relevant? Well, ultimately the information seeker, but that also makes it somewhat subjective. Evaluating and measuring the way users judge relevance has been hotly debated in the literature for many years (Schamber, Eisenberg, & Nilan, 1990). But relevance should not merely consider how much “junk” is mixed in with the useful information, for some of that junk might remind the searcher of knowledge, suggest new sources for information, or take the searcher in new and fruitful directions.

Reference

Schamber, L., Eisenberg, M.B., Nilan, M.S. (1990). A re-examination of relevance: Toward a dynamic, situational definition. Information Processing & Management, 26, 755-776.