The concept of relevance has been heatedly debated in last decade. Not satisfied with the narrow and technical definition of system relevance, researchers turn to the subjective and situational aspect of this concept. How does a user perceive a document as relevant? The literature on relevance has identified numerous factors affecting such judgment. Taking a cognitive approach, this study focuses on the criteria users employ in making relevance judgment. Based on Grice’s theory of communication, this paper proposes a five-factor model of relevance: topicality, novelty, reliability, understandability, and scope. Data are collected from a semi-controlled survey study and analyzed following a psychometric procedure. The result supports topicality and novelty as the key relevance criteria. Theoretical and practical implications of this study are discussed.

This paper provides a comprehensive literature review in the area of modelling relevancy. It was identified from this review that there currently is no consensus amongst authors in the set of factors and the definition of the factors that attribute to the relevancy of objects (e.g. documents) for a user. The authors adopt concepts from Grice's theory of communication to develop a five-factor model for modelling relevancy. It was found that these factors correspond well to findings in existing relevance research. The factors selected and defined by the authors include:

1. Topicality - the extent to which the retrieved document is related to a user's current topic of interest as perceived by the user
2. Reliability - the degree that the content of a retrieved document is perceived to be true, accurate, or believable
3. Understandability - the extent to which the content of a retrieved document is easy to read and understand as perceived by a user
4. Novelty - the extent to which the content of a retrieved document is new to the user or different from what the user has known before
5. Scope - the extent to which the topic or content covered in a retrieved document is appropriate to the user's need, i.e., both the breadth and depth o the document are suitable

Two studies were conducted on a sample of undergraduate and graduate students in a major Southeast Asia university. A questionnaire was developed, which included questions relating to the five factors and taken by selected students. It was discovered that the topicality and novelty factors were found to be statistically significant to relevance judgement while the others were not. It was also found that topicality was the major affecting factor while novelty was the second most important.

I believe that if the authors conducted a similar experiment on a much larger population, they might be able to find some statistical correlation between the other three factors and relevance. Additionally, I had previously considered implementing a relevance factor into a user contribution measurement (UCM) model. However, in order for this to be effective, it may require the implementation of user feedback by rating the relevancy of a particular article or discussion for a user. If numerous users find a particular article/discussion relevant then users that have contributed content towards the topic would receive additional contribution points. I might also be able to apply a similar experiment to what is presented in this paper to validate a developed UCM model.

• Grice's theory of communication
• Psychological relevance
• Situational relevance
• Objective and system-determined relevance
• Psychometrics