Basic task Two types of queries Relevance assessment scale Other buttons
You have been assigned three queries into specialized areas of artificial intelligence (AI), and will be shown a series of "documents" (thesis abstracts restricted to the area of AI). For each document, with respect to each of the queries, you are to decide if that document is RELEVANT to that query.
To make the exercise concrete, imagine you were working as an AI reference librarian and a library patron has come to you with one of these queries. Imagine also that you had been assigned a single, random file drawer full of documents. You must now go through this drawer to decide which of these documents the patron might be interested in reviewing.
NB: All the patrons' queries are ASSUMING that the search is restricted to AI documents, and are expressing MORE SPECIFIC interests.
The first time you execute RAVE, you will be shown the three queries in the bottom third of the window: two 'short' queries in a WHITE middle strip. The third query, at the very bottom and also with a WHITE background, is a 'long' query corresponding to a document which (you are to imagine) is SIMILAR to documents you seek. Spend a minute reading these queries and getting them in mind; these will remain constant throughout your session.
After having a clear conception of the three queries, read the document at the top of the screen with the LIGHT BLUE background and evaluate its relevance (see below for details). After you've done that WITH RESPECT TO EACH OF THE THREE QUERIES, click on the NEXT DOCUMENT button and evaluate its relevance in the same manner. This cycle will be repeated until you've evaluated all of the documents assigned to you, or you have worked for as long as you are expected.