Classification evaluation scenario consists in comparing the labels produced by systems for each item with the value provided by the gold.
The Ranking evaluation scenario focuses on the priority relationships between items in both the gold and system outputs.
Clustering can be interpreted as the problem of predicting if two items belong or not to the same group.
Diversification consists of ranking items according to their relevance while capturing diverse relevance aspects.
EvALL inputs and outputs
A standardized input format
EvALL uses a tsv format in all tasks as standardized input. The number of columns and their interpretation vary depending on the task. EvALL also allows, via wrappers, to use other formats and column separators (tabs, comas, and blanks).