Not logged in.

Contribution Details

Type Conference or Workshop Paper
Scope Discipline-based scholarship
Published in Proceedings Yes
Title A System for Interactive Multimedia Retrieval Evaluations
Organization Unit
  • Luca Rossetto
  • Ralph Gasser
  • Loris Sauter
  • Abraham Bernstein
  • Heiko Schuldt
Presentation Type paper
Item Subtype Original Work
Refereed Yes
Status Published in final form
  • English
Page Range 385 - 390
Event Title International Conference on Multimedia Modeling
Event Type conference
Event Location Prague
Event Start Date July 22 - 2021
Event End Date July 24 - 2021
Publisher Springer
Abstract Text The evaluation of the performance of interactive multimedia retrieval systems is a methodologically non-trivial endeavour and requires specialized infrastructure. Current evaluation campaigns have so far relied on a local setting, where all retrieval systems needed to be evaluated at the same physical location at the same time. This constraint does not only complicate the organization and coordination but also limits the number of systems which can reasonably be evaluated within a set time frame. Travel restrictions might further limit the possibility for such evaluations. To address these problems, evaluations need to be conducted in a (geographically) distributed setting, which was so far not possible due to the lack of supporting infrastructure. In this paper, we present the Distributed Retrieval Evaluation Server (DRES), an open-source evaluation system to facilitate evaluation campaigns for interactive multimedia retrieval systems in both traditional on-site as well as fully distributed settings which has already proven effective in a competitive evaluation.
Related URLs
Digital Object Identifier 10.1007/978-3-030-67835-7_33
Other Identification Number merlin-id:20801
PDF File Download from ZORA
Export BibTeX