Not logged in.
Quick Search - Contribution
Contribution Details
| Type | Conference or Workshop Paper |
| Scope | Discipline-based scholarship |
| Published in Proceedings | Yes |
| Title | An Asynchronous Scheme for the Distributed Evaluation of Interactive Multimedia Retrieval |
| Organization Unit | |
| Authors |
|
| Presentation Type | paper |
| Item Subtype | Original Work |
| Refereed | Yes |
| Status | Published in final form |
| Language |
|
| ISBN | 978-1-4503-9497-0 |
| Page Range | 33 - 39 |
| Event Title | MM '22: The 30th ACM International Conference on Multimedia |
| Event Type | conference |
| Event Location | Lisboa Portugal |
| Event Start Date | November 10 - 2022 |
| Event End Date | November 14 - 2022 |
| Place of Publication | New York, NY, USA |
| Publisher | ACM |
| Abstract Text | Evaluation campaigns for interactive multimedia retrieval, such as the Video Browser Shodown (VBS) or the Lifelog Search Challenge (LSC), so far imposed constraints on both simultaneity and locality of all participants, requiring them to solve the same tasks in the same place, at the same time and under the same conditions. These constraints are in contrast to other evaluation campaigns that do not focus on interactivity, where participants can process the tasks in any place at any time. The recent travel restrictions necessitated the relaxation of the locality constraint of interactive campaigns, enabling participants to take place from an arbitrary location. Born out of necessity, this relaxation turned out to be a boon since it greatly simplified the evaluation process and enabled organisation of ad-hoc evaluations outside of the large campaigns. However, it also introduced an additional complication in cases where participants were spread over several time zones. In this paper, we introduce an evaluation scheme for interactive retrieval evaluation that relaxes both the simultaneity and locality constraints, enabling participation from any place at any time within a predefined time frame. This scheme, as implemented in the Distributed Retrieval Evaluation Server (DRES), enables novel ways of conducting interactive retrieval evaluation and bridged the gap between interactive campaigns and non-interactive ones. |
| Free access at | Official URL |
| Official URL | https://dl.acm.org/doi/10.1145/3552467.3554797 |
| Related URLs | |
| Digital Object Identifier | 10.1145/3552467.3554797 |
| Other Identification Number | merlin-id:22773 |
| PDF File |
Download from ZORA
|
| Export |
BibTeX
EP3 XML (ZORA)
|