Not logged in.

Contribution Details

Type Book Chapter
Scope Discipline-based scholarship
Title Considering Human Perception and Memory in Interactive Multimedia Retrieval Evaluations
Organization Unit
  • Luca Rossetto
  • Werner Bailer
  • Abraham Bernstein
  • Jakub Lokoč
  • Tomáš Skopal
  • Klaus Schoeffmann
  • Vasileios Mezaris
  • Xirong Li
  • Stefanos Vrochidis
  • Ioannis Patras
Item Subtype Original Work
Refereed Yes
Status Published in final form
  • English
Booktitle MultiMedia Modeling
Series Name Lecture Notes in Computer Science
ISBN 978-3-030-67831-9
ISSN 0302-9743
Number 12572
Place of Publication Cham
Publisher Springer
Page Range 605 - 616
Date 2021
Abstract Text Experimental evaluations dealing with visual known-item search tasks, where real users look for previously observed and memorized scenes in a given video collection, represent a challenging methodological problem. Playing a searched “known” scene to users prior to the task start may not be sufficient in terms of scene memorization for re-identification (i.e., the search need may not necessarily be successfully “implanted”). On the other hand, enabling users to observe a known scene played in a loop may lead to unrealistic situations where users can exploit very specific details that would not remain in their memory in a common case. To address these issues, we present a proof-of-concept implementation of a new visual known-item search task presentation methodology that relies on a recently introduced deep saliency estimation method to limit the amount of revealed visual video contents. A filtering process predicts and subsequently removes information which in an unconstrained setting would likely not leave a lasting impression in the memory of a human observer. The proposed presentation setting is compliant with a realistic assumption that users perceive and memorize only a limited amount of information, and at the same time allows to play the known scene in the loop for verification purposes. The new setting also serves as a search clue equalizer, limiting the rich set of present exploitable content features in video and thus unifies the perceived information by different users. The performed evaluation demonstrates the feasibility of such a task presentation by showing that retrieval is still possible based on query videos processed by the proposed method. We postulate that such information incomplete tasks constitute the necessary next step to challenge and assess interactive multimedia retrieval systems participating at visual known-item search evaluation campaigns.
Official URL
Digital Object Identifier 10.1007/978-3-030-67832-6_49
Other Identification Number merlin-id:20564
PDF File Download from ZORA
Export BibTeX
Additional Information 978-3-030-67832-6 (E)