Contributions published at Dynamic and Distributed Information Systems (Abraham Bernstein)

Early risk detection in food safety aims to identify emerging risks and trends before they can impact the health of consumers. In Switzerland, the FSVO established a network-based process to find microbiological, chemical, and nutritional issues in food, food stuffs, and commodities that might impact the Swiss population in the future. This case study investigates the feasibility of improving the current early detection process by implementing crowdsourcing methods. A series of interviews with people who are involved in the process determined the state-of-the-art and main challenges but left questions about the assessment process of potentially relevant information open. These questions were addressed by an online study that concluded that a crowdsourcing-based information filtering process might be feasible. A literature survey that presents different crowdsourcing implementations in the food domain completes the case study. Following up on the results from the case study, the proposal presents ideas on how the next steps towards an improved early detection process at the FSVO could look like.

The management, analysis and sharing of big data usually involves interacting with multiple heterogeneous remote and local resources. Performing data-intensive operations in this environment is typically a non-automated and arduous task that often requires deep knowledge of the underlying technical details by non-experts. MapReduce box (MRbox) is an open-source experimental application that aims to lower the barrier of technical expertise needed to use powerful big data analytics tools and platforms. MRbox extends the Dropbox interaction paradigm, providing a unifying view of the data shared across multiple heterogeneous infrastructures, as if they were local. It also enables users to schedule and execute analytics on remote computational resources by just interacting with local files and folders. MRbox currently supports Hadoop and ownCloud/B2DROP services and MapReduce jobs can be scheduled and executed. We hope to further expand MRbox so that it unifies more types of resources, and to explore ways for users to interact with complex infrastructures more simply and intuitively.

Graph embedding methods can transform any ontology or graph-like structure into a low-dimensional vector representation. An abundant amount of embedding methods have been proposed to date, and several biomedical networks have shown promising results with the use of such representations. However, the analysis of graph embeddings over an evolving network still remains unexplored. Therefore, we use 17 drug-disease association (DDA) graphs (versions) from an evolving network of the same ontology and apply three established embedding methods. Our approach is to determine the robustness of each embedding method across the evolution by analyzing and comparing the results of two application tasks. We first conduct a local neighborhood comparison of embeddings within the same version, then compare the results across the versions for consistency. Secondly, we use link prediction to find potential associations between drugs and diseases. Here, we compare the performance of each version to the others in order to prove consistency. In addition, we modify the parameters in a task to detect how sensitively the embeddings react to such a change and how it affects the task’s result. This provides a further indication of the robustness of embeddings. Our findings demonstrate that certain versions in the evolution yield a consistent result, and some embedding methods react more strongly to parameter adjustments in a task than others.

null

null

null

null

This part focuses on the use of AI in content curation, addressing the impact of data-driven content recommender systems on diversity and media pluralism. This part and the next one highlighting shortcomings of AI-based content curation and targeted advertising provide human rights-centred recommendations to prevent the negative impact of AI tools in content curation on the right to freedom of opinion and expression.

One of the recent notions of privacy protection is Differential Privacy (DP) with potential application in several personal data protection settings. DP acts as an intermediate layer between a private dataset and data analysts introducing privacy by injecting noise into the results of queries. Key to DP is the role of ε – a parameter that controls the magnitude of injected noise and, therefore, the trade-off between utility and privacy. Choosing proper ε value is a key challenge and a non-trivial task, as there is no straightforward way to assess the level of privacy loss associated with a given ε value. In this study, we measure the privacy loss imposed by a given ε through an adversarial model that exploits auxiliary information. We define the adversarial model and the privacy loss based on a differencing attack and the success probability of such an attack, respectively. Then, we restrict the probability of a successful differencing attack by tuning the ε. The result is an approach for setting ε based on the probability of a successful differencing attack and, hence, privacy leak. Our evaluation finds that setting ε based on some of the approaches presented in related work does not seem to offer adequate protection against the adversarial model introduced in this paper. Furthermore, our analysis shows that the ε selected by our proposed approach provides privacy protection for the adversary model in this paper and the adversary models in the related work.

Updates on ontologies affect the operations built on top of them. But not all changes are equal: some updates drastically change the result of operations; others lead to minor variations, if any. Hence, estimating the impact of a change ex-ante is highly important, as it might make ontology engineers aware of the consequences of their action during editing. However, in order to estimate the impact of changes, we need to understand how to measure them. To address this gap for embeddings, we propose a new measure called Embedding Resemblance Indicator (ERI), which takes into account both the stochasticity of learning embeddings as well as the shortcomings of established comparison methods. We base ERI on (i) a similarity score, (ii) a robustness factor $\hatμ $ (based on the embedding method, similarity measure, and dataset), and (iii) the number of added or deleted entities to the embedding computed with the Jaccard index. To evaluate ERI, we investigate its usage in the context of two biomedical ontologies and three embedding methods---GraRep, LINE, and DeepWalk---as well as the two standard benchmark datasets---FB15k-237 and Wordnet-18-RR---with TransE and RESCAL embeddings. To study different aspects of ERI, we introduce synthetic changes in the knowledge graphs, generating two test-cases with five versions each and compare their impact with the expected behaviour. Our studies suggests that ERI behaves as expected and captures the similarity of embeddings based on the severity of changes. ERI is crucial for enabling further studies into impact of changes on embeddings.

There is a growing use of intelligent systems to support human decision-making across several domains. Trust in intelligent systems, however, is pivotal in shaping their widespread adoption. Little is currently understood about how trust in an intelligent system evolves over time and how it is mediated by the accuracy of the system. We aim to address this knowledge gap by exploring trust formation over time and its relation to system accuracy. To that end, we built an intelligent house recommendation system and carried out a longitudinal study consisting of 201 participants across 3 sessions in a week. In each session, participants were tasked with finding housing that fit a given set of constraints using a conventional web interface that reflected a typical housing search website. Participants could choose to use an intelligent decision support system to help them find the right house. Depending on the group, participants received a variation of accurate or inaccurate advice from the intelligent system throughout each session. We measured trust using a trust in automation scale at the end of each session. We found evidence suggesting that trust development is a slow process that evolves over multiple sessions, and that first impressions of the intelligent system are highly influential. Our results echo earlier research on trust formation in single session interactions, corroborating that reliability, validity, predictability, and dependability all influence trust formation. We also found that the age of the participants and their affinity with technology had an effect on their trust in the intelligent system. Our findings highlight the importance of first impressions and improvement of system accuracy for trust development. Hence, our study is an important first step in understanding trust development, breakdown of trust, and trust repair over multiple system interactions, informing improved system design.

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed.

Gendered voice based on pitch is a prevalent design element in many contemporary Voice Assistants(VAs) but has shown to strengthen harmful stereotypes. Interestingly, there is a dearth of research that systematically analyses user perceptions of different voice genders in VAs. This study investigates gender-stereotyping across two different tasks by analyzing the influence of pitch (low, high) and gender (women, men) on stereotypical trait ascription and trust formation in an exploratory online experiment with 234 participants. Additionally, we deploy a gender-ambiguous voice to compare against gendered voices. Our findings indicate that implicit stereotyping occurs for VAs. Moreover, we can show that there are no significant differences in trust formed towards a gender-ambiguous voice versus gendered voices, which highlights their potential for commercial usage.

For research results to be comparable, it is important to have common datasets for experimentation and evaluation. The size of such datasets, however, can be an obstacle to their use. The Vimeo Creative Commons Collection (V3C) is a video dataset designed to be representative of video content found on the web, containing roughly 3800 hours of video in total, split into three shards. In this paper, we present insights on the second of these shards (V3C2) and discuss their implications for research areas, such as video retrieval, for which the dataset might be particularly useful. We also provide all the extracted data in order to simplify the use of the dataset.

News diversity in the media has for a long time been a foundational and uncontested basis for ensuring that the communicative needs of individuals and society at large are met. Today, people increasingly rely on online content and recommender systems to consume information challenging the traditional concept of news diversity. In addition, the very concept of diversity, which differs between disciplines, will need to be re-evaluated requiring an interdisciplinary investigation, which requires a new level of mutual cooperation between computer scientists, social scientists, and legal scholars. Based on the outcome of a interdisciplinary workshop, we have the following recommendations, directed at researchers, funders, legislators, regulators, and the media industry: - Conduct interdisciplinary research on news recommenders and diversity. - Create a safe harbor for academic research with industry data. - Strengthen the role of public values in news recommenders. - Create a meaningful governance framework for news recommenders. - Fund a joint lab to spearhead the needed interdisciplinary research, boost practical innovation, develop reference solutions, and transfer insights into practice.

null

In the research of video retrieval systems, comparative assessments during dedicated retrieval competitions provide priceless insights into the performance of individual systems. The scope and depth of such evaluations are unfortunately hard to improve, due to the limitations by the set-up costs, logistics, and organization complexity of large events. We show that this easily impairs the statistical significance of the collected results, and the reproducibility of the competition outcomes. In this article, we present a methodology for remote comparative evaluations of content-based video retrieval systems and demonstrate that such evaluations scale-up to sizes that reliably produce statistically robust results, and propose additional measures that increase the replicability of the experiment. The proposed remote evaluation methodology forms a major contribution toward open science in interactive retrieval benchmarks. At the same time, the detailed evaluation reports form an interesting source of new observations about many subtle, previously inaccessible aspects of video retrieval.