Contributions published at Department of Informatics (Burkhard Stiller)

Online customer feedback management plays an increasingly important role for businesses. Yet providing customers with good responses to their reviews can be challenging, especially as the number of reviews grows. This paper explores the potential of using generative AI to formulate responses to customer reviews. Using advanced NLP techniques, we generated responses to reviews in different authoring configurations. To compare the communicative effectiveness of AI-generated and human-written responses, we conducted an online experiment with 502 participants. The results show that a Large Language Model performed remarkably well in this context. By providing concrete evidence of the quality of AI-generated responses, we contribute to the growing body of knowledge in this area. Our findings may have implications for businesses seeking to improve their customer feedback management strategies, and for researchers interested in the intersection of AI and customer feedback. This opens opportunities for practical applications of NLP and for further IS research.

When introducing unfamiliar Artificial Intelligence (AI)-based systems, such as conversational agents (CAs), one needs to ensure that users interact with them according to their design. While past research has studied single-user environments, many practical settings involve multiple parties. This study addresses this gap and focuses on financial advisory service encounters and how mental models evolve in multi-party contexts. A multimodal interactive CA is developed and tested in financial consultations with 24 clients. The observations of these consultations and subsequent interviews provide insights into the challenges of using CAs in unfamiliar contexts. The clients have difficulties effectively using the system. This is linked to the institutional setting of financial advisory service encounters and a mismatch between the designer’s conceptual model and the client’s mental model, which we call secondary mental model.

Current approaches to Argument Mining (AM) tend to take a holistic or black-box view of the overall pipeline. This paper, in contrast, aims to provide a solution to achieve increased performance based on current components instead of independent all-new solutions. To that end, it presents the Deployment of Recombination and Ensemble methods for Argument Miners (DREAM) framework that allows for the (automated) combination of AM components. Using ensemble methods, DREAM combines sets of AM systems to improve accuracy for the four tasks in the AM pipeline. Furthermore, it leverages recombination by using different argument miners elements throughout the pipeline. Experiments with five systems previously included in a benchmark show that the systems combined with DREAM can outperform the previous best single systems in terms of accuracy measured by an AM benchmark.

Cybersecurity solutions have shown promising performance when detecting ransomware samples that use fixed algorithms and encryption rates. However, due to the current explosion of Artificial Intelligence (AI), sooner than later, ransomware (and malware in general) will incorporate AI techniques to intelligently and dynamically adapt its encryption behavior to be undetected. It might result in ineffective and obsolete cybersecurity solutions, but the literature lacks AI-powered ransomware to verify it. Thus, this work proposes RansomAI, a Reinforcement Learning-based framework that can be integrated into existing ransomware samples to adapt their encryption behavior and stay stealthy while encrypting files. RansomAI presents an agent that learns the best encryption algorithm, rate, and duration that minimizes its detection (using a reward mechanism and a fingerprinting intelligent detection system) while maximizing its damage function. The proposed framework was validated in a ransomware, Ransomware-PoC, that infected a Raspberry Pi 4, acting as a crowdsensor. A pool of experiments with Deep Q-Learning and Isolation Forest (deployed on the agent and detection system, respectively) has demonstrated that RansomAI evades the detection of Ransomware-PoC affecting the Raspberry Pi 4 in a few minutes with >90% accuracy.

This paper presents findings of the eleventh Video Browser Showdown competition, where sixteen teams competed in known-item and ad-hoc search tasks. Many of the teams utilized state-of-the-art video retrieval approaches that demonstrated high effectiveness in challenging search scenarios. In this paper, a broad survey of all utilized approaches is presented in connection with an analysis of the performance of participating teams. Specifically, both high-level performance indicators are presented with overall statistics as well as in-depth analysis of the performance of selected tools implementing result set logging. The analysis reveals evidence that the CLIP model represents a versatile tool for cross-modal video retrieval when combined with interactive search capabilities. Furthermore, the analysis investigates the effect of different users and text query properties on the performance in search tasks. Last but not least, lessons learned from search task preparation are presented, and a new direction for ad-hoc search based tasks at Video Browser Showdown is introduced.

In the realm of industrial anomaly detection, machine and deep learning models face a critical vulnerability to adversarial attacks. In this context, existing attack methodologies primarily target continuous features, often in the context of images, making them unsuitable for the categorical or discrete features prevalent in industrial systems. To fortify the cybersecurity of industrial environments, this paper introduces a groundbreaking adversarial attack approach tailored to the unique demands of these settings. Our novel technique enables the creation of targeted adversarial samples that are valid within the framework of supervised cyberattack detection models in industrial scenarios, preserving the consistency of discrete values and correcting cases where an adversarial sample transitions into a normal one. Our approach leverages the SHAP interpretability method to identify the most salient features for each sample. Subsequently, the Projected Gradient Descent technique is employed to perturb continuous features, ensuring adversarial sample generation. To handle categorical features for a specific adversarial sample, our method scrutinizes the closest sample within the normal training dataset and replicates its categorical feature values. Additionally, Decision Trees trained within a Random Forest are utilized to ensure that the resulting adversarial samples maintain the essential abnormal behavior required for detection. The validation of our proposal was conducted using the WADI dataset obtained from a water distribution plant, providing a realistic industrial context. During validation, we assessed the mean error and the total number of adversarial samples generated by our approach, comparing it with the original Projected Gradient Descent method and the Carlini & Wagner attack across various parameter configurations. Remarkably, our proposal consistently achieved the best trade-off between mean error and the number of generated adversarial samples, showcasing its superiority in safeguarding industrial systems.

In 5G and beyond, the figure of cross-operator/domain connections and relationships grows exponentially among stakeholders, resources, and services, being reputation-based trust models one of the capital technologies leveraged for trustworthy decision-making. This work studies novel 5G assets on which trust can be used to overcome unsuitable decision-making and address current requirements. First, it introduces a background and general architecture of reputation-based trust models. Afterward, it analyzes pivotal 5G assets on which trust can enhance their performance. Besides, this article performs a comprehensive review of the current reputation models applied to 5G assets and compares their properties, features, techniques, and results. Finally, it provides current trends and future challenges to conducting forthcoming research in the area.

We present a novel analysis of a competitive dynamic present on Ethereum known as "waiting games", where validators can use their distinct monopoly position in their assigned slots to delay block proposals in order to optimize returns through Maximal Extractable Value (MEV) payments, a type of incentive outside the Proof-of-Stake incentive scheme. However, this strategy risks block exclusion due to missed slots or potential orphaning. Our analysis reveals evidence that, although there are substantial incentives to undertaking the risks, validators are not capitalizing on waiting games, leaving potential profits unrealized. Moreover, we present an agent-based model to test the eventual consensus disruption caused by waiting games under different settings, arguing that such disruption only occurs with significant delay strategies. Ultimately, this research provides in-depth insights into Ethereum's waiting games, illuminating the trade-offs and potential profit opportunities for validators in this evolving blockchain landscape.

Today’s knowledge workers face cognitively demanding tasks and blurred work-life boundaries amidst rising stress and burnout in the workplace. Holistic approaches to supporting workers, which consider both productivity and well-being, are increasingly important. Taking this holistic approach, we designed an intervention inspired by cognitive behavioral therapy that consists of: (1) using the term “Time Well Spent” (TWS) in place of “productivity”, (2) a mobile self-logging tool for logging activities, feelings, and thoughts at work, and (3) a visualization that guides users to reflect on their data. We ran a 4-week exploratory qualitative comparison in the field with 24 graduate students to examine our Therapy-inspired intervention alongside a classic Baseline intervention. Participants who used our intervention often shifted toward a holistic perspective of their primary working hours, which included an increased consideration of breaks and emotions. No such change was seen by those who used the Baseline intervention.

Moving to a multi-cloud environment and service-based architecture, 5G and future 6G networks require additional defensive mechanisms to protect virtualized network resources. This paper presents MERLINS, a novel architecture generating optimal Moving Target Defense (MTD) policies for proactive and reactive security of network slices. By formally modeling telecommunication networks compliant with Network Function Virtualization (NFV) into a multi-objective Markov Decision Process (MOMDP), MERLINS uses deep Reinforcement Learning (deep-RL) to optimize the MTD strategy that considers security, network performance, and service level requirements. Practical experiments on a 5G testbed showcase the feasibility as well as restrictions of MTD operations and the effectiveness in mitigating malware infections. It is observed that multi-objective RL (MORL) algorithms outperform state-of-the-art deep-RL algorithms that scalarize the reward vector of the MOMDP. This improvement by a factor of two leads to a better MTD policy than the baseline static counterpart used for the evaluation.

The rapid entry of machine learning approaches in our dailyactivities and high-stakes domains demands transparency andscrutiny of their fairness and reliability. To help gauge ma-chine learning models’ robustness, research typically focuseson the massive datasets used for their deployment,e.g., cre-ating and maintaining documentation to understand theirorigin, process of development, and ethical considerations.However, data collection for AI is still typically a one-offpractice, and oftentimes datasets collected for a certain pur-pose or application are reused for a different problem. Addi-tionally, dataset annotations may not be representative overtime, contain ambiguous or erroneous annotations, or be un-able to generalize across domains. Recent research has shownthese practices might lead to unfair, biased, or inaccurate out-comes. We argue that data collection for AI should be per-formed in a responsible manner where the quality of the datais thoroughly scrutinized and measured through a systematicset of appropriate metrics. In this paper, we propose a Re-sponsible AI (RAI) methodology designed to guide the datacollection with a set of metrics for an iterative in-depth analy-sis of thefactors influencing the quality and reliabilityof thegenerated data. We propose a granular set of measurements toinform on theinternal reliabilityof a dataset and itsexternalstabilityover time. We validate our approach across nine ex-isting datasets and annotation tasks and four input modalities.This approach impacts theassessment of data robustnessusedin real world AI applications, where diversity of users andcontent is eminent. Furthermore, it deals with fairness andaccountability aspects in data collection by providing system-atic and transparent quality analysis for data collections.

Open-set face recognition is a scenario in which biometric systems have incomplete knowledge of all existing subjects. This arduous requirement must dismiss irrelevant faces and focus on subjects of interest only. For this reason, this work introduces a novel method that associates an ensemble of compact neural networks with data augmentation at the feature level and an entropy-based cost function. Deep neural networks pre-trained on large face datasets serve as the preliminary feature extraction module. The neural adapter ensemble consists of binary models trained on original feature representations along with negative synthetic mix-up embeddings, which are adequately handled by the designed open-set loss since they do not belong to any known identity. We carry out experiments on well-known LFW and IJB-C datasets where results show that the approach is capable of boosting closed and open-set identification accuracy.

These days, taking a photo is the most common way of capturing a moment. Some of these photos captured in the moment are never to be seen again. Others are almost immediately shared with the world. Yet, the context of the captured moment can only be shared to a limited extent. The continuous improvement of mobile devices has not only led to higher resolution cameras and, thus, visually more appealing pictures but also to a broader and more precise range of accompanying sensor metadata. Positional and bearing information can provide context for photos and is thus an integral aspect of the captured moment. However, it is commonly only used to sort photos by time and possibly group by place. Such more precise sensor metadata, combined with the increased computing power of mobile devices, can enable more and more powerful Augmented Reality (AR) capabilities, especially for communicating the context of a captured photo. Users can thereby witness the captured moment in its real location and also experience its spatial contextualization. With the help of a suitable data augmentation, such context-preserving presentation can be extended even to non-digitally born content, including historical images. This offers new immersive ways to experience the cultural history of one's current location. In this paper, we present an approach for location-based image presentation in AR on mobile devices. With this approach, users can experience captured moments in their physical context. We demonstrate the power of this approach based on a prototype implementation and evaluate it in a user study.

Trust, security, and privacy are three of the major pillars to assemble the fifth-generation network and beyond. Despite such pillars are principally interconnected, a multitude of challenges arise that need to be addressed separately. 5G networks ought to offer flexible and pervasive computing capabilities across multiple domains according to user demands and assure trustworthy network providers. To this end, distributed marketplaces expect to boost the trading of heterogeneous resources so as to enable the establishment of pervasive service chains between cross-domains. Yet, the need for selecting reliable parties as “marketplace operators” plays a pivotal role in achieving a trustworthy ecosystem. Two of the principal blockages in managing foreseeable networks are the need to consider trust as a property in the resource provisioning process and adapt previous trust models to accomplish the new network and business requirements. In this regard, this article is centered on the trust management of 5G multi-party network resource provisioning. As a result, a reputation-based trust framework is proposed as a Trust-as-a-Service (TaaS) solution for a distributed multi-stakeholder environment where requirements such as zero trust and zero-touch principles should be met. Besides, a literature review is also conducted to recognize the network and business requirements currently envisaged. Finally, the validation of the proposed trust framework was performed in a real research environment, the 5GBarcelona testbed, leveraging 12% of a 2.1 GHz CPU with 20 cores and 2% of the 30 GiB memory. These outcomes reveal the TaaS solution’s feasibility and conservative approach in the context of determining reliable network operators.

In Industry 4.0, security begins with the workers’ authentication, which can be done individually or in groups. Recently, group authentication is gaining momentum, allowing users to authenticate as group members without the need to specify the particular individual. Continuous authentication and federated learning are promising techniques that might help group authentication by providing privacy, by its own design, and extra security compared to traditional methods based on passwords, tokens, or biometrics. However, these techniques have not previously been combined or evaluated for authenticating workers in Industry 4.0. Thus, this paper proposes a novel continuous group authentication privacy-preserving (CGAPP)platform that is suitable for the industry. The CGAPP platform incorporates statistical data from workers’ smartphones and employs federated learning-based outlier detection for group worker authentication while ensuring the privacy of personal data vectors. A series of experiments were performed to measure the framework’s suitability and address the following research questions: (i) What is the cost of using FL compared to full data access in industrial scenarios? (ii) How robust is federated learning against adversarial attacks, specifically, how much malicious data is required to deceive the model? and (iii) How much noise is required to disrupt the authentication system? The results demonstrate the effectiveness of the CGAPP platform in the industry since it provides factory safety while preserving privacy. This platform achieves an accuracy of 92%, comparable to the 96% obtained by traditional approaches in the literature that do not address privacy concerns. The platform’s robustness is tested against attacks in the second and third experiments, and various countermeasures are evaluated. While the CGAPP platform exhibits certain vulnerabilities to data injection attacks, straightforward countermeasures can alleviate them. Nevertheless, the system’s performance experiences a notable impact in the event of a data perturbation attack, and the countermeasures investigated are ineffective in addressing this issue.

The use of drones (or Unmanned Aerial Vehicles) in urban areas has emerged rapidly in the last decade, and continues to expand at an accelerating pace. Alongside the emergent uses of high-impact technology in both public and private sectors, political debates about the potential risks and challenges have arisen, encompassing diverse perspectives and attitudes about the ethical, legal, social, and regulatory implications of introducing and integrating new technology in society. This scoping review offers an assessment of the societal acceptance factors of urban drones discussed in the current academic literature. We used a hybrid approach including quantitative landscape mapping and qualitative content analysis of the selected articles to inductively develop a typology of acceptance factors associated with urban use of drones. This review illuminates areas that have been the focus of attention within the current body of knowledge (e.g., visual and noise pollution of drones), sketches the evolution of the relevant discussions over time (e.g., a focus on the safety of the drone technology toward safety of the cargo it carries and security of the data it collects), and points to areas that have received less considerations (e.g., media appropriation and social group influence). It can, thus, help situate the topic of societal acceptance of urban drones in specific contexts, and orient future research on promoting value sensitive innovation in society more broadly.

Resource-constrained spectrum sensors from the Internet of Battlefield Things (IoBT) monitor the frequency spectrum to communicate over unoccupied bands, intercept enemy transmissions, or decode valuable information. However, they are also susceptible to Spectrum Sensing Data Falsification (SSDF) attacks manipulating the sensing data and impacting the previous services. Detection systems based on fingerprinting and machine learning have shown promising performance while detecting existing SSDF attacks. However, novel attacks reducing their impact on sensors behaviors have not been analyzed yet. Thus, this work redesigns and reimplements seven SSDF attacks by modifying spectrum data in the sensor memory instead of at later stages in the file system. Several experiments with current intelligent detection systems demonstrated that more effort is needed from the defensive perspective since the new SSDF attacks evade their detection. In this sense, literature-based detection methods achieve less than a 0.50 True Positive Rate when detecting the new implementations of the attacks.

With a rising frequency and scale, Distributed Denial-of-Service (DDoS) attacks persist as a critical cybersecurity issue. While shared attack fingerprints aid many intrusion detection systems in identifying threats, their application for DDoS attacks remains limited due to their distinct nature. However, fingerprints observed from multiple locations can provide valuable insights. This paper presents Reassembler, a novel platform for achieving a global DDoS attack analysis using attack fingerprints recorded from various locations. Reassembler consolidates these fingerprints into a unified view allowing to obtain a global overview of DDoS attacks. The evaluation, conducted on four simulated scenarios, demonstrates Reassembler's ability to extract novel properties, such as the count of intermediate nodes and the estimated percentage of spoofed IPs.

As researchers strive to narrow the gap between machine intelligence and human through the development of artificial intelligence multimedia technologies, it is imperative that we recognize the critical importance of trustworthiness in open-world, which has become ubiquitous in all aspects of daily life for everyone. However, several challenges may create a crisis of trust in current open-world artificial multimedia systems that need to be bridged: 1) Insufficient explanation of predictive results; 2) Inadequate generalization for learning models; 3) Poor adaptability to uncertain environments. Consequently, we explore a neural program to bridge trustworthiness and open-world learning, extending from single-modal to multi-modal scenarios for readers.1) To enhance design-level interpretability, we first customize trustworthy networks with specific physical meanings; 2) We then design environmental well-being task-interfaces via flexible learning regularizers for improving the generalization of trustworthy learning; 3) We propose to increase the robustness of trustworthy learning by integrating open-world recognition losses with agent mechanisms. Eventually, we enhance various trustworthy properties through the establishment of design-level explainability, environmental well-being task-interfaces and open-world recognition programs. As a result, these designed open-world protocols are applicable across a wide range of surroundings, under open-world multimedia recognition scenarios with significant performance improvements observed.

Since the proposal of Bitcoin in 2009 and with the inclusion of the first transaction in its genesis block, Blockchains (BC) have been used to store arbitrary data, including texts, images, and documents. However, such data is often not easily discoverable in BCs and is embedded within their binary data structures. Thus, this paper presents the design and implementation of a solution to analyze BC transactions searching for “media” content. This solution, called blockchain-parser, is capable of detecting ASCII strings and files (e.g., PDF, GIF, and SVG) embedded in BC's transactions. To evaluate such a solution, Bitcoin, Monero, and Ethereum cryptocurrencies were examined to find commonalities and differences between different BCs regarding their arbitrary data storage usage. Conclusions from such an evaluation indicate that Ethereum has been the most used BC for media data storage compared to Bitcoin and Monero.