Contributions in Merlin

In today's world, knowledge workers are often overwhelmed by the vast amounts of information they encounter while carrying out their tasks. As a result, it is vital to develop effective strategies for efficiently reusing previously foraged information to minimize foraging effort. One of these strategies is information curation, which is the concept of keeping, managing, and exploiting foraged information. Existing prototypes that addressed this topic have mostly specific use cases, like web resource curation or task history curation. Only a few of them allow the capturing of cross-application settings. None of these prototypes are optimized to support users in information foraging tasks. They lack extensive retrieval functionality, semantic content analysis, and structuring options for curated assets. To fill this void, we designed and developed the ScreenCurator. Our application allows users to capture cross-application screen settings and store them with extensive metadata. This combination shall enable comprehensive retrievability and reusability of curated knowledge. To provide users with a simple and pleasant experience, the ScreenCurator implements a certain degree of automation combined with an intuitive interface. Our application was evaluated in a user study where seven participants used the ScreenCurator for 10-15 working days besides their daily tasks. The gathered feedback implied that our approach improved the experience of taking and retrieving screenshots. Furthermore, two high-level use cases could be identified: long-term backups and short-term to-dos. Nevertheless, we found that the ScreenCurator needs to increase the implemented degree of automation and add further structuring options. Additionally, it would be of great value if the ScreenCurator would enable collaborative curation and knowledge sharing. Besides extending the feature set, care should be taken to maintain the simplicity and intuitiveness of the application.

null

Artificial intelligence (AI) has immersed our daily lives and assists in the decision process of critical sectors such as medicine and law. Therefore it is now more important than ever before that AI systems developed are reliable, ethical, and do not cause harm to humans. The High-Level Expert Group on AI (AI-HLEG) of the European Commission has laid the foundation by defining seven key requirements for trustworthy AI systems. To address concerns about privacy risks associated with centralized learning approaches federated learning (FL) has emerged as a promising and widely used alternative. FL allows multiple clients to collaboratively train machine learning models without the need for sharing private data. Because of the high adaption of FL systems, ensuring that they are trustworthy is crucial. Previous research efforts have proposed a trustworthy FL taxonomy with six pillars, each comprehensively defined with notions and metrics. This taxonomy covers six of the seven requirements defined by the AI-HLEG. However, one notable aspect that has been largely overlooked by research is the requirement for environmental well-being in trustworthy AI/FL. This leaves a significant gap between the expectations set by governing bodies and the guidelines applied and measured by researchers. This master thesis addresses this gap by introducing the sustainability pillar to the trustworthy FL taxonomy and thus presenting the first taxonomy that comprehensively addresses all the requirements defined by the AI-HLEG. The sustainability pillar focuses on assessing the environmental impact of FL systems and incorporates three main aspects: hardware efficiency, federation complexity, and the carbon intensity of the energy grid, each with well-defined metrics. As a second contribution, this master thesis extends an existing prototype to evaluate the trustworthiness of FL systems with the sustainability pillar. The prototype is then extensively evaluated in various scenarios, involving different federation configurations. The results shed light on the trustworthiness of different federation configurations in different settings with varying complexities, hardware, and energy grids used. Importantly, the sustainability pillar’s score corrects the overall trust score by considering the environmental impact of FL systems across seven key pillars. Thus, the proposed taxonomy and prototype are the first to comprehensively address all seven AI-HLEG requirements and lay the foundation for a more accurate trustworthiness assessment of FL systems.

The use of convolutional neural networks (CNNs) in image classification tasks is a rapidly progressing field of research, including the classification of facial attributes. However, it is not yet completely understood how CNNs make decisions. To improve the transparency of the decision-making process and thus enhance interpretability and trustworthiness of CNNs, methods have been developed to visualize this process. In this thesis, we use the Gradient-weighted Class Activation Mapping (Grad-CAM) technique proposed by Selvaraju et al. (2017) to identify the regions of an image that the CNN uses for classification. This technique produces class-specific heatmaps that are intuitively interpretable. In order to evaluate the class activation maps, we define a set of masks, one for each of the 40 facial attributes that we examine. By using an approach called Acceptable Mask Ratio (AMR) we quantify how much of the activated area lies within the masked area. The higher the value of the AMR the more active is the CNN within the area that we expect, which usually corresponds to the location of the attribute being classified. We compare two different CNNs, one considers the class imbalance inherent to the data set (balanced CNN), and the other does not (unbalanced CNN). Our results show that overall the balanced CNN more often uses image regions that lie within the masked area. Furthermore, the results show an unexpected pattern for the unbalanced CNN namely for highly biased attributes the Grad-CAMs for the majority class show no activity at all.

This master’s thesis explores fine-tuning Large Language Models (LLMs) to reformulate condensed coordinated expressions found in job postings. This kind of condensed coordinated expression is frequently used in job postings, which is our target text genre for this work. Four gold-standard datasets were created for two tasks in English and German. The first task focuses on truncated word completion, where elided text like “Haus- und Gartenarbeit” (house and garden work) needs to be completed to “Hausarbeit und Gartenarbeit”. The German GS dataset consists of 510 samples, while the English GS contains 402 samples. The primary goal is to assess the LLMs’ performance in this task and identify promising models for the second, more complex task. The second task involves expanding condensed coordinated soft-skill requirements like “Sie arbeiten sehr selbständig, ziel- und kundenorientiert” into explicit self-contained paraphrases such as “Sie arbeiten sehr selbständig, arbeiten zielorientiert und arbeiten kundenorientiert”. To achieve a proper mapping of soft-skill requirements to a detailed domain ontology, it is crucial to provide self-contained text spans that refer to a single concept. For creating the German GS, we utilized In-Context Learning with ChatGPT, providing 5 examples in the prompt to generate additional samples. Subsequently, these samples were used to fine-tune GPT-3 and later manually verified to form a GS dataset comprising 1968 samples. In the first task, T5-large, and FLAN-T5-large, and GPT models showed similar levels of accuracy. However, in the second task, T5-large and FLAN-T5-large performed poorly. To improve results, we applied PEFT-based techniques, LORA, to fine-tune BLOOM, T5-Large, FLAN T5-XXL, and mT5-XL on a single GPU. Among these, GPT-3 demonstrated superior performance, closely followed by mT5-XL in overall evaluations. For evaluation, we measured how incomplete soft skill text spans were completed, assessed both completed and incomplete soft skills, and evaluated overall sentence similarity. Error metrics such as Rouge-L, average Levenshtein distance, % of matched skills, and Cosine Similarity were used to evaluate soft skill changes and overall text similarity. In conclusion, Large Language Models (LLMs) effectively expanded condensed coordinated expressions into simpler formulations, including completing hyphenated words in German, without relying on traditional methods sensitive to grammatical and spelling errors.

In this study the energy consumption and greenhouse gas emissions of a video livestream with a length of four hours on the livestreaming platform twitch.tv were calculated. The video transmission requires the end devices of the streamer and the viewer, the data centres and the communication network. The communication network is separated into wide area network, radio access network and fixed network. In the reference scenario, the livestreamer uses a desktop PC and two screens to broadcast a video with a resolution of 1080p and 60hz through a fixed network connection. Viewers can play the livestream on different end devices. As a result of the calculations, the range of greenhouse gas emissions generated during the livestream is between 207 and 804g CO2. The difference is due to the choice of the used end device and the difference between wireless and landline connections. For the end devices, the screen size is an important factor for the contribution to the total energy consumption of a livestream. The radio access network connection has the highest energy intensity because of the consumption of the older radio generations compared to 4G. The enhancement of the internet infrastructure to 5G will lead to more efficient transmission and the main consumption will shift to the end devices. The choice of the end device and its usage can offset the savings of the energy and the greenhouse emissions by switching to a better internet infrastructure. Strategies are needed against more intensive production and consumption to oppose the climate change.

In 2011, the introduction of Bluetooth Low Energy (BLE) marked a significant shift in wireless communication, paving the way for the Internet of Things (IoT) and the rise of location-based trackers. While devices like Apple's AirTag provide convenience, they pose security risks, notably the potential for malicious actors to track individuals unbeknownst to them. This work aims to address security concerns related to BLE trackers, especially considering the disparity between protections for iOS and Android users. The research focuses on creating an Android application, improving upon previous tools like HomeScout, which had limited classification capabilities. A feature based prototype was proposed and three classification models including SVM, Random Forest, and Multi-layer Perceptron were evaluated. The result was an effective classification method for BLE devices, with the Multi-Layer Perceptron model outperforming others with a 94.5\% accuracy on test data. The model was further tested on unseen device to evaluate its generalization capability, which achieved a 88\% of accuracy in with binary classification target, tracker and non-tracker. This model was integrated into the HomeScout app after resolving an identified bug in the original application. Eventually, Homescout is able to identify tracker and non-tracker device after integration. Future work entails refining the prototype, enhancing the dataset's diversity, and ensuring user privacy in public datasets.

Poisoning attacks pose a substantial threat to the trustfulness of Federated Learning. For example, malicious participants can degrade the model performance of honest members or implement backdoors that can be exploited at inference time to take advantage of incorrect predictions. Researchers have been highly active to mitigate poisoning attacks. Existing approaches prominently aim for defenses against poisoning attacks in centralized settings. While decentralized Federated Learning has gained significant attention as a promising approach without a central entity, the security aspects related to poisoning attacks remain largely unaddressed. This work introduces a defense approach called “Sentinel” for mitigating poisoning attacks in horizontal, decentralized Federated Learning. Sentinel leverages the advantage of local data availability and defines a three-step aggregation protocol composed of similarity filtering, bootstrap validation and normalization to protect against malicious model updates. The proposed defense mechanism is evaluated on various datasets under different types of poisoning attacks and threat levels. An extension of Sentinel, called SentinelGlobal, is presented, which incorporates a global trust protocol to reduce computational complexity and further improve the effectiveness against adversaries. Both Sentinel and SentinelGlobal demonstrate promising results against untargeted and targeted poisoning attacks. Hence, this work contributes to the advances in research against poisoning attacks in decentralized federated systems. Additionally, the results of this work highlight the need for more sophisticated defense strategies against backdoor attacks, independent of the Federated Learning architecture.

Selling put options can be lucrative; however, the returns tend to exhibit a strong negative skewness. Moreover, the seller may have liquidity issues during the holding period, especially when margin requirements become too large. Existing hedging techniques often overlook potential liquidity problems during the holding period, focusing solely on terminal losses. To address this limitation, we present a novel risk management approach by reformulating the closing time of the short position as an optimal stopping problem. To find the solutions, we decompose the holding period into a sequence of binary stopping decisions, which naturally fit into the reinforcement learning framework. Multiple deep reinforcement learning algorithms, namely Deep Q-Learning, Rainbow, and Synchronous Advantage Actor-Critic, are employed to identify the optimal times for closing the position. Our training framework introduces a new reward function that enables the agents to maximize each option’s profit and enhance its Sharpe ratio. In a simulated environment with nontrivial optimal stopping solutions, we demonstrate the e↵ectiveness of the algorithms and our training setup. Furthermore, we apply these algorithms to market data; specifically, SPY put option data from 2005 to 2022. During this analysis, we encounter a significant imbalance in the training data between paths with negative and positive returns, making it challenging for the algorithms to learn an optimal solution. Consequently, we propose several approaches to tackle this issue in future research. Overall, our work presents a promising approach to address liquidity concerns during option selling strategies, and our findings contribute to the advancement of reinforcement learning techniques in the financial domain.

The Internet of Things (IoT) platform is one of the key drivers of the smart home market, having revolutionized the advancement of smart home technology. Besides the many benefits for convenience and efficiency, there are also concerns about security and privacy in such environments. The increasing complexity of smart homes and hardware limitations of individual devices necessitate the storage and processing of data in remote cloud environments. This raises privacy issues due to potential misuse or disclosure of sensitive information about residents. To the author's knowledge, no existing Privacy Enhancing Technology (PET) offers a lightweight approach to enforce privacy in smart home environments by combining existing tools into a unifying framework. The goal of this thesis is to take a first step towards an extensible open source software system that integrates into the smart home environment with the purpose of monitoring smart home device communications and controlling their communication behavior through user-defined policies. To this end, a prototype application is developed, which monitors smart home devices' Domain Name System (DNS) requests and enforces policies via a DNS sinkhole mechanism. The prototype system is deployed to a system-on-chip platform and evaluated in a live smart home environment to gain insight into the viability of the prototype. The aim is to examine the performance, effectiveness, and limitations of the prototype with the intention of validating the general approach. The results of these experiments indicate that the prototype successfully achieves the goals outlined in this thesis. The application prototype is capable of monitoring the network activity of smart home devices. The collected data are processed to gain insights and make this information transparent to the users. Furthermore, the prototype allows users to define simple allow/block policies which are subsequently enforced by the system.