Contributions published at Interactive Visual Data Analysis (Jürgen Bernard)

Rankings serve to structure large datasets. They are an integral part of decision-making in various fields. To address the complex task of interpreting and comparing ranking algorithms and the results they produce, we created a Visual Analytics (VA) tool called RankViz. This prototype allows users to visualize and explore the output of various ranking algorithms and includes multiple metrics to assess the quality and differences of the rankings. The pairwise comparison data we used to construct the rankings was collected in a previous study by Barth et al. This thesis reviews different ranking algorithms, details the functionality of RankViz, demonstrates its utility with usage scenarios, and discusses potential future work in this field.

Recommender systems have become integral in guiding users through the overwhelming abundance of online content. As these systems assume an ever-increasing role in shaping user decisions and preferences, there is a growing demand for clarity in their decision-making processes to instill trust. Recommendation algorithms with a high degree of accuracy such as matrix factorization are highly regarded and widely adopted. Nonetheless, these algorithms tend to exhibit high complexity in their logic and architecture, rendering them challenging to explain to end-users. This issue has been recognized and many tools have presented possible solutions. Many of the implemented approaches, however, have demonstrated shortcomings due to disregarding some user-centered properties or overly concentrating on unraveling the underlying algorithmic intricacy. This work presents MFExplain, an innovative tool for explaining movie recommendations generated with matrix factorization. The tool aims to explain recommendations by relying on the provision of intuitive justifications. Leveraging interactivity and cutting-edge dimensionality reduction techniques enables the tool to also encourage exploration, allow user feedback, and foster many desirable recommender system properties that enrich the user experience.

With the surge in the volume and dimensionality of large image datasets across fields such as medicine, manufacturing, and quality monitoring, there is an increased emphasis on efficiently curating these datasets. This design study explores the challenges associated with labeling and exploring large, high-dimensional, and unlabeled image datasets. Traditional tools prioritize either data visualization using techniques like dimensionality reduction or labeling automation using AI learning mechanisms. This binary focus often comes at the cost of extensive labeling functionalities or comprehensive overviews, since user interaction is reduced. This research bridges this gap by introducing DaedalusData, an interactive visual analytics approach that combines meaningful visual exploration with efficient labeling and intuitive feedback loops. DaedalusData presents an interactive platform that enables pattern and anomaly exploration, efficient image labeling by integrating metadata, and preliminary steps toward labeling automation. The tool was developed alongside domain experts and built for a dataset containing particle contamination in consumables at Roche Diagnostics. As a design study this thesis, solved a real-world problem, through close collaboration with domain experts. The study posits that merging interactivity, human expertise, and automated processes offer a promising direction for managing large image datasets, with DaedalusData serving as a foundational step.

Artificial intelligence (AI) is becoming increasingly important as the amount of digital data grows. However, AI systems are often opaque and perceived as black boxes, which has a negative impact on user acceptance and trust. We see this in healthcare, where despite the great potential of AI, a lack of understanding and trust has held back physicians from adopting it. One way to address these issues is through Explainable AI (XAI), which focuses on understanding and interpreting AI behavior. In this thesis, we want to contribute to XAI by developing a visual analytics system called VisAIExplorer. We want to find out how an interactive visual analytics system can be designed to explain machine learning models to novice machine learning users, and what types of visualizations within the system can help to build understanding. The goal of VisAIExplorer is to explain the two models, logistic regression and hierarchical clustering, to novice machine learning users by providing various visualizations and support throughout the work process. The machine learning models are trained on a medical dataset about strokes, as healthcare professionals could benefit from a better understanding and increased trust in AI systems. By improving transparency and user support, VisAIExplorer aims to overcome the limitations of existing AI tools and promote more explainability in AI. The thesis includes a literature review, system development, evaluation, and suggestions for future improvements.

The Digital Humanities are increasingly employing computational methods for the curation of their research artifacts. The digitization of historical documents and their subsequent curation and annotation is common practice. The resulting digital editions often utilize a semi-structured data format to enhance the digitized research objects with annotations. Despite the presence of established annotation standards, annotation practices can still differ significantly within and across editions, resulting in considerable heterogeneity. This hampers the interoperability and reusability of digital editions. We contribute a visual analytics (VA) approach for the exploration of annotation practices within and across digital editions. We worked closely with the digital edition community to develop Tag Explorer, a VA tool tailored to their needs. Multiple coordinated views visualize annotation practices on various granularity levels, enabling users to better understand common practices and differences of editions stemming from heterogeneous sources. The users can adapt the visualizations to their information needs by delineating the exploration space and switching between different viewpoints. Tag Explorer fills a gap in the existing landscape of VA tools for the Digital Humanities, allowing the exploration of annotation strategies within and across heterogeneous digital editions. We evaluated our approach by two case studies with domain experts. Tag Explorer enabled the domain experts to check existing hypotheses, inspired potential improvements in their own editions, and uncovered unexpected findings regarding the annotation practices within and across digital editions. These insights help domain experts making more informed decisions during the annotation process, leading to more interoperable and reusable digital editions.

With the ever-increasing amount of and widely spread access to data, decision-making is becoming a complex task. Decision-making tasks can often be supported by utilizing rankings to help decision-makers to understand their different options. Existing solutions for the creation of rankings are often time-consuming, targeted towards experts only, unintuitive to use, or attributebased and therefore need mathematical understanding for their interpretation. We propose Ranking Companion, a human-centered approach combining visual analytics with active learning to create item-based ranking. Ranking Companion makes rankings intuitively and interactively available to end users, while reducing the necessary amount of input provided by users to create meaningful rankings. Our novel approach allows users to create rankings iteratively and remain in full control of the input provided to the machine learning model. Moreover, Ranking Companion provides explainability for the underlying machine learning model. In this work, we apply Ranking Companion in a decision task in the musical domain: finding new artists to listen to. Our approach is evaluated using feedback on design iterations by visual analytics experts, a usage scenario, as well as a case study. The evaluations show that the approach indeed decreases time necessary for providing model input, while increasing user acceptance of the provided rankings for certain user groups.

Information Extraction (IE) deals with the task of extracting targeted information from documents, such as the invoice amount in an invoice. In order to apply IE in practice, a corresponding machine-learning model must first be trained, for example a Named Entity Recognition (NER) model. This poses several challenges: First, the more specifically the models are trained on a concrete document template, the better they are, which requires that the documents be sorted before training. Second, the documents must be annotated by a human. Both of these are time-consuming and repetitive tasks that do not utilize the human’s potential. To address these issues, I have developed IVIE-Docs, an Interactive Visual Information Extraction tool for Documents that includes a clustering component and a NER component to complete the process of training NER models. The clustering component allows users to quickly group their documents. In the NER component, active learning principles are used to identify those documents that can train the NER model the fastest. Users can decide which document they consider most useful based on multiple information sources for active learning. A particular challenge here is that clustering occurs at the document-level, while NER is trained at the word-level. Moreover, in classical active learning, one instance of the same granularity as the prediction is proposed at a time. This was not practical in my approach, since not only a single word should be labeled, but a complete document. In IVIE-Docs, two measures help to close the granularity gap. A new document layout vector based on layout information of the individual words created a consistent basis between the clustering and the NER model. Second, the individual word predictions are aggregated at the document level to enable cross-granularity active learning. IVIE-Docs was tested in two studies with a total of 6 subjects. The results show that users were able to cluster their documents based on the document layout vector and that they achieved better results using the active learning components with fewer labeled documents than with a random selection.

If you have ever used an e-commerce service or a streaming platform, you have already come across something like: "recommended for you", or "other users have also bought this". Our educational article below will give you an introduction to Recommender Systems (RS), and illustrate how this field currently leverages deep-learning techniques. Our article is meant to foster in the reader a broad inuition of RS, highlight common scenarios causing failure in various RS techniques, and provide a visual understanding of how recommender systems work. First, we illustrate how Matrix Factorization (MF) works-- a (relatively) simply designed recommender system. Then, we gradually increase the sophistication of our approach, exploring the effect this has on the prediction accuracy of a model. We explain differences in two models' competence in predicting a user's rating of movies they have seen, through various visualizations. Our two models use the MovieLens 1M dataset, selecting different sets of features (columns) per model [18].

Time-stamped event sequences (TSES) are event sequences without values. Analysts are mainly interested in the temporal signatures of phenomena. It is a hardly investigated data type with growing interest since it is observed across a wide range of domains. There are two main problems in TSES that hamper the design of visual-interactive solutions. First, lack of awareness of affecting factors for problem characterization for TSES and second, lack of specific task characterization for TSES. Consequently, designers have a hard time making correct design decisions when building data analysis solutions for TSES, which ultimately influence the effectiveness of the tool to be built. We conducted two types of studies to address these problems. To address the lack of awareness of affecting factors, we did a systematic characterization of TSES-oriented real-world problems structured by four main aspects: (1) domain context & users, (2) data characteristics, (3) tasks, (4) metrics. In our study approach, we systematically identified a diverse set of factors associated with these above-mentioned main four aspects initially. Then, we collected 65 TSES-oriented real-world problems spanning a wide range of domains, focusing on identified factors using a user-based survey study. Lastly, we systematically analyzed the discovered factors and then related them to identify the relationships between factors. To address the second problem of lack of specific task characterization, we presented a generalized problem characterization for TSES. In our study approach, we used two complementary survey sources: a User-based survey study and a Survey of design studies. Initially, we built a generalization of tasks that are currently supported in vis tools related to TSES based on 16 design studies for TSES and related, which resulted in 26 tasks. Then we built a generalization of user tasks based on 65 survey responses, which resulted in 25 tasks. For the generalization, we used coding and affinity diagramming, well-known techniques of qualitative research. Finally, we unified these two sources and proposed a task characterization consisting of 28 tasks for TSES. We found that questionnaire answers on TSES are extremely heterogeneous, and no two answers are similar or even equal with some degree of abstraction. Results of the second part of the study show that 80% of the user tasks are similar to tasks extracted from design studies focused on TSES or related data types.

null

We present a visual analytics approach for the in-depth analysis and explanation of incremental machine learning processes that are based on data labeling. Our approach offers multiple perspectives to explain the process, i.e., data characteristics, label distribution, class characteristics, and classifier characteristics. Additionally, we introduce metrics from which we derive novel aggregated analytic views that enable the analysis of the process over time. We demonstrate the capabilities of our approach in a case study and thereby demonstrate how our approach improves the transparency of the iterative learning process.

ultiple Sklerose (MS) ist eine unheilbare und heterogene Krankheit, die die Lebensqualität eines Menschen stark beeinträchtigen kann. Smartwatches haben das Potenzial, das Aktivitätsniveau eines MS-Patienten mit geringem Aufwand zu erfassen. Zur Bewältigung der Herausforderungen, die sich aus den daraus resultierenden MS-Patientendaten ergeben, schlagen wir einen Visual Analytics (VA) Ansatz vor, der in enger Zusammenarbeit mit Gesundheitsexperten entwickelt wurde. Das entwickelte VA-Tool, der MS Pattern Explorer, ermöglicht eine kompakte Visualisierung des Verlaufs der Herzfrequenz oder Schrittzahl eines Patienten auf verschiedenen Detailebenen. Mit einem interaktiven Clustering Ansatz, der auf unüberwachtem maschinellem Lernen basiert, ermöglichen wir eine effiziente Exploration von zeitlichen Aktivitätsmustern. Der MS Pattern Explorer ermöglicht darüber hinaus die Suche und Lokalisierung eines bestimmten Musters innerhalb der Patientendaten und gibt zusätzliche Informationen über dessen Auftreten preis. Darüber hinaus erleichtert der Ansatz einer patientenübergreifenden Analyse, um potenziell interessante Patienten zu entdecken. Dies wird durch die Untersuchung von Mustern über Patienten hinweg mittels Clustering-Algorithmen und durch die Suche nach ähnlichen Sequenzen über Patienten hinweg mittels Nearest Neighbour-Suche erreicht. Der MS Pattern Explorer wurde anhand von drei Arten von Evaluierungen bewertet. Erstens wurden wertvolle Rückmeldungen unserer Kollaborateure in verschiedenen Phasen der Entwicklung des VA-Tools berücksichtigt. Zweitens zeigt eine Nutzerstudie, dass ein breites Publikum unser VA-Tool effektiv nutzen kann, um Einblicke in die Aktivitätsmuster von MS-Patientendaten zu gewinnen. Drittens demonstrieren zwei Fallstudien, die von einem unserer Kollaborateure durchgeführt wurden, die Nützlichkeit und den vollen Umfang der Anwendungsfunktionalitäten.

Electrical engines are a key technology all automotive manufacturers must master to stay competitive. Engineers need to analyze an overwhelming number of engine measurements to improve the manufacturing for this technology. They are hindered in the task of analyzing large numbers of engines, however, by the following challenges: 1) Engines comprise a complex hierarchical structure of subcomponents. 2) Locating the cause of errors along manufacturing processes is a difficult procedure. 3) Large numbers of heterogeneous measurements impair the ability to explain errors in engines. We address these challenges in a design study with automotive engineers and by developing the visual analytics system Manufacturing Explorer (ManEx), which provides interactive interfaces to analyze measurements of engines across the manufacturing process. ManEx was validated by five experts. Our results suggest high usability and usefulness scores and the improvement of a real-world manufacturing process. Specifically, with ManEx, experts reduced scraped parts by over 3%.

Graph neural networks (GNNs) are a class of powerful machine learning tools that model node relations for making predictions of nodes or links. GNN developers rely on quantitative metrics of the predictions to evaluate a GNN, but similar to many other neural networks, it is difficult for them to understand if the GNN truly learns characteristics of a graph as expected. We propose an approach to corresponding an input graph to its node embedding (aka latent space), a common component of GNNs that is later used for prediction. We abstract the data and tasks, and develop an interactive multi-view interface called CorGIE to instantiate the abstraction. As the key function in CorGIE, we propose the K-hop graph layout to show topological neighbors in hops and their clustering structure. To evaluate the functionality and usability of CorGIE, we present how to use CorGIE in two usage scenarios, and conduct a case study with five GNN experts. Availability: Open-source code at https://github.com/zipengliu/corgie-ui/ , supplemental materials & video at https://osf.io/tr3sb/ .

null

The topic of this thesis is Human-Centered Ranking of Data Objects with Interactive Attribute Scoring Functions (ASF). The contribution of this thesis includes: - A literature review targeting existing ASFs and ASF creation tools - An introduction to ASF chracteristics and a taxonomy of ASFs based on the literature review - A Visual Interactive Prototype of the tool RankASco that contains interfaces for all ASFs - An evaluation of the developed prototype in the form of expert feedback rounds and a summative user study - A discussion of the results and possible areas of future work

null

Mixed-initiative visual data analysis processes are characterized by the co-adaptation of users and systems over time. As the analysis progresses, both actors – users and systems – gather information, update their analysis behavior, and work on different tasks towards their respective goals. In this paper, we contribute a multigranular model of co-adaptive visual analysis that is centered around incremental learning goals derived from a hierarchical taxonomy of learning goals from pedagogy. Our model captures how both actors adapt their data-, task-, and user/system-models over time. We characterize interaction patterns in terms of the dynamics of learning and teaching that drive adaptation. To demonstrate our model’s applicability, we outline aspects of co-adaptation in related models of visual analytics and highlight co-adaptation in existing applications. We further postulate a set of expectations towards adaptation in mixed-initiative processes and identify open research questions and opportunities for future work in co-adaptation.