Contributions published at Software Evolution and Architecture Lab (Harald Gall)

oftware Developers utilize various data sources to draw a more complete picture of the evolution of an application over its lifetime. What kind of data is being used by developers, and how they use this data, is constantly evolving. Techniques derived from empirical software engineering mostly focus on static information (e.g., change requests in issue trackers, test coverage and build information). However, as multifaceted as the development-time feedback discussed so far may be, it still only covers one part of the application evolution. So far, it rarely bothers with gathering data about the application at runtime while it is actually being used. This is the domain of performance engineering, a relatively decoupled separate research domain, which is mostly concerned with optimizing non-functional properties of applications without actually looking at development-time artifacts, such as source code or change requests. However, with the advent of Cloud Computing and DevOps, software development and operations activities are converging. Hence, feedback for empirical software engineering needs to be extended with runtime information. In the cloud, both development-time and runtime feedback are readily available. However, developers currently lack awareness of the potential data and tools provided in the cloud. In this thesis, an approach is developed that integrates feedback from operational data into the Integrated Development Environment (IDE). A prototypical implementation called PerformanceHat, which uses the gathered runtime data to automatically detect the two common software performance problems “hotspot method” and “critical loop”, is contributed. The obtained feedback is visualized in the existent Java editor inside the IDE. An exploratory study shows that the presented approach combines important characteristics to support the DevOps methodology that has not been covered in its entirety in previous research.

Context: Continuous Integration (CI) has become an established best practice of modern software development. Its philosophy of regularly integrating the changes of individual developers with the master code base saves the entire development team from descending into Integration Hell, a term coined in the field of extreme programming. In practice, CI is supported by automated tools to cope with this repeated integration of source code through automated builds and testing. One of the main problems, however, is that relevant information about the quality and health of a software system is both scattered across those tools and across multiple views. Objective: This paper introduces a quality awareness framework for CI-data and its conceptional model used for the data integration and visualization. The framework called SQA-Mashup makes use of the service-based mashup paradigm and integrates information from the entire CI-toolchain into a single service. Method: The research approach followed in our work consists out of (i) a conceptional model for data integration and visualization, (ii) a prototypical framework implementation based on tool requirements derived from literature, and (iii) a controlled user study to evaluate its usefulness. Results: The results of the controlled user study showed that SQA-Mashup’s single point of access allows users to answer questions regarding the state of a system more quickly (57%) and accurately (21.6%) than with standalone CI-tools. Conclusions: The SQA-Mashup framework can serve as one-stop shop for software quality data monitoring in a software development project. It enables easy access to CI-data which otherwise is not integrated but scattered across multiple CI-tools. Our dynamic visualization approach allows for a tailoring of integrated CI-data according to information needs of different stakeholders such as developers or testers.

With the recent advances of cloud computing, effective resource usage (e.g., CPU, memory or network) becomes an important question as application developers have to continuously pay for rented resources, even if they are not used effectively. In order to maintain required performance levels, it is currently common to reserve resources for peak resource usage or possible resource usage overlaps, if more than one task is executed on a host. While this is a reasonable approach for long-running applications or web servers, for some applications with disperse resource usage over time, this strategy causes significant over-provisioning and thus resource wastage and financial loss. In this paper we present a profiling-based task scheduling approach for factory-worker applications that schedules tasks within the defined resource limitations (e.g., existing machine memory size or network quota) and distributes the tasks in the cloud environment in order to use resources effectively. The evaluation of our approach approved the efficiency of the proposed algorithm and minimal performance overhead. In case of evaluated application, presented scheduling process leads up to 33% resource saving with only 1% of performance loss.

Cloud computing has started to play a major role in IT industry and the numerous cloud providers and services they offer are continuously growing. This increasing cloud service diversity, especially observed for infrastructure services, demands systematic benchmarking in order to assess cloud service performance and thus assist cloud users in service selection. However, manually conducting cloud benchmarks is time-consuming and error-prone. Therefore, previous work addressed these problems with automation approaches but has failed to provide a convenient way to automate the process of installing and configuring a benchmark. In order to solve this benchmark provisioning problem, this thesis introduces a benchmark automation framework called Cloud WorkBench (CWB) where benchmarks are entirely defined by means of code and executed without manual interaction. CWB allows to define configurable benchmarks in a modular manner that are portable across cloud providers and their regions. New benchmarks can be added at runtime and variations thereof are conveniently configurable via a web interface. Integrated periodic scheduling capabilities timely trigger benchmark executions that automatically acquire cloud resources, prepare and run the benchmark, and release previously acquired resources. CWB is used to conduct a case study in the Amazon EC2 cloud to examine a very specific performance characteristic for a combination of different service types. The results reveal limitations of the cheapest service type, show that performance does not necessarily correlate with service pricing, and illustrate that a new service type at the time of conducting this case study is able to reduce variability and enhance performance. CWB has already executed nearly 20000 benchmarks in total and is used in ongoing research.

Static source code analysis is both time consuming and resource intensive. Hence, existing software studies only analyze a small number of snapshots of a few software projects to draw their conclusions. On the other hand, research in the field of Mining Software Repositories (MSR) analyzes commit logs, bug reports and other data over the entire lifetime of thousands of projects, but without analyzing the source code itself. With the goal of bridging the gap between software studies and MSR research, we determined the feasibility of large-scale software analysis. We have developed a graph-based algorithm for calculating code metrics for all revisions of a single software project simultaneously. The prototype we developed is designed as a language-independent, zero-configuration tool, able to produce results given just a Git repository URL. In a pilot study, we established that our prototype is three orders of magnitude faster than comparable tools when analyzing multiple revisions of a software project, making it feasible to analyze every revision of a software project within minutes or hours, whereas producing the same data using existing tools would require weeks or months. As such, our approach enables new MSR research which includes source code as an essential artifact.

The large scale of the Internet has offered unique economic opportunities, that in turn introduce overwhelming challenges for development and operations to provide reliable and fast services in order to meet the high demands on the performance of online services. In this paper, we investigate how performance engineers can identify three different classes of externally-visible performance problems (global delays, partial delays, periodic delays) from concrete traces. We develop a simulation model based on a taxonomy of root causes in server performance degradation. Within an experimental setup, we obtain results through synthetic monitoring of a target Web service, and observe changes in Web performance over time through exploratory visual analysis and changepoint detection. Finally, we interpret our findings and discuss various challenges and pitfalls.

The transition to cloud computing offers a large number of benefits, such as lower capital costs and a highly agile environment. Yet, the development of software engineering practices has not kept pace with this change. The design and runtime behavior of cloud based services and the underlying cloud infrastructure are largely decoupled from one another, which limits both the efficiency of the cloud environment and the Quality of Service which can be delivered to the hosted applications. This paper describes the innovative concepts being developed by CloudWave to utilize the principles of DevOps to create an execution analytics cloud infrastructure where, through the use of programmable monitoring and online data abstraction, much more relevant information for the optimization of the ecosystem is obtained. Required optimizations are subsequently negotiated between the applications and the cloud infrastructure to obtain coordinated adaption of the ecosystem. Additionally, the project is developing the technology for a Feedback Driven Development Standard Development Kit which will utilize the data gathered through execution analytics to supply developers with a powerful mechanism to shorten application development cycles.

At the beginning of a change task, software developers spend a substantial amount of their time searching and navigating to locate relevant parts in the source code. Current approaches to support developers in this initial code search predominantly use information retrieval techniques that leverage the similarity between task descriptions and the identifiers of code elements to recommend relevant elements. However, the vocabulary or language used in source code often differs from the one used for describing change tasks, especially since the people developing the code are not the same as the ones reporting bugs or defining new features to be implemented. In our work, we investigate the creation of a dictionary that maps the different vocabularies using information from change sets and interaction histories stored with previously completed tasks. In an empirical analysis on four open source projects, our approach substantially improved upon the results of traditional information retrieval techniques for recommending relevant code elements.

At the beginning of a change task, software developers search the source code to locate the places relevant to the task. As previous research and a small exploratory study that we conducted show, developers perform poorly in identifying good search terms and therefore waste a lot of time querying and exploring irrelevant code. To support developers in this step, we present an approach to automatically identify good search terms. Based on existing work and an analysis of change tasks, we derived heuristics, determined their relevancy and used the results to develop our approach. For a preliminary evaluation, we conducted a study with ten developers working on open source change tasks. Our approach was able to identify good search terms for all tasks and outperformed the searches of the participants, illustrating the potential of our approach. In addition, since the used heuristics are solely based on textual features of change tasks, our approach is easy and generally applicable and can leverage much of the existing work on feature location.

Software developers make programming mistakes that cause serious bugs for their customers. Existing work to detect problematic software focuses mainly on post hoc identification of correlations between bug fixes and code. We propose a new approach to address this problem --- detect when software developers are experiencing difficulty while they work on their programming tasks, and stop them before they can introduce bugs into the code. In this paper, we investigate a novel approach to classify the difficulty of code comprehension tasks using data from psycho-physiological sensors. We present the results of a study we conducted with 15 professional programmers to see how well an eye-tracker, an electrodermal activity sensor, and an electroencephalography sensor could be used to predict whether developers would find a task to be difficult. We can predict nominal task difficulty (easy/difficult) for a new developer with 64.99% precision and 64.58% recall, and for a new task with 84.38% precision and 69.79% recall. We can improve the Naive Bayes classifier's performance if we trained it on just the eye-tracking data over the entire dataset, or by using a sliding window data collection schema with a 55 second time window. Our work brings the community closer to a viable and reliable measure of task difficulty that could power the next generation of programming support tools.

Software quality reviews are an important part of the software development process. A software quality review is done by one or multiple stakeholders who have to collaborate. Either no single person has all the necessary knowledge or different kinds of stakeholders are responsible for different aspects of the software quality. Since distributed development is a common situation, the collaboration happens often remote via the Internet. We developed the web application SQA-Collaboration, which supports real-time collaboration in distributed software quality reviews. Furthermore, it supports the transfer and comprehension of the review results. The application provides user with shared workspaces, which enable them to query and arrange data as well as establish a shared view on it. The data of the repositories Jira, Sonar and Github can be imported and integrated in a semantic model. By means of the semantic information, queries are recommended to the users and relations between the data on a workspace are highlighted. An event history allows for the reconstruction of operations on the workspace and a replay of conducted reviews.

Chronic pain is a severe and highly complex problem that affects a substantial part of the population in Europe and the United States. The PainHawk project aims at improving the situation of chronic pain patients by developing the sophisticated electronic pain diary CatchMyPain and analyzing the aggregated anonymized user data. In the context of this project, we developed an integrated infrastructure that allows the execution of different data mining approaches on arbitrarily shaped pain diary data, especially pain drawings. A web-based front end that allows the visual analysis of data mining results as well as the comparison of pain diaries was designed and implemented. These tools were used to perform a cluster analysis on the current data stock of CatchMyPain. The discovered clusters were evaluated using the feedback of medical scientists and physiotherapists specialized on chronic pain. The results of the study showed that there are medically relevant structures in the aggregated pain diary data of CatchMyPain and that they can be found by the proposed architecture. Different measurements demonstrated the performance of the approach even on very large datasets.

Persuasive technology to motivate healthy behavior is a growing area of research within HCI and ubiquitous computing. The emergence of commercial wearable devices for tracking health- and fitness-related activities arguably represents the first widespread adoption of dedicated ubiquitous persuasive technology. The recent ubiquity of commercial systems allows us to learn about their value and use in truly “in the wild” contexts and understand how practices evolve over long-term, naturalistic use. We present a study with 30 participants who had adopted wearable activity-tracking devices of their own volition and had continued to use them for between 3 and 54 months. The findings, which both support and contrast with those of previous research, paint a picture of the evolving benefits and practices surrounding these emerging technologies over long periods of use. They also serve as the basis for design implications for personal informatics technologies for long-term health and fitness support.

In software engineering, different tools have become popular to support the development and management of software projects such as version control repositories or issue trackers. These tools are (web-based) approaches that try to offer best possible conditions to develop software, especially in teams using agile development methods. However, if the tools are not used as intended, it is harder to obtain an overview of the project. To avoid this problem, conventions on the usage of these tools can be defined, as in the source code development, where it is generally accepted that defining conventions on how to structure the code and complying them leads to better results. Thus, there are many applications to analyse the code and find violations of these conventions. Similar analyses are possible to find violations of the conventions on the usage of the above mentioned tools, but there are no applications that provide this data. This gap is tried to be filled by the SQA-Pattern approach presented in this thesis. SQA-Pattern mines the data from the different tools and analyses it for certain patterns that describe violations of the tool usage conventions. The goal thereby is to provide the data as fast as possible such that the approach has the potential to be used in agile software development. The conducted evaluation has proved that the data can be analysed in less then ten minutes even for large-scale software projects.

Stakeholders of modern software engineering environments rely on a variety of tools which support the development process. Examples are source code repositories and issue trackers as well as tools to support practices such as Continuous Integration (CI) and Software Quality Assurance (SQA). Considering the available data is scattered across multiple tools featuring different user interfaces, it can be difficult for a stakeholder to maintain awareness of the projects event history. To assess the cause of certain events such as build failures or metric changes, stakeholders require insights into preceding events in context of time. An approach featuring a chronological visualisation of software evolution data called SQA-Timeline was implemented in this thesis. The tool visualises various event types and metrics on a timeline. With its drill-down functionality it offers different information granularities from an overview down to a source code view. A user study was conducted to find out how the approach can foster the understanding of software evolution. The results showed that the participants using SQA-Timeline solved certain tasks with higher correctness and efficiency than users which had a set of commonly used tools at their disposal.