Not logged in.

Contribution Details

Type Master's Thesis
Scope Discipline-based scholarship
Title Efficient in-place iterations in MonetDB
Organization Unit
Authors
  • Xinyu Zhu
Supervisors
  • Michael Hanspeter Böhlen
  • Oksana Dolmatova
Language
  • English
Institution University of Zurich
Faculty Faculty of Business, Economics and Informatics
Date 2023
Abstract Text The enormous growth of stored data continues to challenge our ability to efficiently process and analyze data. Many numerical computations related to state transition based on previous state, e.g., compound interest problems or newton’s method, require a combination of operations from the relational algebra (project, join, select etc.) and iterations. Moreover, many analytical computations, e.g., Markov chain algorithms or various types of regressions, require a combination of operations from the relational algebra, operations from the linear algebra, and iterations. Plenty of attempts have been made to solve combinations of those elements, but still improvement needed on time and space consumption. For example, some solutions focus on fetching data from the database, performing linear algebra operations and iterations, and then putting it back into the database, which lacks the support and optimization of relational algebra. Some solutions focus on linear algebra within the database, but require additional data structures and complicated preset functions, and the correspondence of non-numeric and numerical values is not defined. Moreover, the iteration of some solutions is very space or time consuming due to the operations involving table union and table updates. So, there is still not a very suitable solution that can perfectly combine the three aspects. Our iterations have ability to cover all three aspects at the same time, require no additional knowledge about Python, R, Hadoop or Spark, more tightly integrate with the classic SQL definition, and handle various types of iterations in more flexible and explicit ways. We describe the definition of our iterations in MonetDB, explain major design (motivated by various complicated iterations), and discuss key in-place features and future research directions. Finally, we provide applications and experiment results that show the potential of our solutions.
PDF File Download
Export BibTeX