TU Berlin

Department of Telecommunication SystemsAdaptive Resource Management

Page Content

to Navigation

Adaptive Resource Management (ARM)

The ARM sub-group is lead by Dr. Lauritz Thamsen and works at the intersection of distributed systems, operating systems, and software engineering, focusing on adaptive resource management in context of critical and data-intensive distributed systems.

Team

Lupe

Research Statement

Research Theme

More and more important applications are relying increasingly on the processing of large volumes of data. These include, for instance, IoT applications for monitoring of traffic, transport systems, water networks, and other critical infrastructures within cities. Other applications monitor the vital parameters of remote patients in distributed telemedicine setups and current environment conditions such as seismic activities using large-scale distributed sensor networks. Moreover, businesses and the sciences have to deal with increasingly large amounts of real-time and historic data, be it to quickly detect fraudulent behavior in millions of payment transactions or comparing terabytes of human genomic data to accurately identify genetic disorders.

For many of these applications, there are clearly defined expectations for the required quality of service in terms of end-to-end latencies, throughput, scalability, availability, as well as the reliability of ingested and produced data. Another major concern is efficiency and especially so, when it comes to the consumption of energy generated from fossil fuels. At the same time, distributed data processing applications are being deployed to more heterogeneous and dynamic environments.

As a result, running large-scale distributed applications is often a very difficult task, especially when given critical targets for performance and dependability. In fact, we argue that – while high-level programming abstractions and comprehensive processing frameworks have made it easier to develop data-intensive applications – efficiently operating critical data-intensive systems and applications has become more difficult over the last decade. And there is abundant evidence of low resource utilization, limited energy-efficiency, and severe failures with infrastructures, systems, and applications deployed in practice that back up this claim.

Addressing these problems, we develop methods, systems, and tools to make the implementation, testing, and operation of efficient and dependable data-intensive distributed applications easier. Towards this goal, we work on adaptive resource management and fault tolerance in distributed heterogeneous computing environments from small IoT devices to large-scale clusters of virtual resources, aiming to create systems that automatically adapt to current workloads, dynamic distributed computing environments, and performance as well as dependability requirements of users and applications.

Topics of Interest

  • Resource management, scheduling, data and task placement, system configuration
  • Profiling, performance modeling, testing, simulations, testbeds
  • Distributed data-parallel processing, scalable batch and stream processing, distributed dataflow systems
  • Cluster infrastructures, virtual resources, heterogeneous hardware, real-time operating systems, networked embedded devices, sensor networks
  • Big data analytics, internet of things, urban infrastructure applications
  • Quality of service, efficiency, scalability, dependability, fault tolerance, usability

Research Methodology

We mostly do empirical systems research. Therefore, we evaluate new ideas by implementing them prototypically in context of relevant open-source systems (such as Apache Flink, Hadoop YARN, Kubernetes, and FreeRTOS) and then conduct experiments on actual hardware, with exemplary applications, and real-world input data. For this, we have access to state-of-the-art infrastructures, including a 200-nodes commodity cluster, our faculty's HPC cluster, private clouds, as well as IoT devices and sensors. That is, as far as possible, we empirically evaluate new ideas in their actual environments, making use of emulations and simulations only to be able to investigate more and more large-scale scenarios than physically feasible for us.

At the same time, we also work on practical applications in interdisciplinary projects to experience relevant problems ourselves and, thereby, uncover opportunities for well motivated and impactful research.

Most Recent Publications

Lorenz, Felix and Thamsen, Lauritz and Wilke, Andreas and Behnke, Ilja and Waldmüller-Littke, Jens and Komarov, Ilya and Kao, Odej and Paeschke, Manfred (2020). Fingerprinting Analog IoT Sensors for Secret-Free Authentication. IoTSPT-ML 2020: 10th International Workshop on Security, Privacy, Trust, and Machine Learning for IoT. IEEE, To appear.


Lauritz Thamsen, Jossekin Beilharz, Vinh Thuy Tran, Sasho Nedelkoski, and Odej Kao (2020). Mary, Hugo, and Hugo*: Learning to Schedule Distributed Data-Parallel Processing Jobs on Shared Clusters. Concurrency and Computation: Practice and Experience. Wiley, e5823.


Behnke, Ilja and Pirl, Lukas and Thamsen, Lauritz and Danicki, Robert and Polze, Andreas and Kao, Odej (2020). Interrupting Real-Time IoT Tasks: How Bad Can It Be to Connect Your Critical Embedded System to the Internet?. IPCCC 2020: 39th International Performance Computing and Communications Conference, To appear.


Navigation

Quick Access

Schnellnavigation zur Seite über Nummerneingabe