Inhalt des Dokuments
Adaptive Resource Management (ARM)
[1]
- © ARM / DOS / TUB
The ARM subgroup, lead by Dr. Lauritz Thamsen [2], works at the intersection of distributed systems, operating systems, and software engineering, focusing on adaptive resource management in critical and data-intensive distributed systems.
News
- December 2020: We are going to present a couple of recent works at IEEE Big Data 2020 in December! The pre-print PDFs are online already here: [1] [3], [2] [4], and [3] [5].
- November 2020: We are on twitter now: @ARM_TUBerlin [6]. Follow us for the latest news and research results!
Full-Time Researchers
[7]
- © ARM / DOS / TUB
Lauritz [8], Morgan [9], Felix [10], Ilja [11], Kordian [12], Dominik [13], Philipp [14], Jonathan B. [15], and Jonathan W. [16]
Student Assistants
Martin Haug [17], Benjamin Pfister [18], Alexander Scharmann [19], and Houkun Zhu [20]
Research Theme
More and more important applications are relying increasingly on the processing of large volumes of data. These include, for instance, IoT applications for monitoring of traffic, transport systems, water networks, and other critical infrastructures within cities. Other applications monitor the vital parameters of remote patients in distributed telemedicine setups and current environment conditions such as seismic activities using large-scale distributed sensor networks. Moreover, businesses and the sciences have to deal with increasingly large amounts of real-time and historic data, be it to quickly detect fraudulent behavior in millions of payment transactions or comparing terabytes of human genomic data to accurately identify genetic disorders.
For many of these applications, there are clearly defined expectations for the required quality of service in terms of end-to-end latencies, throughput, scalability, availability, as well as the reliability of ingested and produced data. Another major concern is efficiency and especially so, when it comes to the consumption of energy generated from fossil fuels. At the same time, distributed data processing applications are being deployed to more heterogeneous and dynamic environments.
As a result, running large-scale distributed applications is often a very difficult task, especially when given critical targets for performance and dependability. In fact, we argue that – while high-level programming abstractions and comprehensive processing frameworks have made it easier to develop data-intensive applications – efficiently operating critical data-intensive systems and applications has become more difficult over the last decade. And there is abundant evidence of low resource utilization, limited energy-efficiency, and severe failures with infrastructures, systems, and applications deployed in practice that back up this claim.
Addressing these problems, we develop methods, systems, and tools to make the implementation, testing, and operation of efficient and dependable data-intensive distributed applications easier. Towards this goal, we work on adaptive resource management and fault tolerance in distributed heterogeneous computing environments from small IoT devices to large-scale clusters of virtual resources, aiming to create systems that automatically adapt to current workloads, dynamic distributed computing environments, and performance as well as dependability requirements of users and applications.
Topics of Interest
- Resource management, scheduling, data and task placement, system configuration
- Profiling, performance modeling, testing, simulations, testbeds
- Distributed data-parallel processing, scalable batch and stream processing, distributed dataflow systems
- Cluster infrastructures, virtual resources, heterogeneous hardware, real-time operating systems, networked embedded devices, sensor networks
- Big data analytics, internet of things, urban infrastructure applications
- Quality of service, efficiency, scalability, dependability, fault tolerance, usability
Research Methodology
We mostly do empirical systems research. Therefore, we evaluate new ideas by implementing them prototypically in context of relevant open-source systems (such as Flink, YARN, Kubernetes, and FreeRTOS) and then conduct experiments on actual hardware, with exemplary applications, and real-world input data. For this, we have access to state-of-the-art infrastructures, including a 200-nodes commodity cluster, a GPU cluster, our faculty's HPC cluster, private and public clouds, as well as IoT devices and sensors. That is, as far as possible, we empirically evaluate new ideas in their actual environments, making use of emulations and simulations only to be able to investigate more and more large-scale scenarios than physically feasible for us.
At the same time, we also work on practical applications in interdisciplinary projects to experience relevant problems ourselves and, thereby, uncover opportunities for well motivated and impactful research.
/compact.png
s/texts/GeldenhuysThamsenKao_2020_ChironOptimizingFault
ToleranceInQoSAwareDistributedStreamProcessingJobs.pdf
s/texts/WillBaderThamsen_2020_TowardsCollaborativeOptim
izationOfClusterConfigurationsForDistributedDataflowJob
s.pdf
s/texts/LorenzGeldenhuysSommerJakobsLueringSkwarekBehnk
eThamsen_2020_AScalableAndDependableDataAnalyticsPlatfo
rmForWaterInfrastructureMonitoring.pdf
os/arm-team-2020-10-07.png
auritz/
s_morgan/
elix/
lja/
a_kordian/
t_dominik/
philipp/
nathan/
athan/
ve_resource_management/?showp=2&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=08c47a8975d83449a8b7d2b8b2752
9b7
ve_resource_management/?showp=3&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=41841319fd53fb9c875856b7df313
e49
ve_resource_management/?showp=4&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=e9709156d79453a3ee0953509cfeb
522
ve_resource_management/?showp=5&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=644c1f9ae7f6a1ced5979d8f4fd0e
e52
ve_resource_management/?showp=6&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=64e1938cac532c5e54f3bd981fe55
f18
ve_resource_management/?showp=7&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=aa352d9221091b1e4a007f8b7440d
528
ve_resource_management/?showp=8&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=c03e79e0ea2ce0724359485f2bea3
fbf
ve_resource_management/?showp=9&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=05b738fbae4b4f52eb5564d913518
b97
ve_resource_management/?showp=10&tx_sibibtex_pi1%5B
sort%5D=year%3A1&cHash=5f6273366391d19aa7b605c1236a
b97c
ve_resource_management/?showp=11&tx_sibibtex_pi1%5B
sort%5D=year%3A1&cHash=47815b2ad3007ca0036f5805ba49
b65d
ve_resource_management/?showp=2&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=08c47a8975d83449a8b7d2b8b2752
9b7
ve_resource_management/?type=1&tx_sibibtex_pi1%5Bso
rt%5D=author%3A0&cHash=13ea5c1577787b2620fd0df7331f
7b15
ve_resource_management/?type=1&tx_sibibtex_pi1%5Bso
rt%5D=year%3A0&cHash=321d8e39f0e4cc40afa70b3f34d85d
a2
ve_resource_management/?type=1&tx_sibibtex_pi1%5Bso
rt%5D=journal%3A0&cHash=5c9952c710a58a6b51b15eb7e0d
c03e2
ve_resource_management/?tx_sibibtex_pi1%5Bcontentelemen
t%5D=tt_content%3A998361&tx_sibibtex_pi1%5BshowUid%
5D=10824343&cHash=9d42c83086eec11cc7f2ad7bc8b21a64
ve_resource_management/?tx_sibibtex_pi1%5Bcontentelemen
t%5D=tt_content%3A998361&tx_sibibtex_pi1%5BshowUid%
5D=10824344&cHash=67ec88286c58993a2c82a92f0257d5cc
ve_resource_management/?tx_sibibtex_pi1%5Bcontentelemen
t%5D=tt_content%3A998361&tx_sibibtex_pi1%5BshowUid%
5D=10824596&cHash=86f689486818b9e04605141226417cb7
ve_resource_management/?showp=2&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=08c47a8975d83449a8b7d2b8b2752
9b7
ve_resource_management/?showp=3&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=41841319fd53fb9c875856b7df313
e49
ve_resource_management/?showp=4&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=e9709156d79453a3ee0953509cfeb
522
ve_resource_management/?showp=5&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=644c1f9ae7f6a1ced5979d8f4fd0e
e52
ve_resource_management/?showp=6&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=64e1938cac532c5e54f3bd981fe55
f18
ve_resource_management/?showp=7&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=aa352d9221091b1e4a007f8b7440d
528
ve_resource_management/?showp=8&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=c03e79e0ea2ce0724359485f2bea3
fbf
ve_resource_management/?showp=9&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=05b738fbae4b4f52eb5564d913518
b97
ve_resource_management/?showp=10&tx_sibibtex_pi1%5B
sort%5D=year%3A1&cHash=5f6273366391d19aa7b605c1236a
b97c
ve_resource_management/?showp=11&tx_sibibtex_pi1%5B
sort%5D=year%3A1&cHash=47815b2ad3007ca0036f5805ba49
b65d
ve_resource_management/?showp=2&tx_sibibtex_pi1%5Bs
ort%5D=year%3A1&cHash=08c47a8975d83449a8b7d2b8b2752
9b7
/parameter/en/id/114441/?no_cache=1&ask_mail=YD3Fcw
AAEDSEGFGhW%2BGReFsmA8V9JDet8eJmmEZ123n0TuGX106JcA%3D%3
D&ask_name=Lauritz%20Thamsen