Предавање у оквиру предмета на докторским студијама: Савремена истраживања у области информационих система и технологија

Место и време: ФОН, Д-300, четвртак, 26. 1. 2023. у 18.00 часова
Предавач: Artem Trofimov, doktorand na St. Petersburg State University, Mentor: Prof. Boris Novikov
URL: https://research.jetbrains.org/researchers/artem.trofimov/

Наслов:Distributed Stream Processing: Substream Management and Fault Tolerance

Абстракт:

Stream processing aims to handle potentially unbounded input data. In order to ensure efficient and reliable processing of a potentially infinite stream, stream processing engines should solve the following two problems:

1. The first task is to split an infinite stream into finite substreams: usually, aggregation over an infinite stream is not possible, so the stream processing system must determine when a stream with certain properties has ended in order to return the result of aggregation over a substream.

2. The second challenge is fault tolerance. In a distributed system individual compute nodes may fail. The system must recover from failures in a reasonable time (without a complete repetition of calculations) and provide a consistent result.

In this talk we discuss various approaches to solve these two problems in modern distributed stream processing systems.

О предавачу:
Artem Trofimov has both academic and industrial experience in building distributed systems for data science problems. He was a research engineer at JetBrains Research and then joined Yandex Cloud and worked as a team lead of the Yandex DataSphere. Currently, he is a lead of the machine learning infrastructure team at Nebius.

Презентацију одржаног предавања можете преузети на линку.