2021/01 - 2022/12
Providing a stable data supply for the data science team.

Description of Tasks

Supplying an ML system based on a Hadoop BigData platform.

Connection of data sources via Kafka Connect, further processing via Kafka, nifi, Airflow, Spark.

Data storage in Hive and Postgresql. Encapsulation via Docker.

Monitoring and operation of processes. User support.