Description

Responsibilities:

• Develop, optimize, and maintain ETL processes using Apache Airflow (Python), Azure Fabric (PySpark), and Azure Data Factory.
• Support and enhance the corporate DWH based on Microsoft SQL Server and Azure Fabric.
• Optimize the performance of ETL processes, SQL queries, and jobs; refactor existing solutions, monitor and resolve bottlenecks; continuously improve development processes.
• Create and maintain up-to-date technical documentation for new and existing ETL processes and data pipelines.
• Collaborate with data analysts and data architects to design DWH architecture and ensure high data quality.
• Cooperate with IT and business stakeholders to gather and analyze requirements and develop optimal solutions to support business processes.

Requirements:

• Strong knowledge of SQL and hands-on experience with relational databases (SQL Server, PostgreSQL, Oracle).
• Proficient in Python — experienced in developing ETL scripts, parsers, and DAGs in Apache Airflow.
• Proven experience with data orchestration tools (Apache Airflow, Dagster, Prefect) and supporting ETL processes in a production environment.
• Practical experience with Microsoft Azure cloud solutions (Data Factory, Synapse Analytics, or Fabric) or other cloud platforms (AWS, GCP).
• Experience with Apache Spark (PySpark or Spark SQL) — developing and optimizing distributed data transformations.
• Solid understanding of DWH, Data Lake, and Lakehouse concepts, as well as key principles of data governance, data quality, data lineage, and best practices in data management.