26.08.2026 aktualisiert
100 % verfügbar
Data Engineer | Azure Databricks | PySpark | Python | SQL | Delta Lake | ETL/ELT| Data Fabric
Hattersheim, Germany, Deutschland
Hattersheim, Germany +25 km
Bachelor of ScienceÜber mich
Data Engineer with 4+ years of experience designing and delivering scalable data integration and analytics solutions within Azure Databricks environments. Strong expertise in Python, PySpark, SQL, Delta Lake, ETL/EL T processes, and Lakehouse architectures.
Skills
Azure DatabricksdbtAzure DevOpsSql Data WarehouseData ArchitectureData ControlData Definition LanguageData Manipulation LanguageData TransformationData Transformation ServicesDatenbankentwicklungAzure SQLAzure Data LakeApache SparkMicrosoft Fabric (Analytics-Plattform)
Data Engineering & Pipeline Development
Expertise in designing and developing scalable data pipelines on Azure Databricks using PySpark, SQL, APIs, CDC and Delta Lake. Umfangreiche Kenntnisse in ETL/ELT-Prozessen, Medallion Architecture sowie Lakehouse-Architekturen mit Bronze, Silver und Gold Layern.
Azure Cloud & Data Platform
Fundierte Kenntnisse in Azure Data Factory, Azure Databricks, ADLS Gen2, OneLake, Fabric, Microsoft Entra ID, Purview und Azure DevOps. Erfahrung in der Implementierung von Unity Catalog und CI/CD-Pipelines zur Automatisierung von Build-, Validierungs- und Deployment-Prozessen.
Programming & Querying
Starke Kenntnisse in Python, SQL und Spark SQL fuer Datentransformation, Distributed Computing und Datenverarbeitung. Einsatz dieser Sprachen zur Entwicklung robuster Datenloesungen in Enterprise-Umgebungen.
Data Integration
Kenntnisse in APIs, Change Data Capture (CDC), Data Ingestion, Data Pipelines und Workflow Automation zur Integration heterogener Datenquellen in strukturierte Datenschichten.
Data Governance
Erfahrung mit Unity Catalog, Metadata Management, Data Quality und Access Management zur Sicherstellung von Plattform-Governance und Datensicherheit.
CI/CD & DevOps
Kenntnisse in Azure DevOps, CI/CD Pipelines, Build & Release Automation sowie Version Control zur Automatisierung von Deployment-Prozessen in Data-Engineering-Workloads.
Analytics & Reporting
Erfahrung mit Power BI, Reporting Solutions, Analytics Platforms und Data Warehousing zur Erstellung von Dashboards und KPI-Ansichten fuer Datenqualitaet und operative Performance.
Collaboration & Agile Methodologies
Kenntnisse in Stakeholder Management, Customer Communication, Workshop Facilitation, Presentation Skills, Cross-Functional Collaboration und Agile Methodologies zur Umsetzung von Datenprojekten in kollaborativen Umgebungen.
Expertise in designing and developing scalable data pipelines on Azure Databricks using PySpark, SQL, APIs, CDC and Delta Lake. Umfangreiche Kenntnisse in ETL/ELT-Prozessen, Medallion Architecture sowie Lakehouse-Architekturen mit Bronze, Silver und Gold Layern.
Azure Cloud & Data Platform
Fundierte Kenntnisse in Azure Data Factory, Azure Databricks, ADLS Gen2, OneLake, Fabric, Microsoft Entra ID, Purview und Azure DevOps. Erfahrung in der Implementierung von Unity Catalog und CI/CD-Pipelines zur Automatisierung von Build-, Validierungs- und Deployment-Prozessen.
Programming & Querying
Starke Kenntnisse in Python, SQL und Spark SQL fuer Datentransformation, Distributed Computing und Datenverarbeitung. Einsatz dieser Sprachen zur Entwicklung robuster Datenloesungen in Enterprise-Umgebungen.
Data Integration
Kenntnisse in APIs, Change Data Capture (CDC), Data Ingestion, Data Pipelines und Workflow Automation zur Integration heterogener Datenquellen in strukturierte Datenschichten.
Data Governance
Erfahrung mit Unity Catalog, Metadata Management, Data Quality und Access Management zur Sicherstellung von Plattform-Governance und Datensicherheit.
CI/CD & DevOps
Kenntnisse in Azure DevOps, CI/CD Pipelines, Build & Release Automation sowie Version Control zur Automatisierung von Deployment-Prozessen in Data-Engineering-Workloads.
Analytics & Reporting
Erfahrung mit Power BI, Reporting Solutions, Analytics Platforms und Data Warehousing zur Erstellung von Dashboards und KPI-Ansichten fuer Datenqualitaet und operative Performance.
Collaboration & Agile Methodologies
Kenntnisse in Stakeholder Management, Customer Communication, Workshop Facilitation, Presentation Skills, Cross-Functional Collaboration und Agile Methodologies zur Umsetzung von Datenprojekten in kollaborativen Umgebungen.
Sprachen
DeutschgutEnglischverhandlungssicher
Projekthistorie
Designed and developed scalable data pipelines on Azure Databricks using PySpark, SQL, APIs, CDC and Delta Lake. Implemented Lakehouse data architecture with Bronze, Silver and Gold layers. Developed data transformation and ingestion processes. Established automated data quality validations and monitoring mechanisms. Implemented Unity Catalog and Microsoft Entra ID capabilities. Designed and maintained CI/CD pipelines using Azure DevOps. Supported performance optimisation, production monitoring and operational troubleshooting.
Built ETL pipelines integrating relational databases, APIs and CDC sources into structured data layers for analytical consumption. Developed data models aligned with business requirements. Designed data transformation processes to improve quality and consistency. Created Power BI dashboards and KPI views. Contributed to enterprise data governance by documenting data structures and transformation logic. Worked closely with business stakeholders and data teams in an Agile environment.
Analysed Seclending and SCS requirements and discussed with BAs. Developed UAT test cases and data based on test requirements. Coordinated with multiple Central Securities Depositories stakeholders for UAT interface testing and settlement validation. Validated inbound and outbound SWIFT MT5XX message flows. Performed end-to-end validation of post trade settlement workflows. Worked within regulatory frameworks including CSDR, SFTR and BaFin guidelines.