19.02.2026 aktualisiert


40 % teilweise verfügbar
Senior Data Engineer - Big Data & Cloud Architect
Berlin, Deutschland
Nur Remote
Bachelor of Science in Software EngineeringÜber mich
9+ years of experience,including architecting terabyte-scale pipelines at DHL ,and a history as a Top-Rated Level 2 Freelancer. am looking for a fractional or contract-based role (10–20 hrs/week) where I can serve as a Lead/Architect to build scalable/robust pipelines & architectures to maximise ROI
Skills
JavaAPIsApache AirflowAmazon Web ServicesAnalytische VerfahrenData AnalysisApache HTTP ServerArchitekturMicrosoft AzureBash ShellBig DataC#Cloud ComputingDatenbankenContinuous Integration
Big Data Architecture & Engineering
Expertise in designing, developing, and deploying scalable Big Data, ETL/ELT, and advanced analytics solutions across Azure, AWS, and Hybrid Cloud architectures with 9+ years of experience
Data Lake & Warehouse Solutions
Proficiency in architecting Data Lake, Data Warehouse, and Data Mesh solutions using Snowflake, Azure Synapse, Fabric, Databricks Delta Lake, Apache ICEBERG, and Athena
Apache Spark & Scala Development
Advanced skills in engineering complex transformations using Scala Spark and PySpark APIs for high-performance analytical queries and data processing
Cloud Platforms
Experience with Azure, AWS, GCP, Databricks, Snowflake, Azure Fabric, and Azure Synapse
Data Engineering Tools
Proficiency in Apache Spark, Spark SQL, Spark Streaming, T-SQL, Kafka, AWS EMR, Glue, Athena, and Hadoop Ecosystem
DevOps & CI/CD
Knowledge of Azure DevOps Pipelines, Git, GitHub Actions, Jenkins CI/CD Pipelines
Programming Languages
Skills in Python, Scala, Java, SQL, C#, Shell/Bash Scripting
Data Orchestration
Experience with Azure Data Factory, AWS Step Functions, Apache Airflow, NiFi, Docker, Kubernetes
Database Technologies
Expertise in SQL Server, PostgreSQL, Oracle, Dynamo DB, MongoDB, Firebase, HBase
Expertise in designing, developing, and deploying scalable Big Data, ETL/ELT, and advanced analytics solutions across Azure, AWS, and Hybrid Cloud architectures with 9+ years of experience
Data Lake & Warehouse Solutions
Proficiency in architecting Data Lake, Data Warehouse, and Data Mesh solutions using Snowflake, Azure Synapse, Fabric, Databricks Delta Lake, Apache ICEBERG, and Athena
Apache Spark & Scala Development
Advanced skills in engineering complex transformations using Scala Spark and PySpark APIs for high-performance analytical queries and data processing
Cloud Platforms
Experience with Azure, AWS, GCP, Databricks, Snowflake, Azure Fabric, and Azure Synapse
Data Engineering Tools
Proficiency in Apache Spark, Spark SQL, Spark Streaming, T-SQL, Kafka, AWS EMR, Glue, Athena, and Hadoop Ecosystem
DevOps & CI/CD
Knowledge of Azure DevOps Pipelines, Git, GitHub Actions, Jenkins CI/CD Pipelines
Programming Languages
Skills in Python, Scala, Java, SQL, C#, Shell/Bash Scripting
Data Orchestration
Experience with Azure Data Factory, AWS Step Functions, Apache Airflow, NiFi, Docker, Kubernetes
Database Technologies
Expertise in SQL Server, PostgreSQL, Oracle, Dynamo DB, MongoDB, Firebase, HBase
Projekthistorie
Leading ingestion of TeraByte-scale E-Commerce logistics data, optimizing consolidation pipelines, designing scalable frameworks for JSON data transformation, and implementing Delta Lake solutions
Implementing data engineering workflows, performing data collection and transformation, developing ETL processes, and working with Apache ICEBERG tables and AWS EventBridge for streaming pipelines
Implementing big data pipelines and solutions for ETL/ELT, applying optimized transformations using Spark, persisting data in Snowflake Data Warehouse, and using Azure Data Factory for data transformation