17.11.2026 aktualisiert


verifiziert
Premiumkunde
100 % verfügbarSenior Data & AI Engineer
Berlin, Deutschland
Weltweit
PhD in EngineeringÜber mich
Freelance Senior Data & AI Engineer mit 15+ Jahren Erfahrung in skalierbaren Datenplattformen, verteilten Systemen und Production AI. Fokus: Databricks, Spark, Kafka, Python, Java, Cloud, RAG und LLM-Workflows.
Skills
AI AgentsRAGLLMGCPJavaAmazon Web ServicesBig DataData ArchitectureDateninfrastrukturData IntegrationData TransformationDatenbankentwicklungData IntelligencePythonSmart Data
Senior Data & AI Engineer with 10+ years of experience building high-volume data platforms and production AI features on top of them.
What I do
- Data platform architecture: lakehouse design on Databricks and AWS with Spark, Delta Lake, Iceberg and dbt. Data modeling across raw, canonical and serving layers, with lineage, data quality checks and governance.
- Streaming and high-volume processing: Kafka, Kinesis, Flink and Spark Declarative Pipelines, for platforms handling billions of events per day.
- AI on company data: RAG pipelines, embeddings, vector search, reranking, natural language to SQL and LLM-assisted data validation. Metadata and knowledge layers that give LLMs the right business context.
- Production quality for AI: evaluation sets, tracing, monitoring and cost and latency optimization for LLM workloads.
- Migrations and modernization: legacy data warehouse to lakehouse, hand-maintained SQL to dbt, on-premise Hadoop to cloud. Experience migrating 100+ legacy ETL workflows without disrupting downstream reporting.
- Orchestration and reliability: Airflow, CDC ingestion, incident root-cause analysis, CI/CD for data pipelines, monitoring and alerting with Datadog.
- Backend and platform engineering: Python, FastAPI, Java, TypeScript, SQL, Terraform, CDK, Kubernetes, Docker.
Typical engagements
Typical engagements
- End-to-end data platforms: from ingestion and streaming to lakehouse, modeling and serving layers
- Lead or architect role on complex data projects, hands-on from design to production
- Real-time and high-volume data systems
- Taking an AI prototype to production on real company data
- Audit of an existing AI feature: accuracy, cost and latency
Cloud: AWS, Azure, GCP.
Languages: English, German
Erfahrung
Berufserfahrung5 - 10 Jahre
Freelancer seit2018
Sprachen
DeutschgutEnglischverhandlungssicherRussischverhandlungssicherAmharischMuttersprache
Projekthistorie
- Merchant Product Data Catalog: Designed and delivered an up-to-date, standardized catalog optimized for downstream service consumption.
- Real-Time Behavioral Data Ingestion: Integrated real-time behavioral data into a modern Lakehouse framework, enabling instant insights and decision-making.
- Architectural Oversight: Led design strategies to ensure scalable and seamless data integration across platforms.
- ETL & Real-Time Processing: Developed Spark-based ETL pipelines for the catalog and enabled real-time data processing for behavioral ingestion.
- Lifecycle Management: Managed all phases from conceptualization, design, and implementation to DevOps-related tasks, ensuring system efficiency and resilience.
- Big Data: Apache Spark
- Cloud & Infrastructure: AWS (Kinesis, Glue, SQS, SNS, Lambda)
- Data Storage & Warehousing: Redshift
- DevOps & CI/CD: CloudFormation, Jenkins
- Programming & Scripting: Python, Node, Typescript
- Enhanced data accessibility and system responsiveness across the company's data ecosystem.
- Established blueprints for future data initiatives, emphasizing comprehensive architectural and engineering practices.
- Optimization: Fine-tuned streaming applications to achieve superior performance while ensuring consistency and fault tolerance.
- Consultation & Education: Collaborated with engineering teams, imparting knowledge and expertise in crafting new Flink jobs. Advocated for and instructed on best practices to ensure optimal design and implementation.
- Key Technologies & Stack:
- Streaming & Processing: Apache Flink, optimized for high-throughput and low-latency performance.
- Event Streaming & Change Data Capture: Kafka and Debezium, ensuring real-time data handling and integrity.
- Programming: Java, tailored for streamlined streaming applications.
- Cloud & Infrastructure: GCP, providing a scalable and robust environment for deployment and operation.
- Containerization & Orchestration: Kubernetes, enabling flexible scalability and efficient resource management.
DataLake & Warehousing Systems Enhancement:
- Designed and executed systems targeting improvements in:
- Operational processes
- Data quality and governance
- System robustness and reliability
- Employed tools and platforms to ensure high-standard solutions and data integrity.