Senior Data Engineer | Databricks & Snowflake Data Architect | Azure | dbt | Data Vault 2.0

Responsibilities:

• Design and implement a new data platform on Snowflake Data Cloud within the
  Microsoft Azure Cloud platform 
• Integrate source data from SAP ERP/SAP BW, SQL Server, PostgreSQL, and Oracle
  using Azure Databricks, Azure Data Factory, and Snowpark for Python 
• Develop a Data Vault 2.0 data model and implement it using Data Build Tool (DBT) on
  Snowflake Data Cloud
• Conduct a Proof-of-Concept (PoC) to design a Retrieval-Augmented Generation (RAG)
  system leveraging Snowflake Cortex for intelligent data retrieval and summarization 
• Orchestrate workflows with Apache Airflow 
• Build CI/CD Data pipelines on Gitlab for full automation of testing and deployment
• Provision and manage infrastructure in Azure Cloud and Snowflake Data Cloud with
  Terraform 
• Manage metadata and data governance through the utilization of OpenMetadata
  Catalog 

RESPONSIBILITIES

• Build ETL-Pipelines using Python/Pandas
• Use Gitlab as CI/CD tool for full automation of testing and deployment
• Design and manage containerized applications with Docker
• Develop and deploy data-driven product & services used by customers
• Solve various data integration challenges
• Bring capabilities to deliver fast reporting and analytics solutions

RESPONSIBILITIES

• Design and implement cloud data platform on Snowflake AWS Cloud:
  • Migration of on-premise SQL-Server database to Snowflake on AWS
  • Integration of Okta as Identity Provider, Tableau as BI Tool
•  Design and implement the Snowflake Layered Security:
  • Single Sign-On and SCIM Integration with Okta as Identity Provider
  • Network Security with AWS Private Link
  • Design Role Based Access Control (RBAC)
  • Design Security Monitoring scripts

RESPONSIBILITIES

• Guide customers through the process of migrating to Snowflake and develop methodologies to improve the migration process
• Deploy Snowflake following best practices, including ensuring knowledge transfer sothat customers are properly enabled and are able to extend the capabilities of Snowflake on their own
• Work hands-on with customers to demonstrate and communicate implementation best practices on Snowflake technology
• Maintain deep understanding of competitive and complementary technologies and vendors and how to position Snowflake in relation to them
• Work with System Integrator consultants at a deep technical level to successfully position and deploy Snowflake in customer environments
• Provide guidance on how to resolve customer-specific technical challenges
• Support other members of the Professional Services team develop their expertise
• Collaborate with Product Management, Engineering, and Marketing to continuously improve Snowflake's products and marketing.

RESPONSIBILITIES

Design and building a data lake on Microsoft Azure Cloud:

• Build ETL pipelines with Azure Databricks, Azure Data Factory, Azure Functions and Talend Data Management
• Data integration from variety of data sources (e.g. relational data from databases such as IBM DB2, MS SQL Server, SAP BW and semi-structured data from web services such as weather data, check24 data)
• Deployment of Azure Database for PostgreSQL for relational data within the data lake
• Deployment of Azure Data Lake Storage gen. 2 for unstructured / semi-structured data within the data lake

RESPONSIBILITIES

Building a scalable big data pipeline moving streaming data and batched data to a data lake on
AWS Cloud:

• Data integration from variety of data sources IBM Mainframe, Oracle, MS SQL Server and SAP BW with Informatica Data Integration Tool and AWS Glue
• Develop python application for automatic generation of Informatica mappings, sessions and workflows
• Deployment of Streaming Platform Confluent Kafka, which is one of the main building blocks for data ingestion process
• Deployment of a Snowflake Data Cloud in AWS Cloud Platform for structured and semi-structured data within the data lake
• Deployment of AWS S3 storage for unstructured and semi-structured data within the data lake

RESPONSIBILITIES

Building a big data platform on Azure HDInsight platform:

• Extract, transform and load data from variety of sources (Oracle, MS SQL Server, SAP BW) to Azure blob storage
• Use HDInsight tools Spark und Hive for transforming and storing data from the Azure blob storage to Hadoop-based storage (HDFS)
• Support data scientist team building a Hadoop-based analytical platform leveraging the analytic tools such as Spark Machine Learning, Apache Mahout

Support Proof of Concept activity designing a machine learning based predictive model for credit card fraud detection leveraging Azure HDInsight platform:

• Design a predictive model using HDInsight tools such as HDFS, HBase, Kafka, Spark
• Build a data pipeline, which extract the master data from HBase noSQL database through Kafka streaming platform to Spark for batch processing
• Build a data pipeline for the transaction data, which is streamed through Kafka streaming platform to Spark for real-time Processing
• Use Spark Machine Learning library to model a prediction for credit card fraud detection

RESPONSIBILITIES

• Provide technical support to the customers (Vodafone Group, Deutsche Telekom, Telefonica O2) for deploying and testing of ZTE mobile and data products
• Organize Proof of Concept with the customers to demonstrate ZTE product capabilities and to specify their potential use cases
• Team management in the ZTE mobile and data-based product development
• Planning and organization of pre-sales activities and meetings with suppliers (Qualcomm, Intel, Microsoft and Google)