Profilbild von Anonymes Profil, Big Data Engineer

Verifiziertes Profil

verfügbar

Letztes Update: 01.02.2024

Big Data Engineer

Firma: Levon Travajyan
Abschluss: PhD in Engineering
Stunden-/Tagessatz: anzeigen
Sprachkenntnisse: deutsch (gut) | englisch (verhandlungssicher) | russisch (verhandlungssicher) | armenisch (Muttersprache)

Skills

As a Big Data Engineer, I leverage my 10+ years of experience to deliver robust, scalable, and innovative data solutions that drive business value and performance. I am adept at transforming raw, unstructured data into strategic insights that inform business decisions and optimize processes. I am passionate about using data as a catalyst for enterprise-wide innovation and growth.

My core competencies include architecting and developing data systems that capably handle, process, and analyze colossal data volumes using distributed computing technologies such as Hadoop, Spark, Flink, and Kafka.I also orchestrate seamless data pipelines using Python or Scala, and construct intricate SQL queries.

Additionally, I am proficient in cloud platforms such as AWS, Google Cloud Platform, and Azure, where I deploy solutions, manage resources, and use data-centric tools like Kinesis, AWS Lambda, EMR, SQS, SNS, Redshift, and Snowflake.

Furthermore, I am skilled in data visualization and storytelling, using tools like Google Dataflow/Apache Beam, Google Pub/Sub, Bigquery, Druid, HBase, Hive, and others to present complex data findings in a compelling and accessible way for stakeholders.

Beyond my technical arsenal, I offer a keen understanding of business dynamics and data-driven growth. I align data strategies with business goals and deliver results that not only meet but exceed expectations.

Projekthistorie

03/2021 - bis jetzt
Big Data Consultant
Klarna (Banken und Finanzdienstleistungen, 5000-10.000 Mitarbeiter)

  1. Merchant Product Data Catalog: Designed and delivered an up-to-date, standardized catalog optimized for downstream service consumption.
  2. Real-Time Behavioral Data Ingestion: Integrated real-time behavioral data into a modern Lakehouse framework, enabling instant insights and decision-making.
End-to-End Leadership Across Projects:
  • Architectural Oversight: Led design strategies to ensure scalable and seamless data integration across platforms.
  • ETL & Real-Time Processing: Developed Spark-based ETL pipelines for the catalog and enabled real-time data processing for behavioral ingestion.
  • Lifecycle Management: Managed all phases from conceptualization, design, and implementation to DevOps-related tasks, ensuring system efficiency and resilience.
Common Tech Stack:
  • Big Data: Apache Spark
  • Cloud & Infrastructure: AWS (Kinesis, Glue, SQS, SNS, Lambda)
  • Data Storage & Warehousing: Redshift
  • DevOps & CI/CD: CloudFormation, Jenkins
  • Programming & Scripting: Python, Node, Typescript
Impact:
  • Enhanced data accessibility and system responsiveness across the company's data ecosystem.
  • Established blueprints for future data initiatives, emphasizing comprehensive architectural and engineering practices.

04/2022 - 06/2022
Apache Flink Consultant
Metro Markt

  • Optimization: Fine-tuned streaming applications to achieve superior performance while ensuring consistency and fault tolerance.
  • Consultation & Education: Collaborated with engineering teams, imparting knowledge and expertise in crafting new Flink jobs. Advocated for and instructed on best practices to ensure optimal design and implementation.
  • Key Technologies & Stack:
    • Streaming & Processing: Apache Flink, optimized for high-throughput and low-latency performance.
    • Event Streaming & Change Data Capture: Kafka and Debezium, ensuring real-time data handling and integrity.
    • Programming: Java, tailored for streamlined streaming applications.
    • Cloud & Infrastructure: GCP, providing a scalable and robust environment for deployment and operation.
    • Containerization & Orchestration: Kubernetes, enabling flexible scalability and efficient resource management.

09/2020 - 12/2020
Data Architect
Zalando (Konsumgüter und Handel, 5000-10.000 Mitarbeiter)

DataLake & Warehousing Systems Enhancement:
  • Designed and executed systems targeting improvements in:
    • Operational processes
    • Data quality and governance
    • System robustness and reliability
  • Employed tools and platforms to ensure high-standard solutions and data integrity.
Tech Stack: Python, Airflow, Databricks, Spark, Redshift, AWS, Collibra Data Governance, GreatExpectations Data Quality.

02/2018 - 12/2020
Data Consultant
Siemens Energy (Energie, Wasser und Umwelt, >10.000 Mitarbeiter)

Energy Turbine Streaming Application & Data Pipeline Design:
  • Spearheaded the design and implementation of streaming applications tailored for energy turbines.
  • Developed robust ETL pipelines, facilitating seamless data injection into DataLake.
  • Provided technical mentorship, enhancing capabilities of Data Engineering and Solution Delivery teams.
  • Established and promoted data architectural standards, best practices, and guiding principles.
  • Managed EMR cluster operations and led DevOps activities for data pipelines.
Tech Stack: Kafka, Kinesis, Kinesis Analytics, Firehose, EMR, Spark, ElasticSearch Service, RDS, Scala, Java, Spring Boot, Python, AWS Lambda, Talend, AWS Redshift, Snowflake, Terraform.

08/2018 - 03/2019
Senior Consultant
Tom Tom

Innovative Navigation Product Development:
  • Pioneered and executed scalable, high-performance navigation solutions.
  • Designed robust pipelines for the ingestion and processing of high-volume traffic routing data.

06/2018 - 09/2018
Technology Architect Consultant
Infosys

Enterprise Big Data Platform Architecture & Implementation:
  • Led architectural design and piloted implementations for a comprehensive Big Data platform.
  • Executed an Enterprise DataLake setup and validated the architecture through multiple pilot builds/PoCs.
  • Managed data ingestion from varied complex sources and spearheaded analytics on the Big Data platform.
  • Delivered Big Data BI solutions and downstream data services, while actively engaging stakeholders and managing business expectations.

12/2016 - 06/2018
Senior Big Data Engineer
Glispa Global Group


Ad Tech Data & Tracking Solutions:
  • Spearheaded the development of tracking, analytical, and warehousing solutions tailored for the ad tech landscape.
  • Crafted both streaming and batch processing pipelines to handle large volumes of advertising data.
  • Orchestrated a seamless migration from on-premise systems to a cloud-based DataLake.
  • Rolled out scalable APIs, optimized to manage over 30K requests/second.
  • Set up a multi-regional Kafka infrastructure with mirror makers to ensure low latency and maximized throughput.
  • Took charge of Cloudera cluster management.
Tech Stack: Java, Spring, AWS, Hadoop, Cloudera Distribution, Apache Spark, Kafka, Flink, Hive, Apache Beam/Dataflow, Google Datasource, Druid, Hbase, Aerospike, Google Pub/Sub, BigQuery, bash scripting, NoSQL, MongoDB, HBase, Cassandra, MySQL, Python.


Reisebereitschaft

Verfügbar in den Ländern Deutschland
Profilbild von Anonymes Profil, Big Data Engineer Big Data Engineer
Registrieren