Beschreibung
Data Lake DevOps (m/f/d)
Reference: -en
Start: asap
Duration: 6 MM
Main tasks:
- Innovation and implementation of the cloud Data Lake platform infrastructure as the basis for data engineers to work with their data.
- Optimize existing infrastructure and data ransformation pipelines to become more scalable = Self Service ability
- Optimize data provisioning workflow to allow real time replication of data from on-premise into Data Lake (AWS)
- Implement new features like provider or consumer connections as IaaC (e.g. PowerBI connection towards AWS)
- Implement cost optimizations as part of the platform
- Implement test cases as part of development process
- Support solution architects to define the software architecture
- Take over end to end responsibility for changes throughout delivery pipeline
Main qualifications
- In-depth professional experience in developing and operating software solutions
- Professional experience in handling of large sets of data in the area of big data or data warehousing
- Professional experience in building scalable big data solutions in the area of cloud or Hadoop
- Experience with AWS and hands on experience in building Infrastructure as code and services on top
- Demonstrated ability to build processes that support data transformation, data structures, metadata, dependency and workload management
- Professional experience in designing and developing data pipelines in Python/Spark/Scala
- Practical experience with Terraform and Airflow is preferred
- Fluency in English
Main advantages:
- A highly motivated team and an open way of communication
About us:
IT specialists hold all the cards: for its customers in industry and the public sector, Hays is looking for motivated employees who like looking at the bigger picture and growing with new challenges. We make sure that you benefit from our many years of experience in IT recruitment and will find you the right role to suit your skills and interests – all completely free of charge to you. Register with us and reap the benefits of job offers that are both interesting and relevant to your skills and experience.
My contact at Hays:
Referencenumber:
Make contact:
Phone:
Email: