Big Data Developer

Beschreibung

Responsibilities
Below are some of the responsibilities of a big data Developer:
• Design the Development of our big data platform
• Perform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queries
• Design and implement data stores that support the scalable processing and storage of our high-frequency data
• Maintain our data pipeline
• Customize and oversee integration tools, warehouses, databases, and analytical systems
• Configure and provide availability for data-access tools used by all data scientists
Job Qualifications and Skill Sets
Below are the qualifications expected of a big data engineer:
• 8+ years of relevant data engineering experience
• Bachelor’s degree or higher in computer science, data science, or a related field
• Hands-on experience with data cleaning, visualization, and reporting
• At least 2 years of relevant experience with real-time data stream platforms such as Kafka and Spark Streaming
• Experience working in an agile environment
• Familiarity with the Hadoop ecosystem
• Experience with platforms such as MapReduce, Apache Cassandra, Hive, Presto, and HBase
• Excellent analytical and problem-solving skills
• Excellent communication and interpersonal skills