As a Data Engineer within DT One, you will be responsible for designing, developing and maintaining a data platform that works at scale. You will collaborate with Engineering Leads, Engineers, Data Scientists, Business Analyst and other teams within DT One. Your goal is to deliver a great system that helps people with their daily challenges and not just tickets to be delivered in a sprint.
A career with DT One provides invaluable experience in an exciting and rapidly expanding market and an opportunity to be part of a truly global company with a workforce that includes over 50 different nationalities.
Key Role Responsibilities:
• Design, Build and Maintain our Data Platform
• Design & Build a highly reliable data lake that works at scale with a focus on data quality, data integrity and availability
• Build new complex pipelines to extend the capabilities of the team with new sources of data
• Design & Build Tools to empower our Data Scientists, Business Analysts and Engineers working with data (Automated creation of ETL, Data Exploration Tools, Internal Libraries and framework )
• Migrate Machine Learning Application and other Data Application from a POC/MVP State to production grade (high availability, scalability, security)
• Monitor & Maintain all data systems (ETLs, Data Warehouse, Data Applications )
• Provide insights on area of improvements including Data Governance, best practices, large scale processing and anything else you are interested in
• Support other teams by providing guidance on data usage, processing and how they can best leverage the platform you are building
Act as a technologist
• Have a bias towards action and shipping. Start small and grow iteratively.
• You are obsessed with data compliance and security
• You are excited about learning new technologies, languages, working on infrastructure and being exposed to data analysis and see the business impact of what we do
• Stay creative and come up with new ideas based on your technical expertise and experience
• Communicate and share your knowledge with other people through coaching and participation in internal and external events
• You are an expert in medium to large scale data processing systems, preferably based on open source systems or AWS services
• You have strong software development skills in one or more languages such as python, java or scala.
• You have strong skills in SQL as well as advance knowledge of SQL Databases administration
• You have experience with infrastructure on AWS and docker frameworks. Experience in Linux shell scripting is a plus
• You have experience in distributed processing system such as spark, presto, hadoop... Real Time processing is a plus.
• You have experience in Data Warehouse and Data Modelling
Does this sound like you? Apply now!