Senior Data Engineer

Silver Edge Arabia

As a Data Engineer within our AI practice, you will ensure our data workflows are efficient, our pipelines are reliable, and our systems are scalable. Your key responsibilities include:

• GenAI Optimization: Building and maintaining data lakes, warehouses, and feature stores specifically optimized for GenAI workloads.
• Pipeline Engineering: Designing and implementing robust ETL/ELT processes and automated workflows using tools like Apache Airflow or Luigi.
• Architecture Design: Developing large-scale, high-performance data architectures that support real-time data processing and streaming.
• Cross-Functional Collaboration: Working side-by-side with Data Scientists and ML Engineers to understand feature engineering and data augmentation requirements.
• Innovation: Exploring and implementing emerging tools and technologies to enhance our evolving GenAI product development.

Technical Requirements:

• Programming: Proficiency in Python is a must. Familiarity with Scala or Java is highly valued.
• Big Data Frameworks: Hands-on experience with Apache Spark or Hadoop.
• Database Mastery: Expert knowledge of SQL and NoSQL (e.g., PostgreSQL, MySQL, MongoDB, Cassandra).
• AI/ML Context: Solid understanding of data preparation for Machine Learning, including MLOps tools and practices (MLflow, DVC).
• Version Control: Experience managing codebases and documentation via Git or GitLab.