We are looking for a skilled Data Engineer to join our implementation and professional Services team. In this role, you will be responsible for designing, building, and optimizing the data pipelines that support our FOCAL system. You will work closely with machine learning engineers, software developers, and engagement managers to ensure seamless data integration, efficient query execution, and high-performance system scalability. Additionally, you will collaborate with other teams to create high-quality queries and diverse question sets that improve system accuracy and performance.
What you'll do
- Develop and maintain scalable data pipelines and ETL processes to support FOCAL.
- Optimize SQL query generation by designing efficient database schemas, indexing strategies, and query execution plans.
- Collaborate with ML engineers to preprocess, transform, and structure large-scale datasets for AI models.
- Monitor and enhance data pipeline performance, reliability, and scalability.
- Ensure data quality by implementing validation and logging mechanisms.
- Support real-time and batch query execution in FOCAL system by optimizing database interactions.
- Stay up to date with best practices in data engineering, database optimization, and machine learning infrastructure.
Qualifications
- 2+ years of experience in data engineering or related technical roles
- Proficiency in Python for data processing and ETL development
- Strong SQL skills and experience writing complex queries
- Solid understanding of dimensional modeling (star schema, snowflake schema, fact and dimension tables)
- Experience designing or consuming RESTful APIs
- Hands-on experience with ETL/ELT pipeline development
- Working knowledge of Linux command line and server administration
- Working experience with Docker or containerization technologies
- Strong client-facing and communication skills
Nice to Have
- Experience with Apache Airflow for workflow orchestration
- PySpark or distributed data processing frameworks
- Knowledge of API development using Flask, FastAPI, or similar
- Familiarity with integration patterns: batch processing, streaming, webhooks, event-driven architectures
- Experience with authentication mechanisms (OAuth, API keys, JWT)
- Exposure to message queues or streaming platforms (Kafka, RabbitMQ ...etc)
- Data visualization tools (Tableau, Power BI ...etc)
- Feature engineering experience working alongside data scientists
- Understanding of data quality frameworks and governance practices
