Data Engineer – AI Development

About the role

The Data Engineer is a key technical role on the AI Development team, responsible for building and maintaining data pipelines that power AI transformation and analytics initiatives. You will work with modern platforms such as Databricks, Google Cloud Platform and Azure to ensure data is clean, reliable and ready for model training, RAG‑based search and enterprise reporting.

Key responsibilities

• Design, build and maintain scalable ETL/ELT pipelines ingesting data from ERP, CRM and e‑commerce sources for AI agents and analytics.
• Maintain and optimise the Databricks‑based data lakehouse to meet performance and reliability requirements for data science and BI.
• Implement data quality checks and monitoring to guarantee the integrity of data used in AI applications and dashboards.
• Collaborate with AI developers and business stakeholders to prepare and optimise datasets for specific AI use cases.
• Use Databricks features such as MCP and Feature Store to prepare data for machine‑learning models.
• Support data‑governance practices, including data lineage, security, cataloguing and documentation.
• Leverage native Azure and GCP services to build scalable storage and compute solutions.
• Model clean, reliable datasets for Power BI reporting and business‑intelligence initiatives.

Required profile

• Minimum 3 years of experience in data engineering with production‑grade pipelines.
• Proficiency with Databricks, PySpark and Databricks Workflows.
• Hands‑on experience with cloud platforms Azure and Google Cloud Platform.
• Experience implementing data quality, governance and lineage processes.
• Familiarity with BI tools such as Power BI.

Required skills

• Databricks
• PySpark
• Databricks Workflows
• Azure
• Google Cloud Platform
• Data lakehouse architecture
• ETL/ELT development
• Data quality and governance
• Power BI
• Python

What we offer

• Opportunities for continuous upskilling and career advancement within a Fortune 500 solutions integrator.