Databricks Data Engineer

About the role

We are looking for a Databricks Data Engineer to design, develop and maintain robust data solutions. You will build scalable pipelines, ensure data quality, and support advanced analytics across cloud environments.

Key responsibilities

• Develop high‑quality ETL/ELT pipelines using Databricks technologies such as Delta Lake, Auto Loader and DLT.
• Write production‑grade Python and PySpark code for data transformation, validation and PII masking.
• Configure Unity Catalog, manage access controls, audit logging and data masking.
• Implement ingestion patterns with checkpointing, schema evolution and serverless compute.
• Integrate BI tools (Power BI, Tableau, Looker) via optimized ODBC/JDBC connectors and Unity Catalog security.
• Build stakeholder‑facing SQL dashboards to monitor KPIs and pipeline health.
• Prepare GenAI‑compatible datasets, manage vector embeddings and use Feature Store with MLflow.
• Package and deploy pipelines with Databricks Asset Bundles through CI/CD pipelines (GitHub/GitLab).
• Troubleshoot, tune and optimize jobs using Photon engine, ensuring cost efficiency and SLA reliability.

Required profile

• Minimum 2 years of experience in cloud‑based data engineering, data warehousing and real‑time streaming.
• Strong understanding of data modeling, warehousing concepts and performance optimization.
• Hands‑on experience with at least one cloud platform (AWS, GCP or Azure).
• Familiarity with CI/CD, DevOps practices and data orchestration tools.

Required skills

• Python
• PySpark
• SQL
• Databricks (Delta Lake, Auto Loader, DLT, Unity Catalog)
• Cloud platforms: AWS, GCP, Azure
• CI/CD tools: GitHub, GitLab
• BI integration: Power BI, Tableau, Looker
• Data orchestration: Apache Airflow
• Big data technologies: Spark, Hadoop, Hive, Kafka
• ETL tools: Informatica, Talend, Matillion, Fivetran
• dbt (Data Build Tool)
• MLflow, Feature Store, Vector Search