Data Engineer – PySpark & ETL Specialist

About the role

We are looking for a Data Engineer with strong expertise in PySpark and Python to design, develop and maintain scalable data pipelines and ETL solutions. The role involves working with large datasets, cloud data platforms and ensuring data quality across the organization.

Key responsibilities

• Develop and maintain ETL/ELT pipelines using PySpark.
• Build and optimise data‑processing workflows for large‑scale datasets.
• Implement data transformation, validation and cleansing logic.
• Write efficient Spark SQL and Python code for data‑engineering solutions.
• Troubleshoot and optimise Spark jobs for performance and scalability.
• Collaborate with architects, analysts and business stakeholders to deliver data solutions.
• Support data‑quality, governance and production operations.

Required profile

• Minimum 5 years of professional experience in data engineering.
• At least 2 years of hands‑on experience with PySpark.
• Bachelor’s degree in Computer Science, IT, Engineering or a related field.
• Relevant certifications (e.g., Databricks, Azure Data Engineer, AWS Data Analytics) are a plus.

Required skills

• PySpark
• Python
• SQL / Spark SQL
• Apache Spark
• Git
• Databricks
• Snowflake, BigQuery, Azure Synapse, Amazon Redshift
• Azure, AWS, GCP
• Kafka or other streaming technologies
• Airflow or similar orchestration tools
• Delta Lake and Lakehouse architecture