Data Engineer – Design and Manage Scalable Data Pipelines

About the role

The Data Engineer will design, develop, and maintain robust data pipelines and architectures that enable seamless data flow from multiple sources into databases, data lakes, or warehouses. You will work closely with the data team to ensure data is reliable, clean, and ready for business analysis.

Key responsibilities

• Design, build, and maintain scalable data pipelines for collection, transformation, and integration.
• Maintain databases, data warehouses, and data lakes ensuring integrity, scalability, and reliability.
• Implement efficient ETL/ELT processes to automate data collection, cleaning, and transformation.
• Integrate data from internal and external sources, including APIs, cloud storage, and third‑party systems.
• Optimize data architectures for performance, cost efficiency, and fast retrieval.
• Implement data validation, monitoring, and troubleshooting procedures.
• Manage and optimize cloud infrastructure (AWS, Azure) for storage and processing.
• Automate repetitive tasks using Python and SQL scripts.
• Document pipelines, architectures, and workflows for the IT department.

Required profile

• Degree in Computer Engineering, Data Science, Statistics, Physics, or a related IT field.
• 2‑3 years of experience as a Data Analyst or in a similar data‑engineering role.
• Familiarity with Agile methodologies (Scrum, Kanban).
• Experience with cloud platforms, preferably Azure.

Required skills

• Python (including Apache Spark)
• SQL
• DAX
• Azure
• AWS
• GitHub
• Databricks
• PowerBI
• Linux
• ETL/ELT processes
• API integration
• Data wrangling, modeling, quality, and governance