Data Engineer

General Purpose

This role is responsible for designing ETL (Extract, Transform, Load) processes in the cloud with the objective of obtaining both external and internal data sources to feed the team's data repository following guidelines and best practice standards. They are also responsible for automating, optimizing and monitoring production implementations.
 

Duties and Responsibilities

• Design, create and operate robust data extraction processes from internal / external sources into our systems.
• Execute data transformation to meet the requirements of the Data Science team.
• Automate and optimize analytical solutions (End-to-end)
• Optimize code and solution run-time and their processing costs
• Implement automatic data quality checks in our AA solutions to ensure code stability and output quality
• Creation of tests to ensure that the produced Advanced Analytics solutions are up to user standards
• Manage and execute version control methodologies to ensure project stability and that it stays up-to-date with user feedback
• Support of productive projects to ensure optimal runtime and efficiency, as well as minimizing error reports.

Qualifications

• Bachelor's degree in computer science, software engineering, information technology, or a related field. 
• Advanced degree is a plus.
• Proven experience working on real-world data engineering projects, which can be gained through internships, personal projects, or previous job roles.
• Proficiency in programming languages commonly used in data engineering, such as Python, Java, Scala, or Go.
• Strong SQL skills for database design, querying, and data manipulation.
• Knowledge of scripting languages (e.g., Bash) for automation and data pipeline orchestration.
• Understanding of data serialization formats like JSON, Avro, Parquet, and XML.
• Familiarity with various database systems, including relational databases (e.g., SQL Server, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
• Understanding of data warehousing concepts, including data storage, data retrieval, and query optimization.
• Proficiency in designing and implementing data models to support data pipelines.
• Strong ETL (Extract, Transform, Load) skills, including data extraction, transformation, and loading processes.
• Experience with big data technologies and platforms, such as Apache Hadoop, Apache Spark, Apache Kafka, and distributed data processing frameworks.
• Familiarity with cloud platforms like AWS, Azure, or Google Cloud, including services for data storage, data processing, and orchestration (e.g., AWS S3, AWS Glue, Azure Data Factory).
• Proficiency in data pipeline orchestration and automation tools like Apache NiFi, Apache Airflow, or cloud-based equivalents.
• The ability to write scripts and automate data processes for data ingestion, transformation, and integration.
• 30% travel projected