Data Engineer

Contract - Cape Town
Responsibilities
• Understand business requirements and client needs.
• Analyse and interpret supporting data to recommend effective data solutions.
• Identify opportunities to improve data processes and reporting capabilities.
• Design and develop data layers for Big Data analytics.
• Build and maintain data replication and ETL pipelines.
• Extract, transform, and load data from multiple data sources into centralised data lakes and warehouses.
• Design data warehouse and data product layers.
• Develop, test, and maintain high-quality data pipelines.
• Support existing solutions within GCP and Oracle environments.
• Create and maintain technical documentation.
• Conduct peer reviews and ensure compliance with SDLC standards and documentation practices.
• Work closely with product teams, data analysts, and stakeholders.
• Provide regular progress updates and task estimations.
• Log and manage change requests.
• Participate in standby rotations when required.
• Collaborate effectively within the wider technical team environment.
• Deliver high-quality, value-driven data solutions.
• Monitor ticket progress and ensure timely issue resolution.
• Manage and optimise GCP costs and infrastructure responsibilities.
• Ensure infrastructure supports efficient and reliable data extraction, transformation, and loading processes.
Qualifications & Experience
• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• 4–5+ years’ experience as a Data Engineer or within a data-related environment.
• Strong knowledge of cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP).
• Hands-on experience with the GCP data ecosystem, including BigQuery, Cloud Dataflow / Apache Beam, and Cloud Composer / Apache Airflow.
• Strong SQL skills and experience working with relational databases such as Oracle, PostgreSQL, MySQL, and SQL Server.
• Experience with dimensional modelling and data analysis.
• Knowledge of ETL processes, data pipelines, and data warehouse development.
• Understanding of version control tools such as Git.
• Proficiency in Python and SQL programming.
• Broad understanding of Software Development Life Cycles (SDLC).
• Exposure to Agile methodologies.
• Experience with data visualization tools and retail industry processes will be advantageous.
Competencies
• Strong sense of ownership and accountability.
• Excellent problem-solving and analytical thinking skills.
• Ability to simplify complex technical requirements and environments.
• Strong planning, organisational, and multitasking abilities.
• High attention to detail and accuracy.
• Ability to work effectively in a fast-paced and changing environment.
• Curious mindset with a passion for analysing and solving problems.
• Strong teamwork and collaboration skills.
• Excellent verbal and written communication skills.
• Self-motivated and passionate about technology, people, and business processes.
    © 2024 Rough Diamonds Studio. All rights reserved.