Our client is seeking a Senior Cloud Data Engineer to join their Engineering team. In this role, you will be a key technical contributor responsible for building, optimizing, and maintaining high-performance data processing engines. You will apply your deep expertise in Spark Best Practices to develop modular, portable PySpark applications, ensuring their platform remains provider-agnostic while operating within modern environments like Microsoft Fabric. As an experienced member of the Data Engineering team, you will focus on refactoring legacy logic into scalable Python-centric solutions, optimizing their data lakehouse architecture, and contributing to the overall technical excellence of our client's data platform

Requirements:

·        Bachelor’s degree in computer science, Information Systems, Engineering, or a closely related field.

·        Senior-Level Spark: 6+ years of experience with Spark/PySpark, with the ability to diagnose performance bottlenecks via the Spark UI and optimize complex DAGs.

·        Advanced Python: Strong proficiency in production-grade Python, with experience building reusable libraries and implementing automated testing.

·        SQL Proficiency: Solid T-SQL skills to accurately interpret and migrate existing logic into the new Python-centric environment.

·        Azure Synapse Analytics & Dedicated SQL Pools: Solid experience working within Azure Synapse, specifically with Dedicate SQL Pools and Data Factory working with complex pipelines

·        Data Lakehouse Expertise: Hands-on experience with Delta Lake and Parquet, specifically in high-volume environments.

·        Agnostic Mindset: Experience working with open-source standards and containerization (Docker) to ensure data workloads can be ported across different cloud infrastructures.

·        Collaborative Engineering: Proven track record of working within a team to successfully deliver and modernize large-scale data workloads.

·        Ensure high levels of security, compliance, and data governance across data engineering processes.

Should you meet the requirements for this position, please email your CV to it.vacancy@stand-outstaffing.co.za. You can also contact the team on 031 350 4019 or alternatively you can visit our website www.stand-outstaffing.co.za

Should you not hear from us within 3 days, please consider your application unsuccessful.