How you'll role

As a Senior Data Engineer, you will play a crucial role in developing and optimizing data pipelines and managing data solutions within our cloud-based services. You will collaborate with cross-functional teams to ensure the success of data-driven projects while leveraging your expertise in cloud platforms and agile methodologies. Your efforts will significantly improve our data handling strategies, contributing to our business outcomes and technological advancements.

What you'll do

• Develop and maintain ETL/ELT pipelines using Apache Airflow and PySpark

• Design database schemas and dimensional models to support data normalization

• Develop and manage data warehouses, data lakes, and data lakehouses

• Work with relational and NoSQL databases including Azure SQL, PostgreSQL, MongoDB, and Google Firestore

• Handle various data file formats such as Avro, CSV, Parquet, and Delta

• Create and manage APIs for data integration with an understanding of REST and microservices architecture

• Ensure codebase maintenance and optimization using Git for version control

• Implement integration testing and quality assurance procedures

• Collaborate with team members for technical design support and project delivery

• Integrate new data sources to meet evolving business requirements