The role  

As a Senior Data Engineer, you will be at the forefront of designing and building innovative data solutions within a fast-paced agile data engineering team. You will play a crucial role in modernizing our platform to enhance product offerings, closely collaborating with stakeholders across the organization to maximize the value of data for reporting, analytics, and decision-making processes.

In this role, you'll help transform our multi-cloud data landscape, working with streaming technologies, cloud data warehouses, and Lakehouse architectures to unlock new insights and capabilities for our business units. Additionally, you will leverage automation and generative AI to increase operational efficiency and enhance self-service capabilities across the organization.

Key responsibilities  

• Develop and maintain robust and efficient data solutions and products including data transformation, data modelling and reporting on both on-prem and cloud

• Enhance platform integration by leveraging domain events to enable more efficient and real-time data processing

• Collaborate with business stakeholders and analysts to enable self-service analytics through Power BI and other visualisation tools including the use of generative AI features such as Amazon Q and Co-Pilot

• Collaborate with cross-functional teams to translate business requirements into technical solutions, document them as GAPs by aligning them with Architects 

• Build sophisticated solutions to deliver actionable insights, with a strong focus on performance, scalability, and driving the adoption of generative AI to enable intuitive self-service analytics

• Enhance data engineering processes through automation, implementing CI/CD pipelines, and optimizing ETL/ELT flows to improve efficiency and maintainability

• Initiates and experiments with new ideas and approaches that support A&I department's strategy

• Facilitate data discovery and enhance data management processes using platforms like OpenMetadata and ensure KPIs and documentation remain accurate and up-to-date

• Be a champion of best development practices on existing and new technologies

• Proactively support and maintain data products, including handling release management, troubleshooting incidents, and providing on-call support as necessary

• Ensure that you adhere to the Governance, Risk & Compliance (GRC) obligations for your role.
• Identify and raise any non-compliance incidents promptly to your line manager.
• Challenge processes, policies and projects that will negatively impact compliance within the Group.
• Complete all mandatory compliance training assigned to you.
• Reach out to the Compliance Teams if unsure of any of your compliance obligations or the requirements are unclear.

Skills, knowledge, and experience:  

• 5+ years of experience in data engineering, with demonstrated expertise in designing and implementing data applications and products

• Experience implementing modern Data Lake/Lakehouse architectures (e.g., Apache Iceberg) within cloud environments, with strong knowledge of cloud technologies (AWS S3, Redshift or Azure data services)

• Advanced SQL skills with experience optimizing complex queries across different database systems

• Hands-on experience with Apache Kafka and event-driven architectures

• Implementation of ETL processes and data structure

• Experience with developing dashboards and reporting solution using data visualization tools such as Power BI

• Experience with best practices applied to data e.g., testing, CI/CD 

• Good attitude and willingness to learn new skills and technologies

• Self-motivated, proactive with excellent communication skills