Senior Data Engineer

Position: Senior Data Engineer
Location: London, United Kingdom
Work Model: Hybrid
Employment: Contract
Contract Length: 6 months (extendable)

We are looking for an experienced senior data engineer who will be mainly responsible for building a data platform for one of our clients!

About the Data Platform:

The Data Platform will be built and managed “as a Product” to support a Data Mesh organization.
The Data Platform focusses on enabling decentralized management, processing, analysis and delivery of data, while enforcing corporate wide federated governance on data, and project environments across business domains.
The goal is to empower multiple teams to create and manage high integrity data and data products that are analytics and AI ready, and consumed internally and externally.

What does a Data Infrastructure Engineer do?

A Data Infrastructure Engineer will be responsible to develop, maintain and monitor the data platform infrastructure and operations. The infrastructure and pipelines you build will support data processing, data analytics, data science and data management across the business.
The data platform infrastructure will conform to a zero trust, least privilege architecture, with a strict adherence to data and infrastructure governance and control in a multi-account, multi-region AWS environment.
You will use Infrastructure as Code and CI/CD to continuously improve, evolve and repair the platform.
You will be able to design architectures and create re-useable solutions to reflect the business needs.

Responsibilities will include:

Collaborating across the business departments to develop and maintain the data platform
Building infrastructure and data architectures in Cloud Formation, and SAM.
Designing and implementing data processing environments and integrations using AWS PaaS such as Glue, EMR, Sagemaker, Redshift, Aurora and Snowflake
Building data processing and analytics pipelines as code, using python, SQL, PySpark, spark, CloudFormation, lambda, step functions, Apache Airflow
Monitoring and reporting on the data platform performance, usage and security
Designing and applying security and access control architectures to secure sensitive data

You will have:

6+ years of experience in a Data Engineering role.
Strong experience and knowledge of data architectures implemented in AWS using native AWS services such as S3, DataZone, Glue, EMR, Sagemaker, Aurora and Redshift.
Experience administrating databases and data platforms
Good coding discipline in terms of style, structure, versioning, documentation and unit tests
Strong proficiency in Cloud Formation, Python and SQL
Knowledge and experience of relational databases such as Postgres, Redshift
Experience using Git for code versioning, and lifecycle management
Experience operating to Agile principles and ceremonies
Hands-on experience with CI/CD tools such as GitLab
Strong problem-solving skills and ability to work independently or in a team environment.
Excellent communication and collaboration skills.
A keen eye for detail, and a passion for accuracy and correctness in numbers

Whilst not essential, the following skills would also be useful:

Experience using Jira, or other agile project management and issue tracking software
Experience with Snowflake
Experience with Spatial Data Processing