Senior Data Engineer

Role Name : Senior Data Engineer
Role Type : Contractual (6 months Renewable)

Key responsibilities
• Design and review data pipelines on Azure Databricks and Azure Data Factory, covering ingestion, transformation and orchestration
• Lead code reviews and design reviews for the team, giving clear feedback on performance, maintainability and best practice
• Shape and maintain the data lake / lakehouse architecture (Delta Lake, ADLS Gen2, medallion layers)
• Build and optimise PySpark and Spark SQL workloads, including performance tuning and cost control on Databricks clusters
• Design data models (dimensional and lakehouse) that support reporting, analytics and downstream data products
• Own production support for the platform: monitor pipelines, run root cause analysis on failures and fix issues at the design level so they don’t come back
• Work with business analysts, data consumers and the data architecture lead to turn requirements into solid technical designs
• Keep documentation current for pipelines, data flows and architecture decisions
• Contribute to data quality, lineage and governance standards across the platform

Must-have requirements
• 7 to 10 years of data engineering experience, with recent work on Azure
• Strong hands-on Azure Databricks experience (PySpark, Spark SQL, Delta Lake, notebooks, jobs, cluster tuning). This is non-negotiable
• Azure Data Factory: building and maintaining production pipelines, triggers, linked services and parameterised frameworks
• Design and architecture mindset: proven experience doing code reviews and design reviews, not only writing new code
• Data lake / lakehouse build experience, ideally having designed one end to end on ADLS Gen2
• Solid SQL and data modelling (star schema, dimensional modelling, lakehouse layers)
• Comfortable owning production support alongside development in a small team
• Available to start within 30 days

Nice to have
• Banking or financial services domain experience
• Credit bureau or credit data experience (a real bonus, but rare, so not expected)
• Microsoft Fabric, Azure Synapse or Snowflake exposure
• Streaming experience with Kafka or Event Hubs
• Orchestration with Apache Airflow, and Python or Scala beyond PySpark
• CI/CD for data platforms (Azure DevOps, Git-based release of Databricks and ADF)
• Migration experience from on-premise data warehouses to Azure
• Mentoring junior engineers
• Certifications such as Databricks Certified Data Engineer Professional or Microsoft DP-203 / DP-700
Technical depth matters more than domain background for this role.