Find your next tech and IT Job or contract PySpark

Looking to apply your data engineering skills to work that directly informs decision-making across the UK? This opportunity sits within the Office for National Statistics, working on large-scale data platforms that support national priorities, from crisis response to economic insight. Why this role stands out Work on AI-augmented data products and pipelines Contribute to national statistics and cross-government data platforms Play a role in high-impact work such as COVID data insights and cost-of-living analysis Collaborate with analysts and data scientists to unlock the value of complex datasets Be part of a team building reusable, automated data engineering solutions What you’ll be doing Designing and building scalable data pipelines Preparing and linking complex datasets for analysis Improving data quality through cleansing, validation and enhancement Translating business needs into technical solutions Supporting and mentoring others in a collaborative team environment

Role: Data and AI Tech Lead Role Type: Outside IR35 Contract Rate: Negotiable Location: London, UK – Hybrid (Hybrid 1 or 2 Days Onsite a week) Note: No of positions 1. Role Overview: We are seeking a Data & AI Technical Lead to drive the Discovery & Design phase of a large-scale M&A program for a Retail & CPG client. This role will lead the technical direction across a complex, multi-cloud data landscape spanning Azure, AWS, GCP, Databricks, and Snowflake. You will oversee and guide senior Data Architects, ensuring architectural alignment, technical quality, and business relevance. The role sits at the intersection of business strategy, data architecture, and platform engineering, acting as a key liaison between client stakeholders, IT teams, and the Chief Data Office. This is a high-impact leadership role requiring both hands-on architectural depth and stakeholder influence at senior levels. Key Responsibilities: 1. Technical Leadership & Architecture Governance: Provide strategic and hands-on technical leadership across multi-cloud data platforms (Azure, AWS, GCP). Guide and mentor senior Data Architects, ensuring consistency in design patterns, frameworks, and standards. Define interim-state, target-state, and transitional architectures for M&A integration (data platforms, analytics, AI/ML enablement). Establish best practices across Databricks, Snowflake, and cloud-native data services. 2. M&A Discovery & Design Lead data discovery across merging entities: systems, pipelines, governance, and data models Define integration strategy (consolidation vs federation vs hybrid) Identify synergies, redundancies, and risks in data ecosystems Produce high-quality architecture blueprints, roadmaps, and design artifacts. 3. Stakeholder Management & Liaison: Business stakeholders Act as the primary technical interface between: IT / Engineering teams Chief Data Office (CDO) Client-side architects and leadership Translate business objectives into scalable technical solutions Facilitate architecture reviews, design workshops, and decision forums. 4. Data & AI Strategy Enablement Define foundations for advanced analytics, AI/ML, and data products post-M&A Align architecture with data governance, security, and compliance frameworks Enable scalable, future-proof data platforms supporting Retail & CPG use cases (eg, demand forecasting, customer 360, supply chain optimization). 5. Delivery Oversight: Review and validate design outputs from architects and engineering teams. Ensure architectural integrity during discovery and early delivery phases. Manage risks, dependencies, and technical trade-offs. Support estimation, planning, and prioritization activities. Required Skills & Experience: • 15+ years in Data & Analytics, with 5+ years in leadership roles. Strong hands-on experience across: Azure cloud, ADF, DevOps, Power BI, Fabric workspaces AWS cloud, S3, cross-cloud connectivity, API Gateways GCP (Big Query, GCS, Dataflow) Deep expertise in: Databricks (Lakehouse architecture, Delta Lake, Unity Catalog, Delta Sharing, PySpark, Spark SQL, etc.) Snowflake (data warehousing & data sharing, Iceberg) Strong understanding of: Data architecture patterns, industry benchmarking, Data integration patterns (ETL/ELT, streaming, CDC, APIs) Data governance & metadata management. Architecture & M&A Experience: Proven experience in M&A data integration, platform consolidation, or carve-outs Experience designing multi-cloud or hybrid data architectures Familiarity with enterprise architecture frameworks (TOGAF or equivalent) Leadership & Stakeholder Skills: Experience managing or leading senior architects or technical teams. Strong communication skills with the ability to influence C-level stakeholders. Ability to bridge business strategy and technical execution. Experience working in consulting or client-facing roles.

We are looking for an SC cleared Data Fabric engineer strong experience with: Synapse databricks pyspark data factory SQL around 1 years hands on experience with fabric Candidate needs to hold active SC clearance Due to the nature and urgency of this post, candidates holding or who have held high level security clearance in the past are most welcome to apply. Please note successful applicants will be required to be security cleared prior to appointment which can take a minimum 10 weeks. LA International is an award-winning partner of choice for many of the world's most influential companies and government organisations. Holding Enhanced Government Security Accreditation, we are recognised as the European market leader in the delivery of Security Cleared talent to organisations that demand the very highest levels of security, compliance and assurance. An award-winning organisation, having secured the prestigious Queens Award for Enterprise: International Trade over multiple years. We are committed to fostering an inclusive, equitable and accessible workplace where everyone feels valued and supported. We welcome applications from all individuals, regardless of background or identity, and we encourage candidates who may not meet every listed requirement to still apply. If you require any adjustments or support during the recruitment process, please let us know and we will work with you to ensure a fair and accessible experience. Please Note: If a high volume of applications is received, only candidates shortlisted will be contacted.