Manager, AI Quality Assurance - #2116507

Deloitte View all jobs

• Manchester Belfast

• Permanent

• Full-time

• 15 hours ago

Connect to your Industry

Deloitte provide premium quality engineering services to a wide range of global clients, solving the most complex testing challenges on their most significant and highest profile IT change programmes. Our team has grown rapidly over recent years, and continues to do so, combining a wealth of deep industry knowledge with both business advisory experience and technical expertise.

Our proposition differs from our competitors as we are not purely delivery focused, we pride ourselves on providing a more consultative and strategic approach to quality engineering which results in our clients experiencing a premium service and gaining higher value outcomes.

Business Operations

Want to work with the latest technologies, as part of a team helping international organisations transform their core business capabilities? Whether you're working with on-premise systems, Cloud technology, or custom-developed solutions (to name just a few), your expertise will help clients deliver continuous improvement and innovation, from initial design right through to implementation and optimisation.

Connect to your career at Deloitte

Deloitte drives progress. Using our vast range of expertise, we help our clients' become leaders wherever they choose to compete. To do this, we invest in outstanding people. We build teams of future thinkers, with diverse talents and backgrounds, and empower them all to reach for and achieve more.

What brings us all together at Deloitte? It's how we approach the thousands of decisions we make every day. How we behave, our beliefs and our attitudes. In other words: our values. Whatever we do, wherever we are in the world, we lead the way, serve with integrity, take care of each other, foster inclusion, and collaborate for measurable impact. These five shared values lead every decision we make and action we take, guiding us to deliver impact how and where it matters most.

Connect to your opportunity

We're looking for Managers who enjoy leading strong QE teams and influencing how quality is built into AI-based solutions from the ground up. As AI becomes a core part of most upcoming programmes, you'll play a key role in shaping how quality is assured and measured across the AI SDLC, working closely with data scientists, engineers and business stakeholders. You'll be responsible for making practical decisions around model and data quality, addressing challenges like bias and drift. Along the way, you'll stay at the forefront of the field by exploring emerging tools and technologies in AI quality assurance, while championing the value of AI testing across the wider Deloitte community, helping our teams and clients deliver complex, AI-based programmes with confidence and credibility.

You might find yourself:

• Designing and delivering AI QA approach for Traditional AI and Generative AI systems, aligned to the AI development lifecycle and client objectives.
• Developing structured test cases to evaluate model behavior, performance and risk across a range of use cases, including edge cases and adversarial scenarios.
• Validating input and output data across AI pipelines, assessing completeness, consistency, correctness and relevance
• Supporting and governing test data preparation, validation and augmentation, including the creation of representative datasets
• Implementing AI test automation strategies, selecting appropriate frameworks and approaches to enable scalable, repeatable and efficient testing of AI systems.
• Apply appropriate tools and techniques for model observability and evaluation.
• Analyse AI model behavior and outputs to identify ethical, fairness and responsible AI risks
• Collaborating closely with data scientists, DevOps and delivery teams to embed AI testing practices early and consistently across the development lifecycle.
• Mentoring others and contributing to Deloitte's internal tools, accelerators, and communities of practice.

Connect to your skills and professional experience

You're curious, collaborative, and hands-on.

You know AI systems are different to conventional systems and require a tailored QA approach to handle characteristics such as non-determinism. You're comfortable moving beyond simple pass/fail thinking, bringing clarity to complex problems through strong communication and helping teams build confidence in AI systems through effective, real-world quality engineering.

• In-depth knowledge of Traditional AI and Generative AI model architectures, including training processes and lifecycle considerations.
• Knowledge of AI testing methodologies, including specialised techniques for testing AI models and relevant evaluation metrics.
• Experience in test automation for AI systems, including strategy, design and implementation.
• Ability to perform data analysis to ensure quality and adherence to relevant data compliance standards.
• Ability to identify and assess ethical, fairness and responsible AI risks within AI-enabled systems.
• Knowledge of AI model monitoring, including drift detection, performance tracking and ongoing model evaluation.
• Advanced data preparation and augmentation capabilities to support AI testing scenarios.
• Experience analysing AI model behaviour and outputs, and embedding findings into quality, risk and assurance decisions.
• ISTQB Certified Tester Foundation Level certificate (or equivalent practical experience).
• Strong experience using test management tools (e.g. JIRA) to govern quality activities at engagement level.
• Client-facing experience, including stakeholder management at senior levels.
• Proficiency in driving adherence to testing governance, including documentation controls and sign-off readiness.
• Excellent interpersonal skills, with the ability to lead, influence and mentor teams.
• Acute attention to detail, particularly in complex or AI-driven delivery contexts.

Desirable Skills

• Proven track record delivering within a top-tier consultancy environment.
• ISTQB Certified Tester - AI Testing.
• Experience administering and governing test management tools (e.g. HP ALM, Zephyr).
• Experience working in agile teams, using tools such as Jira, or Azure DevOps.
• A collaborative mindset and strong communication skills.

Connect to your business - Technology and Transformation

Distinctive thinking, deep expertise, innovation and collaborative working. That's what connects us. That's what makes us Deloitte. If you want to help solve some of the biggest tech and transformational challenges around, join us. Together, we'll make an impact that matters.

Engineering, AI and Data

We...