Job description

Jodayn is seeking an AI QA & Evaluation Lead to establish and manage enterprise-grade quality assurance, validation, and evaluation processes for AI-powered systems and intelligent automation platforms.
The successful candidate will be responsible for ensuring the reliability, consistency, security, explainability, and governance alignment of AI-generated outputs and AI-driven workflows.
This role is critical in maintaining enterprise trust, operational quality, and compliance standards across AI initiatives delivered to government and enterprise clients.
The ideal candidate must possess strong experience in enterprise quality assurance, automation testing, AI evaluation methodologies, and governance-oriented validation frameworks.
AI Evaluation & Validation Design and maintain AI testing and evaluation frameworks Develop benchmarking datasets and output validation methodologies Evaluate AI-generated responses for consistency, accuracy, and reliability Define confidence scoring and quality measurement approaches AI Quality Assurance Implement automated regression testing for AI workflows Monitor prompt performance and behavioural consistency Detect model drift and output degradation Validate multilingual AI outputs including Arabic and English content Governance & Compliance Ensure AI outputs align with enterprise governance standards Support auditability and traceability of AI-generated results Implement review and approval workflows for AI systems Support compliance validation activities for enterprise deployments Security & Risk Testing Conduct AI security and prompt injection testing Validate protections against unsafe outputs and misuse scenarios Support secure AI deployment and operational practices Reporting & Continuous Improvement Produce quality assurance reports and evaluation summaries Recommend improvements to prompts, workflows, and AI models Support continuous optimisation of AI solution quality Required Qualifications Education Bachelor’s degree in Computer Science, Information Systems, Software Engineering, or related field Experience Minimum 5 years experience in enterprise quality assurance or software testing Experience with automation testing frameworks Experience evaluating AI-powered systems preferred Experience in enterprise or regulated environments preferred Required Technical Skills Quality Assurance Enterprise QA methodologies Automation testing frameworks API testing Regression testing methodologies Test case development AI Evaluation Large language model evaluation Prompt engineering concepts AI benchmarking methodologies AI governance and validation concepts Security & Governance Risk assessment concepts AI security testing Auditability and traceability principles Preferred Skills Experience in enterprise AI initiatives Exposure to regulated industries Experience with bilingual AI validation Familiarity with enterprise governance frameworks Core Competencies Exceptional attention to detail Strong analytical thinking Structured problem-solving capability High quality and governance awareness Strong documentation and reporting capability

Preferred candidate

Years of experience

No experience required

Degree

Bachelor's degree / higher diploma