AI Software Test Engineer

Job Description

AI Software Test Engineer:
On behalf of our Technology client, Procom is searching for an AI Software Test Engineer for an 18-month role. This position is a hybrid position with 4 days onsite at our client’s Ann Arbor, Michigan office.

AI Software Test Engineer - Job Description:
We are seeking a Mid-Level AI Software Test Engineer to design, automate, and execute testing strategies for AI-powered applications, machine learning systems, AI agents, copilots, and generative AI solutions. This role combines traditional software quality engineering with emerging AI validation techniques to ensure AI systems are reliable, safe, accurate, performant, and production-ready.

AI Software Test Engineer - Responsibilities:
• Develop comprehensive testing strategies for AI applications, platforms, and services.
• Validate AI model outputs for accuracy, consistency, reliability, and safety.
• Design and execute functional, integration, end-to-end, regression, and performance tests for AI solutions.
• Create test cases for prompt-driven, agentic, and retrieval-based AI workflows.
• Build and maintain automated test frameworks for AI applications.
• Partner with software engineers, AI engineers, product owners, architects, and security teams.

AI Software Test Engineer - Mandatory Skills:
• Bachelor's degree in Computer Science, Software Engineering, Information Systems, or related field.
• 3-6 years of software testing, QA automation, or quality engineering experience.
• Experience developing automated test solutions using Python, Java, JavaScript/TypeScript, or C#.
• Strong understanding of Test automation and Software development lifecycle (SDLC).
• Experience with API testing and automation tools.
• Experience testing distributed systems, web applications, and APIs.
• Familiarity with CI/CD pipelines.

AI Software Test Engineer – Nice-to-Have Skills:
• Experience testing Generative AI applications and LLM-based systems.
• Familiarity with OpenAI, Claude, Gemini, or Azure OpenAI.
• Experience building evaluation and benchmarking frameworks for AI solutions.
• Experience testing cloud-native applications on Azure, AWS, or GCP.
• Knowledge of responsible AI, AI governance, and AI risk management practices.

AI Software Test Engineer – Assignment Length:
This is an 18-month contract position.

AI Software Test Engineer - Start Date:
ASAP.

AI Software Test Engineer - Assignment Location:
Ann Arbor, Michigan, United States. Hybrid position with 4 days onsite and 1 day remote.

Benefits

Procom provides Basic Life/AD&D insurance, Short- and Long-Term Disability, and EAP at no cost to eligible employees after a waiting period. Medical and dental coverage (employee premium contributions), a non-matching 401(k) plan, and additional voluntary coverage are also available. This role includes legislatively required paid leave; no bonus, commission, or equity compensation applies.