Find Jobs
Find Jobs Near You – Available Work in Your Location
Automation QA w/ AI testing
Career Insights for Software QA Engineer / Tester
See where this job fits in the broader career landscape. Knowing your career path helps you see what's possible from here.
Scorecard
Based on South Carolina data
Review key factors to help you decide if this role fits your goals. How is this calculated?
What they do
A Software QA Engineer or Tester designs and runs in-depth diagnostic tests to evaluate software and check for problems before new products are marketed. Pilots software and applies tests to check for errors and glitches.
$105,400 / year median in South Carolina
-15% projected decline
Job Description
Automation QA w/ AI testing South Carolina, Fort Mill 09/04/2026 Contract Active
Job Summary
The Evaluation Engineer - AI Models will be responsible for evaluating and validating AI/ML and Generative AI models across business and technical use cases. This role combines extensive Quality Engineering and software testing experience with hands-on AI model evaluation, production-grade LLM testing, test automation, and data-driven analysis. The engineer will design evaluation strategies, validate model accuracy and performance, detect hallucinations and other quality issues, and develop automated and manual testing frameworks. Strong Selenium and Playwright expertise is required, along with excellent communication skills and the ability to collaborate with engineering, product, data science, and business stakeholders. Financial Services or Wealth Management domain experience is preferred. Required Qualifications
- 10+ years of experience in Quality Assurance, Quality Engineering, Software Testing, or related disciplines.
- 2+ years of hands-on experience in AI model evaluation, Generative AI testing, or ML validation.
- Strong hands-on experience with Selenium and Playwright.
- Strong understanding of AI/ML concepts, LLM behavior, prompt evaluation, and model testing methodologies.
- Experience testing production-grade LLMs and AI-powered applications.
- Experience with API testing, test automation frameworks, and data validation techniques.
- Understanding of evaluation metrics such as precision, recall, accuracy, grounding, relevance, and hallucination detection.
- Experience creating automated and manual test strategies and frameworks.
- Strong analytical and problem-solving skills.
- Excellent verbal and written communication skills.
- Ability to work independently and collaborate effectively with cross-functional teams.