Lead Software Test Automation Architect | Agentic AI Houston, TX or Coppell, TX ( Local candidates will be given high priority) Hybrid 2 days on site Fulltime(Perm role) Job Description

•We are seeking a Lead Test Automation Architect to define and lead an enterprise-grade test automation strategy across UI, API, data, and platform layers-while evolving our quality engineering capabilities to support Agentic AI / LLM-based systems (probabilistic outputs, confidence-based validation, hallucination detection, and model evaluation). This role will architect scalable automation frameworks, drive engineering excellence, and accelerate delivery through highly productive, maintainable automation programs. Key Responsibilities

•Automation Strategy & Architecture Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e), aligned to product risk and release cadence. Establish automation standards, patterns, and reference implementations (framework structure, page-object/screenplay usage, test data patterns, reporting). Drive shift-left testing practices (contract testing, component testing, pre-merge validation) and reduce reliance on fragile end-to-end tests. Agentic

AI / LLM

Test Model Alignment Design and implement testing approaches for non-deterministic AI outputs, including: confidence/threshold-based assertions semantic similarity checks and rubric-based validation goldens/benchmark datasets and evaluation harnesses hallucination and groundedness testing strategies regression detection across model/prompt/version changes Define AI test coverage across accuracy, robustness, bias/safety (as applicable), performance, and reliability. Partner with ML/AI, Product, and Engineering to establish acceptance criteria for AI capabilities and measurable quality gates. High-Productivity Automation Programs Build and scale high-throughput automation projects that are reliable, fast, and easy to extend. Improve test execution time, reduce flaky tests, and optimize CI/CD pipelines (parallelization, test selection, retries with guardrails). Create actionable, engineering-friendly quality dashboards (pass/fail trends, flake rate, coverage, defect escape metrics). Tooling, CI/CD, and Quality Gates Integrate automation into CI/CD with quality gates (PR checks, nightly suites, release readiness criteria).

Standardize observability for testing:

logs, tracing hooks, screenshots/video, artifacts, and test run diagnostics. Define practices for test environments, data management, service virtualization, and dependency control. Technical Leadership & Mentorship Serve as the QE technical authority: coach engineers on automation design, code quality, and best practices. Lead design reviews for test architecture and automation PRs; enforce coding standards and maintainability. Influence roadmap and delivery by translating risk into clear testing priorities. Required Qualifications

•12+ years in Quality Engineering / Test Automation with architect-level ownership of frameworks and strategy. Proven experience designing scalable automation across: API testing (contract + integration) UI automation with stable locator strategies (test IDs, accessibility roles) Test data strategy and environment reliability Strong programming skills in at least one modern language (e.g., Java, Python, TypeScript/JavaScript, C#). Expertise with automation tools/frameworks (examples):

UI:

Playwright / Cypress /

Selenium API:

REST-assured / SuperTest / Postman/Newman / pytest

CI:

GitHub Actions / Jenkins / Azure DevOps Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions. Preferred Qualifications (Strongly Desired) Experience testing LLM/Agentic AI features or other probabilistic systems, including: evaluation harnesses, benchmark datasets, semantic scoring, and drift detection prompt/version regression strategies and structured evaluation reports Familiarity with modern quality approaches: test pyramid, contract testing, consumer-driven contracts, risk-based testing, shift-left. Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus). Experience building reusable automation libraries for multiple teams/products. Regards,  Mayank Xchange Software Inc. 10 Austin Avenue, 2 nd Floor | Iselin, NJ

•08830

Direct Line:

•732-660-7038 Hangout

•recruit.mayanks@gmail.com | msingh@xchangesoft.net