Lead Quality Automation Engineer - AI Platform at InSite Lead Quality Automation Engineer - AI Platform at InSite in Mount Rainier, Maryland Posted in 2 days ago.

Type:

full-time

Job Description:

Role Summary We are seeking a Senior Test Automation Engineer & Quality Engineering Lead to establish and scale quality engineering for a modern AI platform and AI-enabled applications. This is a hands-on, player-coach role responsible for defining the automation strategy, building reusable frameworks and critical test suites, integrating quality into CI/CD, and helping engineering teams increasingly own product quality. The role spans web and mobile applications, APIs and backend services, data and integrations, and AI/agent behavior. Key Responsibilities Test Automation & Continuous Quality Define an automation-first strategy across UI, APIs, services, integrations, data, and AI workflows. Build reusable automation frameworks and high-value regression suites, with API/service-level testing as the foundation . Automate critical web, mobile, and end-to-end journeys, including authentication, authorization, multi-tenant behavior, asynchronous interactions, and failure scenarios. Integrate automated tests and quality gates into CI/CD pipelines. Establish test execution monitoring, failure triage, reporting, and flaky-test management. Automate critical performance, latency, resilience, concurrency, and dependency-failure scenarios. AI, Agent & Data Quality Establish repeatable evaluation and regression testing for Generative AI and agentic applications. Validate appropriate tool and data usage, workflow and authorization boundaries, human-approval controls, and handling of missing or conflicting information. Validate groundedness, factual accuracy, relevance, evidence quality, hallucination risk, instruction adherence, latency, and reliability. Test retrieval, tool-calling, multi-step agent workflows, and AI application failure scenarios. Automate data-quality validation covering freshness, integrity, completeness, transformation, traceability, and tenant isolation. Establish representative synthetic and simulated datasets for deterministic regression and AI evaluation. Engineering Leadership Build the initial automation architecture and highest-value test suites. Lead automation design, code quality, and engineering standards. Partner with software, platform, data, AI, and DevOps engineers to embed testability early. Participate in architecture and design reviews and mentor engineers contributing to shared automation frameworks. Promote the right mix of unit, API, integration, end-to-end, performance, exploratory, and AI evaluation testing while driving distributed quality ownership. Experience & Technology Skills Strong candidates will have: Senior-level test automation or quality engineering experience, including designing automation frameworks from the ground up. Strong Python skills for API, service, data, and AI test automation. Experience with TypeScript/JavaScript and modern web application automation; TypeScript preferred. Strong web, API/service, integration, and end-to-end automation experience. Experience testing React-based applications ; Playwright or comparable browser automation preferred. Ant Design experience is desirable. Experience with automated testing of iOS and Android applications; cross-platform mobile automation is desirable. Strong experience testing REST APIs , asynchronous services, authentication, authorization, and multi-tenant systems. Strong SQL and data-validation skills across relational, operational, time-series, and document-oriented data. Hands-on experience testing applications and services deployed on AWS ; AWS application, data, container, and observability services are strongly preferred. Experience testing applications built with OpenAI models/APIs or comparable production LLM platforms; OpenAI experience preferred. Hands-on experience with Generative AI and agentic systems , including retrieval, grounding, tool-calling, multi-step workflows, behavioral regression, hallucination/evidence validation, and AI observability. Experience integrating automated testing into Git-based CI/CD workflows; Bitbucket and Bitbucket Pipelines experience preferred. Experience with Docker/containerized applications and container-based test environments preferred. Strong debugging and failure-analysis skills using application logs, metrics, traces, APIs, data, and infrastructure. Candidates should be comfortable contributing production-quality automation code and working directly with software engineers. Success in the Role Success means establishing: Reusable automation frameworks and quality standards. Strong API/service regression coverage with focused UI and end-to-end automation. Continuous data-quality and tenant-isolation validation. Repeatable AI/agent evaluation and behavioral regression testing. Actionable CI/CD quality signals and rapid failure diagnosis. Increasing quality ownership across engineering teams. The goal is to provide fast, trustworthy quality signals that enable teams to release AI-enabled products confidently and efficiently .