Search and AI Evaluations

Join NIST to lead research and development for AI and Generative AI testing, evaluation, and validation across diverse modalities. You will independently design and code web-based measurement applications for data quality, scoring, and analysis. This role involves leading technical publications and providing programming expertise to core research areas. By driving these evaluation processes, you will ensure the integrity of measurement and the rigorous standards of critical agency AI systems.

Qualifications:

Basic Requirements:

Bachelor's degree in computer science or bachelor's degree with 30 semester hours in a combination of mathematics, statistics, and computer science. At least 15 of the 30 semester hours must have included any combination of statistics and mathematics that included differential and integral calculus. All academic degrees and coursework must be from accredited or pre-accredited institutions. In addition to the above requirements, all applicants must have one year (52 weeks) of specialized experience equivalent to at least the GS-12 level (ZP-III at NIST).

Specialized experience is defined as:

Experience in engineering custom web-based applications (Python/React preferred) for manual and automated data labeling, quality assurance, scoring, and performance measurement for text and multimodal AI systems. Experience executing test, evaluation, validation, and verification (TEVV) protocols across text, vision, and audio modalities. Experience in designing scoring frameworks to evaluate generative AI outputs and LLM-as-a-judge protocols. OR Master's or equivalent graduate degree OR A combination of education and experience. Experience refers to paid and unpaid experience, including volunteer work done through National Service programs (e.g., Peace Corps, AmeriCorps) and other organizations (e.g., professional, philanthropic, religious, spiritual, community, student, social). Volunteer work helps build critical competencies, knowledge, and skills, and can provide valuable training and experience that translates directly to paid employment. You will receive credit for all qualifying experience, including volunteer experience. The qualification requirements in this vacancy announcement are based on the U.S. Office of Personnel Management (OPM) Qualification Standards Handbook . If requesting reconsideration of your qualification determination, please refer to the following site: Applicant Reconsideration