Prompt Engineer/LLM Specialist - Support & Innovation

Job Summary The Prompt Engineer/LLM Specialist - Support & Innovation is responsible for building production-grade large language model (LLM) systems that go far beyond prompt writing. This role focuses on designing and evaluating prompts, developing testing pipelines, orchestrating LLM-based workflows, ensuring system reliability, and supporting innovation initiatives. The specialist will work with evaluation frameworks, RAG architectures, and Python-based automation to maintain high-quality LLM performance in production environments. Key Responsibilities Design and optimize prompts for LLM systems using systematic evaluation methods. Build Python pipelines to support prompt testing, scoring, and evaluation. Architect, refine, and optimize Retrieval-Augmented Generation (RAG) pipelines. Implement orchestration for LLM workflows, including tool usage and agent-based systems. Monitor and improve LLM performance, quality, drift behavior, and failure modes. Document best practices, evaluation outcomes, and system improvements. Support innovation initiatives involving prompt engineering, LLM usage, and applied AI development. Required Qualifications 4+ years of experience in AI, machine learning, natural language processing, or related fields. Strong proficiency in Python development. Hands-on experience working with LLMs, embeddings, and vector databases. Experience deploying LLM systems into production environments. Preferred Qualifications Experience with agentic AI frameworks. Strong background in LLM evaluation, observability, and quality monitoring. Experience with GenAI solutions in financial services or similar regulated industries.