LLM Gateway Developer Position Available In Middlesex, Massachusetts

Tallo's Job Summary: This job listing in Middlesex - MA has been recently added. Tallo will add a summary here for this job shortly.

Company:
Unclassified
Salary:
JobPart-timeRemote

Job Description

About the RoleWe’re seeking a talented student to join our team in for leaning and developing OS Gateway – our proprietary LLM interface that revolutionizes how businesses interact with AI models. This is a unique opportunity to work on production-grade infrastructure that serves as the backbone for AI applications.

Core ResponsibilitiesAPI Development & IntegrationImplement OpenAI-compatible endpoints for completion, embedding, and image generationBuild translation layers to standardize inputs/outputs across 100+ LLM providersDevelop consistent response formatting ensuring compatibility at [‘choices’][0][‘message’][‘content’]Create robust retry/fallback logic for multi-deployment scenarios (Azure/OpenAI)Security & Authentication ImplementationIntegrate SSO and

OIDC/JWT

authentication protocolsImplement secure key rotation mechanismsDevelop read/write permissions for secret managementBuild comprehensive security features for enterprise deploymentInfrastructure & Management SystemsDesign and implement virtual key management for users and teamsCreate organization and team-based administrative controlsBuild advanced logging systems with audit trails and configurable retentionDevelop key/team-based guardrails with request-based controlsMonitoring & OptimizationImplement comprehensive spend tracking across all LLM usageBuild budget management systems with rate limiting capabilitiesCreate LLM fallback mechanisms for high availabilityDevelop always-on logging infrastructure for debugging and complianceRequired QualificationsBachelor’s degree in Computer Science or related STEM fieldProficiency in Python and/or JavaScript/TypeScriptExperience with

REST API

development and integrationUnderstanding of authentication protocols (OAuth, JWT)Familiarity with cloud services (AWS, Azure, or GCP)OPT/STEM students can also apply for this rolePreferred QualificationsExperience with OpenAI API or similar LLM servicesKnowledge of proxy server architectureFamiliarity with rate limiting and caching strategiesUnderstanding of enterprise security practicesExperience with logging and monitoring toolsBackground in distributed systems

Technical SkillsRequired:
Programming:
Python, JavaScript/TypeScriptAPIs:

RESTful design, JSON, async programming

Version Control:
Git/GitHubDatabases:

SQL and/or NoSQLNice to

Have:

LLM/AI frameworks:

LangChain, OpenAI SDKInfrastructure:
Docker, KubernetesMonitoring:

Prometheus, Grafana, ELK stack

Security:

Key management systems, secret storesWhat We OfferHands-on experience building production AI infrastructureMentorship from experienced engineersExposure to cutting-edge LLM technologiesFlexible remote work environment20 hours/week minimum commitmentOpportunity to contribute to open-source projectsProfessional development in AI/ML engineeringLearning ObjectivesMaster API gateway design patterns and implementationGain expertise in multi-provider AI system integrationDevelop enterprise security and authentication skillsLearn production-grade logging and monitoring practicesUnderstand cost optimization in AI infrastructureAbout Our StartupWe’re a Wyoming-registered tech startup focused on democratizing access to AI through innovative infrastructure solutions. Our OS Gateway platform enables businesses to seamlessly integrate multiple LLM providers while maintaining security, compliance, and cost control.

Application RequirementsLegally authorized to work in United States ( CPT, OPT and STEM is acceptable )Ability to work minimum 20 hours/weekStrong communication skillsSelf-motivated with ability to work independentlyPassion for AI/ML technologiesEqual Opportunity Employer We are committed to building a diverse team and encourage applications from all qualified candidates regardless of race, gender, age, religion, identity, or background. recblid fgdr82ozf3mk2w4d6y0v2xpptv3asr

Other jobs in Middlesex

Other jobs in Massachusetts

Start charting your path today.

Connect with real educational and career-related opportunities.

Get Started