Description
LTS is seeking an AI Platform and Harness Engineer to develop and maintain the infrastructure, tooling, and evaluation frameworks that power enterprise AI solutions. This role is responsible for building the AI platform and reusable "AI harnesses" that enable Large Language Models (LLMs), AI agents, Retrieval-Augmented Generation (RAG), and Generative AI applications to be securely developed, tested, evaluated, monitored, and deployed at scale.
The ideal candidate has experience with AI platforms, LLMOps, software engineering, cloud-native technologies, and backend systems, along with a passion for building reliable, observable, and production-ready AI solutions. You will work closely with AI architects, software engineers, data scientists, and product teams to ensure AI solutions are scalable, secure, cost-effective, and continuously improving.
What You'll Do:
- Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications.
- Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance.
- Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance.
- Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics.
- Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement.
- Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking.
- Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity.
- Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations.
- Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications.
- Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures.
- Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment.
- Apply security, governance, and Responsible AI controls throughout the AI development lifecycle.
- Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity.
- Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience.
- Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices.
What We're Looking For:
- Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field.
- 5+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering.
- 2+ years building or supporting Generative AI, Large Language Model (LLM), or machine learning applications.
- Strong programming experience in Python.
- Experience developing APIs, backend services, and distributed systems.
- Experience with cloud platforms including AWS, Azure, or Google Cloud Platform.
- Experience deploying applications using Docker and Kubernetes.
- Experience working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation.
- Strong understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt engineering, Embeddings, Vector databases, AI agents and agentic workflows
- Familiarity with AI evaluation techniques, automated testing, benchmarking, regression testing, and model validation.
- Experience building scalable, production-grade software platforms.
- Strong problem-solving, debugging, and performance optimization skills.
Nice to Have:
- Experience with AI orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, or AutoGen.
- Experience implementing LLMOps or MLOps platforms and deployment pipelines.
- Experience with AI observability tools such as LangSmith, OpenTelemetry, Prometheus, Grafana, Evidently AI, or Arize AI.
- Experience with vector databases including Pinecone, Qdrant, Weaviate, Azure AI Search, or pgvector.
- Experience with OpenAI, Azure OpenAI, AWS Bedrock, Anthropic Claude, Google Vertex AI, or similar enterprise AI platforms.
- Experience implementing Responsible AI, AI governance, model security, and AI safety best practices.
- Experience supporting Federal Government or other regulated environments.
- Experience evaluating AI systems for quality, reliability, accuracy, explainability, latency, and cost optimization.
- Familiarity with healthcare, enterprise modernization, or mission-critical systems.
What’s In It for You?
- The Opportunity to support high-visibility federal missions
- A culture that values innovation, growth, and collaboration
- Access to cutting-edge tools and technologies
- Comprehensive benefits for you and your family
- A career path that rewards ambition and performance
If you’re ready to push boundaries, sharpen your skills, and join a team that is passionate about building what’s next, we’d love to meet you. Apply today and let’s build a future together!
LTS shares salary ranges to promote transparency. Compensation ranges are provided for informational purposes, and final compensation may vary based on experience, skills, location, and role requirements.
LTS is committed to offering eligible employees comprehensive benefits that will provide them with options intended to meet their needs and the needs of their family.
Similar jobs
Est. 140,000 USD
Location: United States – RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a highly skilled Senior Applied AI Engineer to focus on continuously improving the intelligence behind the platform.…
Est. 144,000 USD
Location: United States – RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a Senior Agentic AI Software Engineer to build the intelligence behind the platform—the autonomous agents, orchestra…
Est. 175,000 USD
Location: United States - RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a Lead AI Architect / Strategist to define and lead the architecture, strategy, and implementation of next-generatio…
Est. 175,000 USD
Location: United States - RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a Senior AI Identity Platform Engineer to design and build the identity foundation that enables AI agents, enterpris…
Est. 140,000 USD
Location: United States – RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a highly skilled Agentic AI Security Engineer to ensure our AI systems are secure, trustworthy, resilient, and gover…
Est. 144,000 USD
United States – RemoteClearance: Ability to obtain and maintain a Public Trust LTS is seeking a Senior Front-End Agentic AI Engineer to own the experience layer of an AI platform built for one of the most consequential l…
Est. 200,000 USD
LTS is seeking an accomplished, entrepreneurial AI growth leader to establish, commercialize, and scale our AI services practice. The Director, AI Practice - Development, Growth, and Strategic Partnerships will personall…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
Est. 120,000 EUR
N-iX is a global software development company founded in 2002, connecting over 2,400+ tech professionals across 40+ countries. We deliver innovative technology solutions in cloud computing, data analytics, AI, embedded s…
Est. 195,000 USD
United States – RemoteClearance: Ability to obtain and maintain a Public TrustSalary Range: 175K – 215K LTS is seeking an experienced high-caliber Technical Project Manager to lead day to day execution of a major AI-powe…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
Est. 127,500 USD
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
We're looking for an engineer with hands-on experience building and evaluating GenAI services - from RAG and agentic reasoning systems to production-grade LLM deployments. You'll work closely with Frontend and Backend te…
Est. 144,000 USD
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
Est. 150,000 USD
Staff Platform Engineer Location: This position is a remote role based in the US Company Overview Robots & Pencils is an applied AI engineering firm building the next frontier of business architecture. We design and…
Est. 237,500 USD
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Company Overview Robots & Pencils is an applied AI engineering firm building the next frontier of business architecture. We design and ship AI co-workers that integrate into enterprise operations and deliver measurab…
Est. 185,500 USD
Our Purpose At SentinelOne, we are driven by a clear purpose: to give the advantage to those who secure our future. As AI reshapes how organizations build, operate, and innovate, the responsibility to protect them become…
Est. 157,000 USD
Our Purpose At SentinelOne, we are driven by a clear purpose: to give the advantage to those who secure our future. As AI reshapes how organizations build, operate, and innovate, the responsibility to protect them become…
Est. 180,000 USD
Who We Are Cross River builds the infrastructure behind the world’s most innovative financial products. Our technology and capital solutions power payments, cards, lending, and digital asset capabilities that move money…
Est. 160,000 USD
We’re ALTEN Technology USA, an engineering company helping clients bring groundbreaking ideas to life—from advancing space exploration and life-saving medical devices to building autonomous electric vehicles. With 3,000+…
Est. 215,000 USD
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Est. 193,500 USD
Staff Forward Deployed AI Engineer We're looking for a Staff Forward Deployed AI Engineer to lead the design, delivery, and adoption of AI/ML systems directly within client environments. This role is ideal for an experie…
Est. 85,000 GBP
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Est. 185,000 USD
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Est. 120,000 GBP
At Anaplan, we are a team of innovators focused on optimizing business decision-making through our leading AI-infused scenario planning and analysis platform so our customers can outpace their competition and the market.…
Est. 215,000 USD
Who We Are Lightning AI is the company behind PyTorch Lightning. Founded in 2019, we build an end-to-end platform for developing, training, and deploying AI systems—designed to take ideas from research to production with…
Location: Lviv, Ukraine Company Overview Robots & Pencils is an applied AI engineering firm building the next frontier of business architecture. We design and ship AI co-workers that integrate into enterprise operati…