AI Product Manager, AI Platform - Quality and Evaluation
RidgelineSan Ramon, California, United States · Posted 23 days agoDescription
Are you passionate about building AI products that customers can trust by making quality measurable rather than subjective? Do you enjoy partnering with engineers and researchers to evaluate large language models, improve AI system performance, and turn data into product decisions? Are you excited by the opportunity to define the future of AI quality, observability, and evaluation platforms that power intelligent applications? If so, we invite you to be a part of our innovative team.
As a Product Manager on Ridgeline's AI Platform team, you'll define and execute the strategy for the systems that ensure our AI capabilities are accurate, reliable, and continuously improving. You'll lead initiatives spanning LLM evaluation frameworks, AI model benchmarking, evaluation harnesses, prompt and model experimentation, AI quality platforms, and observability capabilities that help teams understand, measure, and improve production AI performance. You'll work closely with engineering, AI researchers, design, and product teams to deliver platform capabilities that enable every AI-powered experience across Ridgeline. You'll leverage cutting-edge AI technologies and tools in a fast-moving, creative, progressive work environment while helping establish best practices for developing and operating enterprise AI products.
At Ridgeline, how we work matters as much as what we build. Ridgeliners act like owners, choose growth over comfort, and communicate with transparency. We assume positive intent, bias toward action, and bring solutions—not just problems. We celebrate wins, learn from setbacks, and thrive in a resilient, collaborative, high-performing culture. If this excites you, we’d love to meet you!
You must be work authorized in the United States without the need for employer sponsorship.
The impact you will have
- Define and execute the product strategy and roadmap for Ridgeline's AI Quality Platform
- Build scalable LLM evaluation frameworks and automated evaluation harnesses that enable continuous model validation
- Establish standardized offline and online evaluation methodologies, golden datasets, and quality benchmarks across AI products
- Define the roadmap for AI observability, enabling teams to monitor quality, latency, cost, reliability, and user outcomes in production
- Partner with engineering and AI researchers to build continuous regression testing and experimentation capabilities for generative AI systems
- Drive product decisions using AI quality metrics, customer feedback, and quantitative evaluation data
- Collaborate with engineering teams to deliver platform capabilities that enable trusted, measurable, and continuously improving AI experiences across Ridgeline
- Translate advances in foundation models, evaluation techniques, and AI tooling into platform capabilities that create meaningful customer value
- Prioritize product investments using customer insights, product analytics, and measurable business outcomes
- Communicate product vision, roadmap, and priorities clearly across technical and executive stakeholders
- Foster a collaborative environment rooted in ownership, continuous learning, experimentation, and transparency
What we look for
- 5+ years of product management experience building highly technical platform, infrastructure, or developer-focused products
- Demonstrated experience shipping production AI products powered by large language models
- Experience defining products that incorporate LLM evaluation frameworks, AI quality measurement, and continuous model evaluation
- Deep understanding of offline and online evaluation methodologies, model benchmarking, regression testing, and AI quality metrics
- Experience with LLM observability platforms such as Datadog LLM Observability, Langfuse, LangSmith, Arize AI, Braintrust, Galileo, or similar AI quality and evaluation solutions
- Familiarity with prompt engineering, experimentation frameworks, evaluation datasets, and LLM-as-a-judge techniques for generative AI applications
- Experience partnering effectively with software engineers, ML engineers, AI researchers, and cross-functional stakeholders
- Strong analytical skills with the ability to translate AI performance signals into product decisions
- Excellent written and verbal communication skills
- Willingness to learn about cutting-edge technologies while cultivating expertise in investment management workflows and enterprise AI
- An aptitude for problem solving
- Ability to communicate effectively
- Serious interest in having fun at work
Bonus
- Experience building products around retrieval-augmented generation (RAG), AI agents, MCP, or model orchestration frameworks
- Experience with synthetic data generation and automated evaluation pipelines
- Experience building internal developer platforms or ML infrastructure products
- Experience in enterprise SaaS, developer platforms, or financial technology
About Ridgeline
Ridgeline is headquartered in Lake Tahoe, with offices in New York, Reno, the Bay Area, Dublin Ireland. Ridgeline is recognized by Fast Company as a “Best Workplace for Innovators,” by Frost & Sullivan as a “Technology Innovation Leader,” and by The Software Report as a “Top 100 Software Company.”
Ridgeline is proud to be a community-minded, discrimination-free equal opportunity workplace.
Ridgeline processes the information you submit in connection with your application in accordance with the Ridgeline Candidate Privacy Policy. Please review the Ridgeline Candidate Privacy Policy in full to understand our privacy practices and contact us with any questions.
This posting is for an existing vacancy.
Compensation and Benefits
The typical starting salary range for new hires in this role is listed below. In select locations (including, the San Francisco Bay Area, CA, and the New York City Metro Area), an alternate range may apply as specified below.
The typical starting salary range for this role is: $136,000-$170,000.
The typical starting salary range for this role in the select locations listed above is: $149,500-$187,000.
Final compensation amounts are determined by multiple factors, including candidate experience and expertise, and may vary from the amount listed above.
As an employee at Ridgeline, you’ll have many opportunities for advancement in your career and can make a true impact on the product.
In addition to the base salary, Ridgeline employees can participate in our Company Stock Plan subject to the applicable Stock Option Agreement. We also offer rich benefits that reflect the kind of organization we want to be: one in which our employees feel valued and are inspired to bring their best selves to work. These include unlimited vacation, educational and wellness reimbursements, and $0 cost employee insurance plans. Please check out our Careers page for a more comprehensive overview of our perks and benefits.
Similar jobs
Est. 241,250 USD
Are you passionate about helping engineers and employees unlock entirely new ways of working through generative AI, intelligent agents, and AI-native products? Do you enjoy building and leading exceptional engineering te…
Est. 155,750 USD
Staff AI Data Platform Engineer San Ramon, CA or Reno, NV About Ridgeline Ridgeline is the first front-to-back system of record for investment managers. Founded by visionary entrepreneur Dave Duffield (co-founder of both…
Est. 200,750 USD
AI Enablement & Workforce Transformation Lead About Ridgeline Ridgeline is building the next generation of investment management technology. Founded by Tech visionary Dave Duffield, Ridgeline delivers a modern, cloud…
Est. 205,250 USD
Are you a seasoned engineer with a passion for building extensible, scalable platform frameworks? Do you enjoy designing systems that other engineering teams build on top of and guiding teams through technical complexity…
Est. 230,500 USD
Staff Security Engineer - AI Security & Platforms Location: San Ramon, CA; Reno, NV About Ridgeline Ridgeline is the industry cloud platform for investment management. It was founded by visionary tech entrepreneur Da…
Est. 184,500 USD
Senior Security Engineer - AI Security & Platforms Location: San Ramon, CA; Reno, NV About Ridgeline Ridgeline is the industry cloud platform for investment management. It was founded by visionary tech entrepreneur D…
Est. 230,500 USD
In this foundational leadership role, you will build and lead Ridgeline’s Product & Technology Operational Excellence function, bringing together Product Operations, Engineering and QA Operations, and Project Managem…
Est. 134,000 USD
The Performance & Analytics team works on calculating, reporting, and effectively presenting performance and attribution metrics across investments leveraging modern OLAP and Columnar technologies. This includes anal…
Est. 144,000 USD
Are you excited to build seamless client-facing experiences that strengthen communication and collaboration between investment managers and their clients? Do you enjoy developing scalable fullstack applications that powe…
Est. 117,000 USD
Are you a customer-obsessed professional who enjoys helping investment managers unlock the full value of modern technology? Do you thrive in fast-paced environments where you can manage multiple relationships, drive prod…
Est. 153,000 USD
Can you solve the issue in front of you and then redesign the system so it doesn’t keep happening? If you’re energized by moving between Workday configuration, HR operations, service delivery, automation, data, and cross…
Est. 172,000 USD
The Performance & Analytics team works on calculating, reporting, and effectively presenting performance and attribution metrics across investments leveraging modern OLAP and Columnar technologies. This includes anal…
Est. 172,000 USD
The Performance & Analytics team works on calculating, reporting, and effectively presenting performance and attribution metrics across investments leveraging modern OLAP and Columnar technologies. This includes anal…
Est. 215,250 USD
The Performance & Analytics team works on calculating, reporting, and effectively presenting performance and attribution metrics across investments leveraging modern OLAP and Columnar technologies. This includes anal…
Est. 139,250 USD
As a Full Stack Software Engineer at Ridgeline, you’ll have the unique opportunity to build high-quality, complex, FinTech industry-defining cloud applications for our Client Management solutions. Our team of engineers a…
Est. 172,000 USD
Are you passionate about building highly accurate, calculation-intensive financial systems where correctness, scale, and transparency are critical? Do you enjoy solving complex quantitative challenges involving large dat…
Est. 250,000 USD
Senior Staff Production Engineering Lead San Ramon, CA | Reno, NV | New York, NY Life at Ridgeline At Ridgeline, how we work matters as much as what we build. Ridgeliners act like owners, choose growth over comfort, and…
Est. 159,000 USD
Do you have a passion for finance & investing? Are you excited by the challenge of modeling industry-critical data and making it highly available? Do you enjoy solving complex technical problems and delivering high-q…
Est. 215,750 USD
Are you passionate about designing large-scale reporting systems that power mission-critical analytics and decision-making? Do you enjoy mentoring others and shaping the technical direction of large-scale platforms? Are…
Est. 172,000 USD
Senior Software Engineer, Migrations Engineering Reno, NV / San Ramon, CA (Hybrid) About Ridgeline Ridgeline is redefining investment management software for the modern era. Ridgeline delivers a cloud-native platform pur…
Est. 165,000 USD
Do you thrive in a dynamic environment where you can dive into technology alongside your team? Do you have a passion for finance and investing? Are you ready to lead a world-class team building an industry defining, fast…
Est. 275,000 USD
Are you a dynamic salesperson who thrives on building meaningful relationships in the investment management industry? Do you enjoy opening new doors, crafting strategic pitches, and driving consultative sales from first…
Est. 225,750 USD
Are you passionate about revolutionizing the financial industry? Do you bring an engineering-first mindset while also possessing the quantitative skills to understand financial instruments and their valuations? Are you a…
Est. 216,000 USD
Staff Software Engineer, Site Reliability Engineering Reno, NV; San Ramon, CA; NYC - Hybrid Are you passionate about building resilient, highly available cloud platforms that enable engineering teams to move quickly and…
Est. 175,000 USD
Are you passionate about building large-scale backend systems that power mission-critical reporting and data-intensive workflows? Do you enjoy solving deep technical challenges around distributed systems, performance, sc…
Est. 144,000 USD
Do you have a passion for finance and investing? Are you interested in modeling the industry’s data and making it highly available? Are you a technical leader who enjoys refining both technology performance and team coll…
Est. 161,000 USD
Are you a software integration expert who enjoys improving how enterprise implementation teams deliver technical solutions? Do you thrive on standardizing complex integration workflows and enabling others to implement wi…
Est. 117,500 USD
IT Automation & Support Engineer San Ramon, CA; Reno, NV; NYC About Ridgeline Ridgeline is the first front-to-back system of record for investment managers. Founded by visionary entrepreneur Dave Duffield (co-founder…
Est. 216,000 USD
About Ridgeline Ridgeline is the first front-to-back system of record for investment managers. Founded by visionary entrepreneur Dave Duffield (co-founder of both PeopleSoft and Workday), the company was created to moder…
Est. 213,500 USD
About Ridgeline Ridgeline is the first front-to-back system of record for investment managers. Founded by visionary entrepreneur Dave Duffield (co-founder of both PeopleSoft and Workday), the company was created to moder…