Description
Company Overview
At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine learning is the key to unlocking these capabilities for the widespread deployment of robots within society. Our team consists of individuals with varying levels of experience and backgrounds, from new graduates to domain experts. Relevant industry experience is important, but ultimately less so than your demonstrated abilities and attitude. We are looking for passionate individuals who are eager to explore uncharted waters and contribute to our innovative projects.
Position Overview
We are looking for a highly detail-oriented and technically strong Senior Master Annotator to define, maintain, and continuously improve annotation standards for robotic datasets. Various type of data sets can be generated which needs to be annotated and quality checked before they are put in production
This role will act as the gold standard authority for labeling tasks, own Inter-Annotator Agreement (IAA) measurement, train and calibrate annotators, and ensure that annotation quality meets the high standards required for robotics model training.
The ideal candidate understands image, video, 3D and audio annotation, quality metrics, and how annotation accuracy impacts model performance.
Responsibilities
- Define and maintain gold-standard annotations and benchmark (“gold set”) datasets for new and existing tasks
- Establish clear acceptance criteria, tolerance thresholds, and task-specific quality standards (e.g., frame-level tolerances)
- Develop, document, version-control, and continuously refine annotation guidelines and SOPs based on disagreement patterns and quality findings
- Design and implement Inter-Annotator Agreement (IAA) frameworks (e.g., Cohen’s/Fleiss’ kappa, tolerance-based agreement models) and produce regular quality reports
- Analyze disagreement trends, identify root causes, and recommend data-driven process improvements
- Lead onboarding and ongoing training for annotators, including structured calibration sessions
- Build and maintain edge-case libraries and example banks to improve annotation consistency
- Provide targeted feedback and retraining to improve annotator performance and alignment with gold standards
- Conduct precision-based QC audits according to task requirements and validate pilot batches prior to production scaling
- Review escalated edge cases and partner with QA and Operations teams to optimize sampling strategies and quality processes
Preferred Qualifications
- 4–7+ years of experience in data annotation, quality control, or video labeling environments
- Strong expertise in temporal video annotation; experience with robotics, computer vision, or autonomous systems datasets preferred
- Deep understanding of Inter-Annotator Agreement methodologies (e.g., Cohen’s/Fleiss’ kappa), quality metrics, sampling strategies, and gold set creation
- Experience mentoring, training, or guiding annotators to improve accuracy and consistency
- Strong analytical skills with proficiency in Excel or Google Sheets (Python preferred but not required)
- Ability to translate ambiguous SOPs into precise, actionable annotation guidelines
- Exceptional attention to detail and sound judgment in ambiguous visual tasks
- Data-driven, process-oriented mindset with strong ownership and accountability
- Excellent written and verbal communication skills
Similar jobs
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
Est. 92,500 USD
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Rhombus Power, Inc. delivers AI-powered predictive intelligence in real time for defense and national security organizations worldwide. Our mission-built products— Ambient AI and Guardian— are transforming strategic, ope…
Rhombus Power, Inc. delivers AI-powered predictive intelligence in real time for defense and national security organizations worldwide. Our mission-built products— Ambient AI and Guardian— are transforming strategic, ope…
Est. 114,400 USD
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence…
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 275,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 85,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 141,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Queue Manager / Annotation Lead with Trust & Safety Background (AI Data Ops) Start date: Immediate / TBC End Date: TBC Location: Remote Type of contract: Freelance Languages required: English Hourly Rate: Include the…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Queue Manager / Annotation Lead with Trust & Safety Background (AI Data Ops) Start date: Immediate / TBC End Date: TBC Location: Remote Type of contract: Freelance Languages required: English Hourly Rate: Include the…
Est. 27,500 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 80,000 USD
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
Est. 65,000 USD
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
About HumanSignal Real-world data is the competitive edge in AI. HumanSignal is a human data partner for companies building AI models and products. Our customers ship better AI, faster, because we partner with their rese…
Est. 156,000 USD
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…
Est. 200,000 USD
Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to unseen scenarios without failing. We believe massive scale through data-driven machine lea…