Description
Who we are
DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, and certificate lifecycle management, to secure infrastructure, software, devices, messages, AI content and agents. Learn why more than 100,000 organizations, including 90% of the Fortune 500, choose DigiCert to stop today’s threats and prepare for a quantum-safe future at www.digicert.com
Job summary
We are looking for a Senior Data Platform Engineer to design, build, and operate the foundational data and ML infrastructure that powers analytics, reporting, and machine learning across DigiCert. This role sits at the intersection of data engineering and ML platform work — you will own the systems, pipelines, and tooling that data scientists, analysts, and engineers rely on every day. You bring deep expertise in Databricks, a strong engineering mindset, and hands-on experience building and maintaining ML pipelines in production. You will partner closely with data science, analytics, product, and business teams to deliver a platform that is reliable, governed, and built to scale.
What you will do
- Design, build, and maintain scalable data and ML pipelines using Python and SQL, processing large-scale datasets across batch and streaming workloads
- Own and evolve core platform infrastructure on Databricks — including Delta Lake table architecture, Unity Catalog governance, Databricks Workflows orchestration, and compute optimization
- Build and maintain end-to-end ML pipelines: feature engineering, model training pipelines, experiment tracking (MLflow), and model deployment/serving infrastructure
- Collaborate with data scientists to operationalize models — bridging the gap between experimentation and production-grade ML systems
- Define and enforce data platform standards: ingestion patterns, data modeling conventions, medallion architecture (Bronze/Silver/Gold), and pipeline reliability practices
- Implement data quality, observability, and monitoring frameworks to ensure platform health and data trustworthiness
- Optimize pipelines for performance, cost, and reliability at scale using Spark and PySpark
- Evaluate, integrate, and govern new platform tooling and data sources within the Databricks ecosystem
- Contribute to architectural decisions and help drive the long-term data platform roadmap
- Participate in code reviews, technical design discussions, and engineering standards
- Mentor junior engineers and elevate overall platform and data engineering practices
- Document platform architecture, pipeline design, and operational runbooks
What you will have
- 6+ years of experience in data engineering, data platform, or ML engineering roles
- Strong proficiency in Python and SQL, with a track record of building production-grade data pipelines using both
- Hands-on Databricks expertise: Delta Lake, Unity Catalog, Databricks Workflows, PySpark, and the Databricks ecosystem broadly
- Experience building and maintaining ML pipelines in production — feature engineering, training pipelines, experiment tracking, and model deployment
- Familiarity with MLflow or comparable experiment tracking and model registry tools
- Experience working on cloud data platforms (AWS, Azure, or GCP)
- Strong understanding of data modeling, dimensional design, and analytics-friendly data architecture
- Experience with batch and incremental/CDC pipeline patterns
- Proficiency with Git, version control, and CI/CD practices for data and ML workflows
- Strong engineering judgment — you think about reliability, maintainability, and cost, not just correctness
- Clear communication and comfort working with both technical and non-technical stakeholders
Nice to have
- Experience with streaming or near real-time pipelines (Kafka, Kinesis, Spark Structured Streaming)
- Familiarity with feature store platforms (Databricks Feature Store, Feast, or Tecton)
- Experience with LLM pipelines, RAG architectures, or AI/BI tooling (Genie, AI Functions)
- Knowledge of data quality and observability tooling (Great Expectations, Monte Carlo, etc.)
- Exposure to dbt or similar SQL-based transformation frameworks
- Infrastructure-as-code experience (Terraform, Databricks Asset Bundles)
- Experience working in Agile or Scrum environments
- Prior experience mentoring engineers or shaping platform standards
Benefits
- Generous time off policies
- Top shelf benefits
- Education, wellness and lifestyle support
To protect candidate information and maintain a secure hiring process, all applications must be submitted through our careers portal. Resumes or CVs sent directly via email will not be reviewed or considered.
#LI-SD1
Similar jobs
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Est. 120,000 USD
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Est. 125,000 USD
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
Est. 120,000 USD
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Est. 120,000 GBP
ReqID: FEQ327R328 Recruiter: Kanwal Matharu Location: London, United Kingdom - Hybrid Skills: Data Science, Machine Learning, AI, LLM, GenAI As a Senior Specialist Solutions Engineer (SSE), ML Engineering, you will be th…
P-1377 Mission As a Senior ML and AI Technical Solutions Engineer, you play a critical role by helping customers debug and maintain stable GenAI and ML Workloads with AI agent systems using the Databricks Platform. You w…
Est. 175,000 USD
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
Est. 120,000 GBP
ReqID: FEQ327R501 Location: London Skills: Data Science, Machine Learning, AI, LLM, GenAI Mission As a Senior Specialist Solutions Architect (ML & AI), you will serve as the trusted technical ML and AI expert for Dat…
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Est. 140,000 USD
Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…
Est. 260,200 USD
FEQ227R233While candidates in the listed location(s) are encouraged for this role, candidates in other locations will be considered.At Databricks, our core principles are at the heart of everything we do; creating a cult…
Est. 140,000 USD
CSQ227R165 At Databricks, we are passionate about empowering data teams to tackle the world’s most challenging problems — from bringing the next mode of transportation to reality to accelerating the development of medica…
Est. 332,500 USD
P-1125 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data a…
Est. 315,875 USD
P-1125 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best data a…
Est. 213,750 USD
FEQ427R382 As an AI/ML Specialist Solutions Architect (SSA), you will lead the advanced AI/ML technical strategy for your customers — owning complex architecture discussions, driving platform adoption, and serving as a t…
Est. 90,000 EUR
(FEQ427R351) Senior Specialist Solutions Engineer - Platform Security and Cloud Infrastructure As a Senior Specialist Solutions Engineer, you will guide customers in building big data solutions on Databricks that span a…
Est. 261,000 USD
RDQ427R368 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthrough…
Est. 194,950 USD
P-1137 Summary At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's bes…
Est. 120,000 EUR
(REQ ID: CSQ427R209)Databricks is redefining what organisations can achieve with data and AI. As the unified platform powering the world's most ambitious data-driven enterprises, we sit at the intersection of cloud migra…
Est. 264,200 USD
GAQ327R224 About Databricks At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from making the next mode of transportation a reality to accelerating the development of medi…
Est. 260,200 USD
As a Sr. Solutions Architect on the team supporting the strategic Digital Natives Business (DNB) customers, you will shape the future of the Data & AI landscape by working with the most sophisticated data engineering…