Description
Analytical Wizards is part of the Definitive Healthcare family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide customers from what’s now to what’s next by unlocking the value of their data and applications to solve their challenges, achieving outcomes that benefit both business and society. Our people are our biggest asset, they drive our innovation advantage and we strive to offer a flexible and collaborative workplace where they can thrive. We offer industry-leading benefits packages to promote a creative and inclusive culture. If driving real change gives you a sense of pride and you are passionate about powering social good, we’d love to hear from you.
Job Description – Data Engineer
About the Role
We are looking for a candidate who is passionate about building scalable data pipelines, optimizing data workflows, and ensuring high data quality across systems. Candidates should demonstrate strong technical foundations, the ability to work independently, and a willingness to collaborate in a dynamic environment.
Core Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines to support business and analytical needs.
- Work extensively with Databricks, Python, and PySpark to process large datasets.
- Build and manage DAGs using Apache Airflow for workflow orchestration.
- Collaborate with cross-functional teams to understand data requirements and translate them into efficient engineering solutions.
- Develop and optimize complex SQL queries and participate in data modeling activities for relational and cloud data warehouses.
- Work with Amazon S3 for data storage, ingestion, partitioning, and integration within broader data lake and pipeline ecosystems.
- Ensure high standards of data quality, reliability, and performance across all data processes.
- Contribute to documentation, best practices, and continuous improvement initiatives.
Core Technical Requirements
- Python Programming
- Strong experience writing clean, efficient Python code for data manipulation, automation, scripting, and ETL workflows.
- Familiarity with widely used data libraries (e.g., pandas, numpy).
- Databricks
- Hands-on experience with Databricks for distributed data processing.
- Proficiency in PySpark, Delta Lake, notebooks, and building scalable pipelines.
- Orchestration Tools
- Apache Airflow (Required): Ability to design, implement, and maintain complex DAGs for scheduling and orchestrating workflows.
- Argo Workflows (Preferred): Experience with Kubernetes-native orchestration platforms is an added advantage.
- SQL Skills
- Advanced SQL expertise including writing complex queries, query optimization, and working with relational/cloud data warehouses.
- Experience in data modeling and performance tuning.
- Cloud Storage (Amazon S3)
- Practical knowledge of S3 for ingestion, storage, data partitioning, access control, and integration as part of data lake architectures.
Experience Level
2–5 years in Data Engineering or related roles..
Preferred Personal Attributes
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
- Ability to work in a fast-paced, evolving environment.
Similar jobs
Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Senior Data Engineer. Embark on a journey where your deep-rooted expertise in computer science fundamentals, alongside an intricate…
Project Description We are looking for an experienced Data Engineer with deep expertise in Databricks to join our advanced analytics and data engineering team. The ideal candidate will play a key role in designing, build…
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
Analytical Wizards is part of the Definitive Healthcare family. We balance innovation with an open, friendly culture and the backing of a long-established parent company, known for its ethical reputation. We guide custom…
Job Title: AWS data Engineer About Us “Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Compan…
About us: Working at Tech Holding isn't just a job, it's an opportunity to be a part of something bigger. We are a full-service consulting firm that was founded on the premise of delivering predictable outcomes and high-…
About Workato Workato delivers enterprise infrastructure for the agentic era, redefining iPaaS and helping enterprises unify data, applications, processes, and AI into a single, governed platform. A leader in Enterprise…
Staff Software Engineer (Data Platform)Join our dynamic team at the forefront of cutting-edge technology as we seek a seasoned Staff Data Engineer. Embark on a journey where your deep-rooted expertise in distributed syst…
Sigmoid empowers enterprises to make smarter, data-driven decisions by blending advanced data engineering with AI consulting. We collaborate with some of the world’s leading data-rich organizations across sectors such as…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
About the Role We are looking for a Senior Data Engineer with strong experience in modern data platforms and cloud-based architectures. The ideal candidate has deep hands-on experience with Snowflake or Databricks, has b…
P-995 About the Team At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical br…
Opportunity Overview: We are looking for a Data Engineer to build and optimize scalable data pipelines and analytical datasets on our AWS-based lakehouse platform. You will work with Apache Iceberg, AWS Glue, Athena, dbt…
CSQ227R28 As a Manager (Forward Deployed Engineering), you will hire & manage Big Data Engineers / Architects and participate in technical and design discussions with regions and customers. You will also assist our c…
GAQ327R175 About the Role This role is a part of the Finance Organisation. As Finance Data Lead on the Finance Data and AI team, you will be the technical authority behind the data pipelines, AI systems, and internal app…
P-1403 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems - from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. W…
P-375 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems - from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We…
Est. 120,000 PLN
Company Background Our client is a global leader in access solutions and smart home technologies. With over a decade of innovation in the IoT space, the company is a pioneer in connected home solutions, including garage…
CSQ427R228 As a Senior Forward Deployed Engineer (FDE) you will work with customers to build and productionize solutions to their data & AI challenges using the Databricks platform. You will own the architecture, lea…
Est. 90,000 GBP
Senior Data Engineer Who we are Artefact is a new generation of data service provider, specialising in data consulting and data-driven digital marketing, dedicated to transforming data into business impact across the ent…
(P-1384) About the Team At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical…
P-1403 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems - from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. W…
Est. 84,000 PLN
Xebia is a global AI-first, digital transformation, and engineering partner. With over 25 years of experience and a team of 5,000 professionals across 16 countries, we help organizations design and build scalable product…
P-1348 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems, from security threat detection to cancer drug development. We do this by building and running the world's best dat…
Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…
We are looking for a Senior Data Engineer to build and evolve the data platform powering our global workforce management ecosystem. You will design, implement, and maintain scalable data pipelines that consolidate data f…
FEQ427R107 As a Solutions Architect, you will shape the future of the Data & AI landscape by working with the most sophisticated data engineering and data science teams in the world. You will be a technical advisor i…
P-995 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. We…
Who We Are Konrad is a next generation digital consultancy. We are dedicated to solving complex business problems for our global clients with creative and forward-thinking solutions. Our employees enjoy a culture built o…
P-1563 At Databricks, we are passionate about enabling data teams to solve the world's toughest problems — from making the next mode of transportation a reality to accelerating the development of medical breakthroughs. W…