inhousefyi
← Back to listings

Senior Data Engineer Databricks, PySpark

Aspenview Technology PartnersRemote · Posted 8 days ago
Full-timeRemote
Apply now

Description

Build the Future with AspenView Technology Partners

At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help North American clients innovate faster and more efficiently.

As we continue to grow, we’re looking for exceptional people to join our team and help drive impactful change across industries.

Why Join AspenView?

At AspenView, we’re more than a nearshore IT partner—we’re a people-first, purpose-driven company that believes great culture drives great outcomes. We’re passionate about connecting talent and technology to deliver measurable value for clients—and meaningful career paths for our people.

Here’s what you can expect:

  • Competitive base
  • Flexible work model: hybrid, remote, or in-office
  • Real growth opportunities and leadership visibility
  • Inclusive, respectful culture that blends U.S. innovation with Colombian heart
  • A company that listens, invests in you, and celebrates wins together

We are looking for a skilled Senior Data Engineer with hands-on experience in Databricks, Unity Catalog, and Azure Data Factory (ADF) to support a large-scale enterprise data modernization initiative. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines, enabling secure data governance, and supporting cloud-based analytics solutions in Azure.

What you will do:

This role will collaborate closely with architects, business stakeholders, and cross-functional engineering teams to build reliable, high performance data platforms and accelerate data-driven decision-making.

AI Application Development & Orchestration

  • Design, develop, and optimize ETL/ELT pipelines using Azure Data Factory and Databricks.
  • Build and maintain scalable data ingestion frameworks for structured and unstructured data sources.
  • Develop data transformation processes using PySpark, Spark SQL, and Databricks notebooks.
  • Implement and manage Unity Catalog for enterprise data governance, security, and access controls.
  • Create and optimize Delta Lake architectures for analytics and reporting workloads.
  • Monitor pipeline performance and troubleshoot production issues.
  • Collaborate with data architects to implement enterprise data model and best practices.
  • Ensure data quality, integrity, security, and compliance standards are met.
  • Support CI/CD deployment processes and DevOps practices for data engineering solutions.
  • Work closely with business and analytics teams to understand requirements and deliver scalable solutions.

What you bring: Experience:

4-7+ years of experience in Data Engineering or Big Data development.

Strong hands-on experience with Azure Databricks.

Proficiency in PySpark, Spark SQL, and distributed data processing.

Experience implementing and managing Unity Catalog.

Strong experience with Azure Data Factory (ADF).

Experience working with Delta Lake and modern Lakehouse architectures.

Solid understanding of data modeling, ETL/ELT, and data warehousing concepts.

Experience with Azure cloud services and ecosystem.

Proficiency with Git, version control, and CI/CD processes.

Strong analytical and problem-solving skills.

Excellent communication and stakeholder management abilities.

Preferred Qualifications:

Experience with Azure Synapse Analytics.

Familiarity with Collibra or other data governance platforms.

Experience with Infrastructure as Code (Terraform, ARM Templates).

Databricks Certification (Data Engineer Associate/Professional).

Microsoft Azure Data Engineer Associate Certification

Nice to have:

Experience supporting enterprise-scale data migration projects.

Knowledge of healthcare, retail, or consumer products data environments.

Exposure to real-time data processing and streaming technologies.

Visa Sponsorship

AspenView does not sponsor employment visas for this role. Applicants must be permanently authorized to work in their country of residence and must not require visa sponsorship now or in the future.

Equal Opportunity Employer

AspenView is proud to be an equal opportunity employer. We believe in creating an environment where all employees feel welcome, valued, and empowered to succeed. We celebrate diversity and strive to build a culture of inclusion where all individuals, regardless of their race, color, gender, gender identity or expression, sexual orientation, disability, age, or any other characteristic, can thrive. We encourage applicants from all walks of life to join our team and make a lasting impact.

Similar jobs

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Est. 120,000 USD

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote
CaylentRemote

Est. 120,000 USD

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
DigiCertRemote

Who we are DigiCert is a global leader in intelligent trust. We protect the digital world by ensuring the security, privacy, and authenticity of every interaction. Our AI-powered DigiCert ONE platform unifies PKI, DNS, a…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote

Build the Future with AspenView Technology Partners At AspenView, we are passionate about transforming the way organizations approach technology. We specialize in creating high-performing, nearshore IT teams to help Nort…

Full-timeRemote
EncoraRemote

Job Title: Senior Data EngineerKey Skills: SQL, Databricks, PySpark, Data Engineering, PostgreSQL, AWSExperience: 3+ YOE.Location: Costa Rica, Peru, Colombia, and Bolivia.Mode: Remote. We at Coforge are hiring Senior Dat…

Full-timeRemote
DatabricksAmsterdam, Netherlands

Est. 90,000 EUR

P-1448 About the Role Databricks is seeking an experienced Sr. Designated Support Engineer/Premier Support Engineer with a strong technical foundation in Apache Spark and a passion for customer engagement. This self-moti…

Full-time
CaylentRemote

Est. 120,000 USD

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
DatabricksRemote

Est. 213,750 USD

FEQ427R143 As an Infrastructure & Platform Specialist Solutions Architect (SSA), you will serve as a trusted technical advisor, guiding enterprise customers through the architecture, administration, and security of t…

Full-timeRemote
CaylentRemote

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
EncoraRemote

Job Title: Senior Data Engineer - Azure DatabricksKey Skills: Azure Databricks, DBT, Snowflake, Python, SQL, TerraformExperience: 5+ YOE.Location: Costa Rica, Peru, Colombia, and Bolivia.Mode: Remote. We at Coforge are h…

Full-timeRemote
DatabricksRemote

Est. 213,750 USD

FEQ327R420 As a Data Engineering and Warehousing Specialist Solutions Architect (SSA), you will lead the advanced technical strategy for your customers — owning complex architecture discussions, driving platform adoption…

Full-timeRemote
CaylentRemote

Est. 140,000 USD

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
DatabricksStockholm, Sweden

Est. 840,000 SEK

REQ# FEQ427R34 At Databricks, our core principles are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified platform that makes data science an…

Full-time
CaylentRemote

Est. 175,000 USD

Caylent is an AI-first cloud services company that helps organizations turn ambitious ideas into meaningful business impact. As an AWS Premier Tier Services Partner and a charter member of Anthropic’s Claude Partner Netw…

Full-timeRemote
DatabricksDenmark

Est. 624,000 DKK

REQ# FEQ127R23 At Databricks, our core principles are at the heart of everything we do; creating a culture of proactiveness and a customer-centric mindset guides us to create a unified platform that makes data science an…

Full-time
DatabricksMexico City, Mexico

As a Sr. Solutions Engineer at Databricks within the Field Engineering org you will partner with our customers to design scalable data architectures using Databricks technology and services. You have technical depth and…

Full-time