Job Overview / Summary
The Data Engineer will support the implementation, configuration, and maintenance of data pipelines and transformation workflows across IRC's data environment. This role plays an active part in building and operating ELT processes, dbt models, and cloud-based data platforms including Azure Databricks and the IRC Lakehouse.
The successful candidate has foundational to growing expertise in dbt, SQL, and Databricks, and is eager to take on increasing ownership of data engineering deliverables. Working closely with senior engineers and the Data Architect, this role contributes to data modeling, pipeline development, and platform reliability in a collaborative and fast-paced environment.
Major Responsibilities
Pipeline Engineering & Orchestration
Build and maintain ELT data pipelines using Databricks Workflows and Azure Data Factory for batch and scheduled processing from internal and external sources.
Support the ingestion of data from key systems (e.g., D365 CRM, ServiceNow) into Lakehouse.
Monitor pipeline execution, identify failures, and troubleshooting issues in collaboration with senior engineers.
Contribute to pipeline documentation and help maintain runbooks and process standards.
dbt Development
Develop and maintain dbt models across staging, intermediate, and mart layers under the guidance of senior team members.
Write dbt tests and contribute to source freshness checks to support data quality.
Learn and apply dbt best practices including modular design, ref dependencies, and incremental model patterns.
Work with analysts and business teams to translate data requirements into dbt models.
SQL & Data Transformation
Write intermediate to advanced SQL for data extraction, transformation, and validation tasks.
Apply SQL techniques including joins, CTEs, window functions, and aggregations to support reporting and analytics needs.
Assist in query optimization and performance troubleshooting within
Databricks SQL environments.
Support data model maintenance and help accommodate new source fields or schema changes.
Databricks & Cloud Platform
Develop and maintain Databricks notebooks and jobs for data transformation workloads.
Gain hands-on experience with Delta Lake concepts and PySpark for data processing.
Follow Lakehouse design patterns (Bronze/Silver/Gold) as defined by the Data Architect.
Support cloud resource management including basic cluster configuration and job scheduling.
Collaboration & Learning
Actively collaborate with the Data Team on pipeline design, troubleshooting, and delivery.
Participate in code reviews and incorporate feedback to improve code quality.
Support documentation of processes, standards, and data flows
Engage with Finance, FP&A, and other business teams to understand data needs and assist in solution delivery.
Key Working Relationships
Data Team (Data Architect, Lead Data Engineer, Data Analysts)
Business / Departmental Priority Setters (Finance, FP&A, Operations)
Enterprise Systems Owners (D365 CRM, Integra)
Travel Requirements
Special projects may require travel to enable on-the-ground learning to support data strategy deliverables.
Minimum Requirements
Experience
3 - 6 years of hands-on experience in data engineering, analytics engineering, or a related technical role.
Demonstrated experience building or maintaining data pipelines in a professional setting.
Exposure to cloud-based data platforms, preferably Azure (Databricks, Data Factory, or Synapse).
Technical Skills — Required
dbt:
Working knowledge of dbt model development including staging and mart layers.
Familiarity with dbt tests, documentation, and source configurations.
Eagerness to deepen dbt skills including incremental models and CI/CD integration.
Databricks:
Hands-on experience with Databricks notebooks and basic job/workflow setup.
Familiarity with Delta Lake concepts and Databricks SQL.
Exposure to PySpark for data transformation tasks.
SQL:
Solid SQL skills: joins, CTEs, window functions, aggregations, and basic performance awareness.
Experience writing SQL for data transformation and validation in a cloud data warehouse.
Pipeline Engineering:
Experience building or supporting ELT pipelines with monitoring and basic data validation.
Familiarity with pipeline orchestration tools such as Azure Data Factory or Databricks Workflows.
Python:
Basic to intermediate Python skills for data processing, scripting, and automation.
Familiarity with PySpark is a plus.
Data Modeling:
Understanding of star/snowflake schemas and fact & dimension table concepts.
Exposure to Lakehouse or medallion architecture (Bronze/Silver/Gold) is a plus.
Soft Skills
Curious and eager to learn with a proactive approach to problem-solving.
Good communication skills — able to collaborate across technical and non-technical teams.
Attention to detail and a strong sense of data quality.
Comfortable working in a collaborative, fast-paced, and remote team environment.
Preferred Additional Requirements
Experience with Databricks or Azure Synapse Analytics.
Familiarity with D365 CRM or Similar data structures.
Exposure to Git-based workflows and CI/CD practices for data pipeline deployments.
Experience in a humanitarian, nonprofit, or international development context.