Data Engineer Job Description

Key skills

Responsibilities

• • Design and implement robust ELT/ETL pipelines that ingest data from diverse sources into the warehouse
• • Build and maintain dbt models that provide clean, tested, and documented data layers for analysts and scientists
• • Instrument pipelines with data quality tests, freshness checks, and anomaly alerting
• • Optimize warehouse queries and cluster configurations for cost and performance
• • Evaluate and onboard new data sources, working with source-system owners to understand schemas and change patterns
• • Partner with data scientists to productionize models and build serving infrastructure
• • Document data lineage, model definitions, and pipeline architecture in shared knowledge systems
• • Manage access controls and data governance policies within the warehouse environment

Requirements

• • 3+ years of data engineering experience building production pipelines
• • Strong SQL skills and hands-on dbt experience in a production data warehouse
• • Operational experience with at least one cloud data warehouse (Snowflake, BigQuery, or Redshift)
• • Experience designing and running orchestrated pipeline DAGs in Airflow or equivalent
• • Solid Python skills for pipeline development, scripting, and automation
• • Demonstrated data quality practices: testing, monitoring, and alerting on pipeline health

Nice to have

• • Experience with streaming data infrastructure using Apache Kafka or AWS Kinesis
• • Familiarity with data catalog or metadata management tools (DataHub, Alation, or dbt Docs)
• • Knowledge of data mesh principles and distributed data ownership models
• • Experience with Spark or distributed computing frameworks for large-scale batch processing

What to look for in a great Data Engineer

Data engineers who treat their pipelines like production software — with tests, monitoring, versioning, and documentation — are far more valuable than those who build fast but break silently. In interviews, ask how they handle late-arriving data, schema changes from upstream sources, and partial pipeline failures. Strong candidates have concrete answers and have dealt with these problems in production. Look for modeling sensibility: do they think about how analysts will query their tables, or do they just dump data into a landing zone? Business awareness — understanding which pipelines are most critical and prioritizing accordingly — is a differentiating trait.

Interview questions to ask a Data Engineer

Ask the candidate to design a data pipeline for a specific use case, such as ingesting a high-volume event stream and making it available for daily reporting. Listen for how they handle idempotency, late data, schema evolution, and failure recovery. Ask how they test data quality: what checks do they run, how do they alert, and how do they communicate data issues to downstream consumers? Include a SQL or dbt modeling question that requires thinking through grain, joins, and incremental refresh strategies. Ask about a pipeline that broke in production and how they diagnosed, fixed, and prevented the recurrence.

Where to source Data Engineers

The dbt Community Slack is one of the most active and high-quality talent pools for modern data engineers. Conferences like Data Council, Coalesce, and the various Modern Data Stack meetups surface practitioners who are engaged with current tooling. GitHub repositories for popular open-source data tools (Airflow, dbt, Airbyte) often have contributors who are experienced practitioners. LinkedIn searches combining specific warehouse and orchestration tools narrow to the right profile. Former software engineers who have transitioned into data infrastructure roles can be strong hires, especially if they bring production reliability instincts into the data domain.