Data Engineer

What it is

SQL analysis turns raw relational and event data into cohorts, funnels, user journeys, reporting tables, and defensible business metrics.

Why it matters

SQL is still the shared language across BI, product analytics, data engineering, governance, health analytics, and OR work. The practical bar is not syntax; it is avoiding double-counts, fanout joins, stale filters, and silent null bugs.

Proof to build

Solve a cohort or funnel analysis from raw tables, include validation queries, and explain how you checked row grain, duplicates, nulls, and date boundaries.

What it is

Python and pandas are the repeatable analysis layer: load data, clean it, reshape it, validate assumptions, automate checks, and package work so another analyst or engineer can rerun it.

Why it matters

SQL answers many warehouse questions, but real career work often needs Python for messy files, healthcare extracts, simulation inputs, forecasting prep, exploratory analysis, and reproducible notebooks.

Proof to build

Build a notebook or script that loads a raw dataset, profiles quality problems, cleans it, validates row counts and nulls, and exports a decision-ready table with clear assumptions.

What it is

Files is the practical work inside sql and python foundations: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Files matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Build the query and scripting base for repeatable data movement. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

A sql and python foundations portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one sql and python foundations artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for sql and python foundations is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns sql and python foundations into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

Warehouse architecture covers storage, compute, marts, permissions, cost controls, performance tuning, deployment layers, and how data products are exposed.

Why it matters

Data architects and analytics engineers need enough platform judgment to avoid expensive, slow, or brittle designs. Cost, query latency, lineage, and ownership are architecture decisions, not afterthoughts.

Proof to build

Write an architecture decision record comparing two warehouse/lakehouse designs for the same workload, with cost, freshness, governance, and performance tradeoffs.

What it is

ELT is the practical work inside warehousing and elt: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

ELT matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Model source data into clean analytical tables with clear grain and history. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Dimensional models is the practical work inside warehousing and elt: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Dimensional models matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Model source data into clean analytical tables with clear grain and history. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

A warehousing and elt portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one warehousing and elt artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for warehousing and elt is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns warehousing and elt into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

dbt is the practical work inside dbt and quality: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

dbt matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Version, test, document, and deploy transformations as production code. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Tests is the practical work inside dbt and quality: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Tests matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Version, test, document, and deploy transformations as production code. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

BI tooling turns governed data into dashboards, scorecards, drilldowns, alerts, permissions, refreshes, and recurring decision workflows.

Why it matters

Real BI value is not a pretty chart. It is a trusted operating surface that busy teams check without asking an analyst to explain every filter and caveat.

Proof to build

Build a dashboard case study with a metric dictionary, screenshot walkthrough, refresh plan, permission model, known caveats, and a QA checklist for the top numbers.

What it is

A dbt and quality portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one dbt and quality artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for dbt and quality is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns dbt and quality into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

Airflow is the practical work inside orchestration: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Airflow matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Schedule, observe, retry, and alert on pipelines that fail in real ways. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Dagster is the practical work inside orchestration: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Dagster matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Schedule, observe, retry, and alert on pipelines that fail in real ways. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Retries is the practical work inside orchestration: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Retries matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Schedule, observe, retry, and alert on pipelines that fail in real ways. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

A orchestration portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one orchestration artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for orchestration is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns orchestration into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

Spark is the practical work inside spark and lakehouse: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Spark matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Handle large datasets with distributed processing and table formats. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Delta is the practical work inside spark and lakehouse: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Delta matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Handle large datasets with distributed processing and table formats. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

AI-ready data architecture extends the warehouse into feature pipelines, vector indexes, lakehouse tables, privacy controls, and data products that models can safely consume.

Why it matters

AI teams are constrained by data access, data quality, permissioning, and retrieval quality. The best architecture work makes model-facing data reliable before an ML or LLM team depends on it.

Proof to build

Design an AI data product with source tables, feature or embedding pipeline, access rules, freshness checks, and an evaluation query for retrieval or model input quality.

What it is

A spark and lakehouse portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one spark and lakehouse artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for spark and lakehouse is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns spark and lakehouse into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

Kafka is the practical work inside streaming: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Kafka matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Process events with durable contracts, ordering expectations, and late data handling. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Flink is the practical work inside streaming: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Flink matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Process events with durable contracts, ordering expectations, and late data handling. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

SQL analysis turns raw relational and event data into cohorts, funnels, user journeys, reporting tables, and defensible business metrics.

Why it matters

SQL is still the shared language across BI, product analytics, data engineering, governance, health analytics, and OR work. The practical bar is not syntax; it is avoiding double-counts, fanout joins, stale filters, and silent null bugs.

Proof to build

Solve a cohort or funnel analysis from raw tables, include validation queries, and explain how you checked row grain, duplicates, nulls, and date boundaries.

What it is

A streaming portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one streaming artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for streaming is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns streaming into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

AI-ready data architecture extends the warehouse into feature pipelines, vector indexes, lakehouse tables, privacy controls, and data products that models can safely consume.

Why it matters

AI teams are constrained by data access, data quality, permissioning, and retrieval quality. The best architecture work makes model-facing data reliable before an ML or LLM team depends on it.

Proof to build

Design an AI data product with source tables, feature or embedding pipeline, access rules, freshness checks, and an evaluation query for retrieval or model input quality.

What it is

Terraform is the practical work inside cloud and dataops: the concepts, tools, checks, and deliverables a Data Engineer uses to turn this stage into real output.

Why it matters

Terraform matters for Data Engineer because this stage is where a learner turns the role from a title into evidence. Deploy infrastructure, permissions, cost controls, and production operations. A strong learner should be able to explain the tradeoffs, build a small artifact, and connect the result to a business or product decision.

Proof to build

Build a small artifact for this topic: a query, notebook, dashboard, architecture note, evaluation table, or decision memo that shows the input, method, validation, caveat, and recommendation.

What it is

Production bridge work turns a validated model or analysis into something a real system can use: deployment boundary, monitoring, ownership, rollback, and product metric follow-through.

Why it matters

Applied science and ML work only matters when the result survives handoff. Monitoring and deployment planning catch regressions, drift, latency issues, and mismatches between offline metrics and user impact.

Proof to build

Create a production-readiness note with serving path, model or metric owner, monitoring signals, alert thresholds, rollback plan, and the product metric that should move after launch.

What it is

A cloud and dataops portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one cloud and dataops artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for cloud and dataops is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns cloud and dataops into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.

What it is

LLM application engineering connects prompts, structured outputs, retrieval, tool calls, and evaluation into a product workflow that users can trust repeatedly.

Why it matters

AI roles in 2026 are less about toy prompts and more about building systems that survive ambiguous inputs, private data, latency budgets, source grounding, and regressions after a model update.

Proof to build

Ship a small RAG or structured-output app with traces, citations, an evaluation set, fallback behavior, and a short writeup explaining the failure modes you found.

What it is

AI-ready data architecture extends the warehouse into feature pipelines, vector indexes, lakehouse tables, privacy controls, and data products that models can safely consume.

Why it matters

AI teams are constrained by data access, data quality, permissioning, and retrieval quality. The best architecture work makes model-facing data reliable before an ML or LLM team depends on it.

Proof to build

Design an AI data product with source tables, feature or embedding pipeline, access rules, freshness checks, and an evaluation query for retrieval or model input quality.

What it is

Metadata and lineage explain what data exists, who owns it, how it changes, where it flows, and which downstream assets depend on it.

Why it matters

Governance without operational metadata becomes policy theatre. Real users need searchable definitions, lineage for impact analysis, and owners who can fix broken data.

Proof to build

Document a mini data catalog: glossary, table owners, lineage diagram, access level, freshness SLA, and a downstream impact analysis for one schema change.

What it is

A ai data infrastructure portfolio artifact is a public proof piece for this stage: a small but complete deliverable that shows how a Data Engineer frames the problem, chooses tools, validates the result, and explains the tradeoffs.

Why it matters

This is the work product that makes the stage credible. For Data Engineer, the artifact should show inputs, assumptions, methods, validation, tradeoffs, and a decision-ready output rather than a tutorial clone.

Proof to build

Publish one ai data infrastructure artifact with README, inputs, assumptions, method, validation checks, screenshots or outputs, caveats, and a short summary of what the artifact proves for Data Engineer readiness.

What it is

The interview signal for ai data infrastructure is your ability to explain the work under pressure: assumptions, tradeoffs, failure modes, implementation choices, and how the output would help a real team decide what to do next.

Why it matters

Hiring teams need to see judgment, not just vocabulary. This topic turns ai data infrastructure into interview-ready stories, diagrams, live explanations, and examples that map to real Data Engineer work.

Proof to build

Prepare a two-minute explanation, one diagram or query/notebook walkthrough, and three follow-up answers for this stage: why this approach, what could fail, and how you would improve it in production.