Data Gibberish

Data Gibberish

Share this post

Data Gibberish
Data Gibberish
Beyond Senior Data Engineer: Skills & Strategies for the Next Big Step
Copy link
Facebook
Email
Notes
More

Beyond Senior Data Engineer: Skills & Strategies for the Next Big Step

Going beyond senior isn’t just about writing better code. It’s about impact, strategy, and navigating complex systems. Here’s how to take the next step in your data engineering career.

Yordan Ivanov's avatar
Yordan Ivanov
Mar 26, 2025
∙ Paid
12

Share this post

Data Gibberish
Data Gibberish
Beyond Senior Data Engineer: Skills & Strategies for the Next Big Step
Copy link
Facebook
Email
Notes
More
1
Share
A comic book-style illustration of a female data engineer wearing a yellow safety helmet, seen from a top-down perspective. She is climbing a metal ladder and struggling, with one hand slipping slightly and a determined yet slightly panicked expression on her face. Her outfit includes a work jumpsuit with a techy twist – pockets with tools like USB drives, cables, and a tablet clipped to her belt. Around her are scattered digital elements like floating code snippets, data icons, and charts, giving a whimsical feel. The scene uses flat, vibrant colors with bold black outlines in classic comic book style. The setting appears industrial-tech, like a data center or server room, blending technical and construction vibes. The 3:2 composition emphasizes the vertical drop below her, enhancing the drama and humor of the moment.

Greetings, Data Engineer,

You’ve made it to senior.

You deliver solid work. You unblock your team. You know the tech inside out.

But now what?

The leap from senior to staff, principal, or head of data isn’t always clear. It’s not just about scaling your code—it’s about scaling your impact.

Why listening to me?

I’ve been a software engineer for over 7 years. Then restarted my career and got promoted every other year or so. Now, I’m the Head of Data Engineering at my org.

So, here’s how to grow beyond senior by shifting your focus from execution to influence, from tasks to systems, from self to organisation.

Loading...

Now, keep in mind. This article is extremely long and packed with a ton of value.

I encourage you to skim it first. Then, read each section, and spend as much time as you need on the challenge.

Once you succeed with your task, write down your wins. Use them in your yearly performance review.

Show consistency. Keep applying your new skills even after that.

I’m warning you: This guide can lead to a serious career advancement you may not have been prepared for.

Subscribe now and grab my 3-min-a-day email course—everything you need to go end-to-end in data engineering. It’s free!

🔍 Strategic Thinking & Systems Perspective

🧠 Shift from feature delivery to system design and business outcomes

At the senior level, you’re used to being handed features or tickets. You implement. You ship. Done.

But beyond senior, you stop reacting to specs—and start shaping them.

This means asking bigger questions before you even write a line of code:

❓ What’s the business problem?

❓ Why now?

❓ How will this system evolve over time?

You stop thinking about how to build and start asking why it matters.

📍 Example: Instead of creating a new ingestion pipeline because product says so, a principal-level engineer might challenge whether the data already exists elsewhere, propose a shared ingestion pattern, or flag that ingesting this data has privacy implications.

🧰 Tips to apply this:

💡 Next time you get a ticket, write down what the business outcome is supposed to be. Is the implementation actually solving that?

💡 Join backlog grooming or roadmap planning meetings and listen for why projects exist.

💡 When reviewing PRs, ask “what business problem is this solving?” in your comments.

🎯 Your Challenge: Find one feature you’re working on and map it to a business goal. Rewrite your task title or comment in Jira to reflect that goal. Then share your thinking with a teammate or your manager.

🌐 Think in systems: architecture, data flows, organisational processes

Systems thinking means looking beyond the component you’re working on.

Your are not building a table or a DAG. You’re shaping how data moves, how decisions get made, and how other teams depend on your work.

The system isn’t just technical. It includes humans, processes, feedback loops, and incentives.

The Data Engineer's Guide to Mastering Systems Design

The Data Engineer's Guide to Mastering Systems Design

Yordan Ivanov
·
June 26, 2024
Read full story

📍 Example: Let’s say marketing complains about “bad data.” A junior might patch the transformation. A senior would fix the pipeline. But someone beyond senior looks at:

🔎 Where the source data is generated.

🔎 How validation is tested.

🔎 Whether teams are aligned on definitions.

🔎 If documentation is stale or missing.

🧰 Tips to apply this:

💡 Map out upstream and downstream dependencies before you make changes.

💡 Talk to teams using your data—what do they struggle with?

💡 When debugging, ask “what systemic failure led here?” not just “what broke?”

🎯 Your Challenge: Choose one system you own (a dataset, a job, a dashboard) and draw out all its dependencies—upstream, downstream, people, and processes. Share it with a peer and ask, “what’s fragile here?”

Don’t miss any of the guides and mini-courses for Pro Members

Upgrade Your Experience

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Yordan Ivanov
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More