Most leaders grow their teams organically. They backfill when someone leaves, hire when a new project lands, and add an analyst when the CFO wants a dashboard. Two years later they look around and see a mess. One sub-team has six people tripping over each other, another has one person drowning, everyone is busy, yet nothing ships fast.

And you know what? You already know how to fix this.

Every engineer who made it past their first year knows SOLID principles. You apply them to your code, but nobody taught you how to use them when building team. And that’s why most engineering teams grow like spaghetti.

I have been applying SOLID to team building for years. Three years ago I wrote about using it to structure a data migration. That article still holds up. Read it here if you want the technical version.

But today, I am walking you through the people version.

I have seen this work for teams up to 15 people, which is great, because, nowadays most data teams that actually ship are even smaller.

Here is how it works and how you do it.

But before that, I’d greatly appreciate if you boost my metrics a bit by clicking the link below.

This issue is brought to you by Cube.

Agents do the analysis. Your team does the thinking.

A question about your own business data still means a ticket, a queue, and a week of waiting. Cube hands that workflow to AI agents: they model the data, explore it, and build the reports — your team guides and approves — so week-long analysis comes back in minutes. And because every agent works from one governed set of metric definitions, the answers come back right. 400+ companies run on Cube, including Brex, Wix, Patagonia, and Webflow.

Get started for free

Single Responsibility Means Splitting The Work Before You Hire

I do not love repo sprawl, but we run three separate dbt projects because the use cases, skill sets, and velocities are too different to force into one place.

The first project is for data engineers. They clean data, fix types, and enforce structure. It is the plumbing. The second project is for analysts. They build business logic, define KPIs, and model the objects that feed our BI tools. The third project is lightweight vendor modeling. Our vendors see their work, and we see what we owe them.

I split them intentionally:

• An analytics engineer should focus on their models without worrying about ingestion pipelines.

• A product person should think like a product person, and when they have to reason about data types and ingestion logic, something is wrong.

Each sub-team moves at its own speed and owns its own scope, and no one steps on another sub-team’s work.

I do not ask analysts to optimize costs or write complex macros, and I also do not want data engineers to create dashboards. Everyone knows what they own.

This is where most teams go wrong. They hire before they split the work, so they bring in five analysts and two data engineers and throw them at a monorepo, then they wonder why the analysts break pipelines and the engineers resent the noise.

How You Do It

Write down the ownership rules and post them where people see them:

• When someone asks who owns the vendor schema, the answer should be obvious.

• When someone asks who approves a change to the BI model, the answer should be obvious.

• When the answer is not obvious, people slow down, they ask around, they wait.

Clear boundaries fix that.

Open For Extension Means Your Team Grows Without Reorgs

In code, the Open/Closed Principle means you extend behavior without modifying existing code. On my team, it means you extend people without restructuring the org chart. Career growth comes from doing the work instead of watching someone else do it. When an engineer wants to try analytics engineering, give them a project.

That’s how the org chart stays the same, and they grow anyway.

I’ve had one to ones with one of our embedded analysts for years. She was more of a stakeholder than a team members. Then, she started showing interest in writing BI code, and then dbt. Our senior analytics engineer, and I helped her learn all the necessary skills. A few months later, she build one of our 3 dbt projects from scratch. After a couple more months, she fully transitioned to becoming an analyst and joined the team.

Now, this only works with strong leaders. You need to see potential in your members and give them challenging tasks that push them in the direction they want to grow. You need to foresee where your team needs to expand and support your org to reach its goals. Weak leaders give projects to people who already have experience with the area. They wait for a re-org to justify a title change, and they let people stagnate because it is easier than finding them a stretch project.

How You Do It

Map the skills on your team. Think about who writes SQL, who knows Python, who do stakeholders actually listen to, and who fixes Airflow when it breaks. Then map the gaps. What work sits unclaimed because no one has time? What projects stall because they need a skill that lives on another sub-team? Those gaps are where people grow.

Give your people a real task with stakes and support. Pair them with someone who already has the skill, and let them ship something that matters.

The key is to keep the core structure intact. The data engineering sub-team still owns ingestion, and he analytics sub-team still owns BI, but the people inside those sub-teams should move.

Remember to set a rule. Every quarter, each person should spend at least twenty percent of their time on work on the area they want to grow. Give them real work that the team needs, and track that work in your planning tool and talk about it in one-on-ones.

Liskov Substitution Means Everyone Knows What Everyone Else Does

In software, the Liskov Substitution Principle says a subtype should be replaceable with its base type. In team building, people translate this to a simple idea: if Engineer A goes on vacation, Engineer B steps in seamlessly. Every engineer must know what the others do.

Modern data teams are small and do not have the headcount for deep specialization. I like this, because every engineer knows what the others do. They learn it in dailies, show-and-tells, and pull requests where we talk out loud.

We also try to have at least two people for every role. The lead should do the job too. Managing is not the whole job.

One more note in case this gets confusing. Single Responsibility separates the roles, while Liskov Substitution makes people interchangeable within the role. The boundary sits between roles, and not inside them.

So when someone takes annual leave, the system does not break because Engineer B is a clone. Engineer B keeps the system running because they paid attention. They read the PRs and asked questions in standup. They knows which jobs fail, and which tables no one trusts.

How You Do It

Read more

I built a live chat feature on MongoDB when the company already ran on MySQL for everything else.

I told myself it was the right call. Chat data is unstructured, MongoDB handles that better, and everyone on LinkedIn was writing about document stores as the future. I also had a quiet thought I never said out loud, that this would look good on my CV.

The founder sat on the desk next to mine. He asked why we needed to introduce a new piece of technology, and I gave him the technical answer when he wanted the business one. Over the next few months, he stopped asking my opinion on decisions he used to loop me into, and I earned that. I was building for what felt like the future, or what felt impressive, or what felt safe against some imaginary scale we were nowhere near, not for the business in front of me.

It took me years and a couple of good mentors to unlearn that habit. Once I did, the same instincts that picked MongoDB for the wrong reasons picked our move from Redshift to Snowflake, our shift from ETL to ELT, and the status page that ended our incident-question flood, for the right ones. That shift took me under four years from data engineer to head of data engineering. I have been writing strategy documents for close to a decade now, and the framework underneath all of them is simpler than anyone selling you a maturity model wants you to believe.

This issue is brought to you by Cube.

Agents do the analysis. Your team does the thinking.

A question about your own business data still means a ticket, a queue, and a week of waiting. Cube hands that workflow to AI agents: they model the data, explore it, and build the reports — your team guides and approves — so week-long analysis comes back in minutes. And because every agent works from one governed set of metric definitions, the answers come back right. 400+ companies run on Cube, including Brex, Wix, Patagonia, and Webflow.

Get started for free

What Everyone Calls a Data Strategy

Ask ten data teams what their strategy is and most will name a tool.

• “Our strategy is moving to Snowflake“

• “Our strategy is finally adopting AI“

• “Our strategy is switching to data mesh“

Those are purchases wearing a strategy costume, the same mistake I made with MongoDB when I picked a tool and backfilled a reason for it instead of starting from a reason and letting the tool follow.

This especially bites small and mid-sized teams. You don’t have the headcount or the runway to recover from a wrong bet the way a 10,000-person company can. Data Mesh proves the point: only 10% of companies under 50 employees adopt it, against 27% of large enterprises, while 43% of small teams centralize around a single warehouse compared to 29% of enterprises, per the 2026 State of Data Engineering Survey by . Small teams already apply real-gap thinking to architecture. The habit just hasn’t spread to every other tool decision yet.

Search “data strategy“ and you’ll find the same pattern from consultancies and platform vendors, long guides on maturity models and governance frameworks that confidently define what a data strategy is while skipping the five decisions that actually produce one.

What a Data Strategy Actually Is

A data strategy is the connection between where the company is going and the specific data infrastructure, pipelines, and capabilities you fund to get it there. It’s not a tool, and it’s not a slide of principles. It’s an answer, backed by a real price tag, to what data capability the business needs and how you’re closing the gap between that and what you can actually deliver today.

You get to that answer by working through five decisions, in order:

1. Where the company is actually going

2. What that growth demands from data specifically, in volume, speed, sources, and metrics

3. Where your current data infrastructure falls short of that demand today

4. The portfolio of data projects that closes the gap

5. The price of each project

Skip any one of the five and you get a tool decision masked as a strategy. I skipped the first step for years without realizing it.

Here’s how to build it step by step:

1. Where The Company Is Actually Going

Most engineers and tech leads don’t know where the company is headed in the next 12 to 24 months, and that’s not a personal failure. Nobody hands you that information by default, and most technical roles are structured so you never have to ask for it.

It’s your manager’s job to know this, or to go find out, since that’s part of what they’re paid for. If you don’t have this information, ask for it instead of guessing.

Bring this into your next 1:1: ask what the business needs to be true in 12 months, and what changes for the team if it happens. If your manager doesn’t have an answer, that’s useful information too. It tells you the gap starts above your team.

Don’t skip this step if you aim and building your data strategy at any capacity.

2. What That Growth Actually Demands From Data

This is where “what if we suddenly had 100,000 users“ belongs, and where I used to get it wrong. The instinct to plan for scale isn’t wrong, only pointed at a number you invented instead of one the business gave you.

A semantic layer is a good test case for how this is supposed to work. A year ago, teams argued over whether a standalone semantic layer was worth it for a small setup. Adoption has since tripled, from roughly 8% to 28%, and none of the data leaders tracked in the latest survey still question its value, per Hex’s State of Data Teams report. The debate ended because the demand became real: too many teams needed one place where a metric meant the same thing everywhere.

Once you have a real growth target from step one, translate it into what it demands from data:

• How much volume?

• How fast the data needs to move?

• Which new sources show up?

• Which metrics the business will start asking for that don’t exist yet?

Every one of those demands has to trace back to something the company actually said. Do not assume.

3. Where You Fall Short Today

This is the step where the founder lost trust in me. The gap between what I built and what the business needed wasn’t technical, since I was already a strong engineer. The real gap was that my decisions and the company’s needs had stopped lining up, and I couldn’t see it because I never checked.

Name your gaps against the real demands from step two, not against what feels outdated or what a blog post said was falling behind. A tool can be old and still be exactly right for the demand in front of you.

4. The Portfolio of Projects That Closes The Gap

A strategy is the full set of projects required to close every gap you named in step three, not one project picked on its own.

Small teams that get this right tend to converge on the same kind of tools. Dagster shows almost four times the adoption in companies under 50 employees, 11%, compared to large enterprises at 3%, not because it’s trendy, but because it fits the orchestration gap a small team actually has, low administrative overhead over more knobs to turn. Meanwhile, more than 20% of data professionals surveyed still run with no orchestration at all, per the 2026 State of Data Engineering Survey, which is usually a sign nobody has walked through steps one to three yet.

This is what it looked like for me once I started doing this the right way:

• Redshift to Snowflake, because concurrent query demand from a growing analytics team was hitting a real ceiling, not because Snowflake was trending

• ETL to ELT, because the business needed raw data available faster than our transform layer could keep up with

• The status page, because stakeholder trust was breaking down over incident visibility, the same kind of trust I’d already lost once with the founder

Each project closes a named gap, not a line I wanted on my résumé.

5. The Price of Each Project

This is the step that turns a strategy into a plan, because every project on your list competes with every other project for the same headcount and the same hours.

The ELT migration cost us months of both engineers’ full attention, the entire data team at the time, and for a while it was the only thing getting worked on. There was nobody else to pick up anything else, so everything else waited, including things I wanted to build. That was the right trade, because the demand it closed mattered more than what it delayed. But it was still a real cost, and pretending otherwise would have made the next prioritization conversation dishonest.

Name the price before you commit:

• Budget

• Headcount

• Time

• What you deprioritize to fund it

If you can’t name what you’re giving up, you haven’t actually decided anything yet. Small and mid-sized businesses that adopted cloud computing and modern data infrastructure saw 19% higher turnover per worker, even after controlling for company size, per GOV.UK’s Business Data Use and Productivity Study. That’s the return on doing the work instead of buying the tool that looks good on a slide.

I turned this exact five-step framework into a worksheet, one section for each step above, so you can run your own team’s data through it instead of mine. Grab the worksheet here and fill it in against a real goal your company has right now, not a hypothetical one.

Get the strategy worksheet

Why Maturity Models Are Useless for Most of Us

Every top-ranking article on “data strategy“ leans on the same shape, a maturity model with four or five pillars, usually governance, architecture, people, and technology, each staffed by a dedicated owner and reviewed on its own cadence. McKinsey and Gartner built this model for a client with two hundred people in the data org and a budget line for each pillar, but this is is rarely the case for data teams in 2026.

If you’re running a team of two, or five, or even fifteen, data strategy is something completely different. You have no governance lead, no architecture review board, and no spare headcount, only you, maybe one other engineer, and a backlog that never gets shorter. Running a four-pillar maturity model on a team that size spends a whole quarter drawing a framework nobody on the team has the headcount to staff, instead of adding any real rigor.

Most of that content is bullshit for a team our size. The five decisions in this article are what’s actually left once you strip away every part of the enterprise version that assumes resources you don’t have.

Final Thoughts

A lot of data leaders fall for this at scale, not only individual engineers. The share naming AI their #1 priority jumped from 4% to 27% in six months, per Hex’s State of Data Teams report, and for plenty of them, AI became the strategy by default, a company-wide bet made because of the hype, not because anyone ran it through five decisions.

I’d guess an equal number work inside companies making that exact bet and recognize it for what it is. Experienced data leaders treat the mandate as the demand in step two, not something to fight, and build the actual gap-closing projects underneath it, the four decisions the company skipped, for their own team.

After seeing the this kid of tool-driven strategy many times in the past, and even making the same mistake myself, I find this “spray and pray” approach ridiculous. Obviously, you can get lucky, but luck is not enough when you are trying to build long-term career and business.

That said, I strongly encourage you to use AI and build AI-powered projects. There’s is value in this, but you need to avoid putting all your eggs in the same basket. At the end of the day, your job only matter for the business if you help it make more money, save costs or avoid risks.

—

Until next time,

Yordan

PS: Want to learn how to think and behave like a leader and actually deserve your promotion? Upgrade today.

More on the Topic

• Strategy-first tooling

• How to Think Holistically About the Data Ecosystem

• The Signs Your Data Team Is Becoming a Cost Center

Subscribe now

And still, he opened our session with a fear I hear from experienced engineers every single week.

If I don’t move beyond this, I won’t find myself anywhere.

Hari wants machine learning and AI skills before those tools make his job obsolete. This article walks through how I coached him from that vague fear to a concrete action he started within one hour of the call. Steal the process, because it works on any career goal and you need no coach to run it.

Interrogate the Fear Behind the Goal

When Hari named his goal, I played back what I heard before touching any plan.

You’re a strong engineer, you’re praised, and yet I hear a fear of becoming obsolete because of AI.

He confirmed it. The buzz says move to AI or lose your job, and he reads that buzz daily on LinkedIn.

So I pushed on the goal itself:

• Do you want this because the work pulls you, or because you believe organizations demand it?

• What happens if your company decides tomorrow that they no longer want ML engineers?

A goal built on someone else’s noise collapses the moment the noise changes direction.

Hari’s honest answer moved the whole session forward. He wants to shift from reactive work to predictive work, and he wants to show the business things nobody asked about yet. Fear was the trigger, but underneath it sat genuine curiosity, and curiosity survives obstacles far better than fear does.

Before you plan any skill jump, write down your reason and read it back. If the reason quotes the internet instead of you, keep digging until it quotes you.

The Four Jobs Hiding Inside “Learn AI”

Most people who say they want AI skills have never separated the jobs behind that phrase. I gave Hari four lenses, and the one you pick changes everything downstream.

• MLOps engineer. You deploy what others build. CI/CD, resources, servers. You live between infrastructure and machine learning.

• AI engineer. You build applications with AI. Hari’s three-day Streamlit dashboard, built with Claude, is this lens in action.

• Machine learning engineer. You build models. You spend weeks, sometimes months, perfecting something that predicts churn, device failure, or a financial forecast. Deep, slow, less flashy work than the hype suggests.

• Data platform engineer. You build the platform where models learn and pipelines run. My team currently works through this lens on a project that reads conversational data and surfaces what customers complain about.

These are different jobs with different daily work, and a plan aimed at all four hits none of them. Hari picked prediction. He wants forward-looking models built on business data he already understands, which meant we finally had something specific enough to plan.

Walk the Goal Backward With 5-4-3-2-1

Big goals fail for a predictable reason. The picture is vague, the plan is huge, and nobody knows what to do today, so nothing happens. I borrowed a fix from a journaling book years ago, and I still use it for myself. The 5-4-3-2-1 cascade walks one goal backward through five horizons, from five years out to the next sixty minutes, and every level produces one specific commitment that supports the level above it.

This is how it goas

Read more

This is not how you lead a team. You got the title, but are still thinking and acting like a senior engineer, who gets paid more.

I was harsh, but Nikol needed to hear this.

She came to me asking why her manager, and her own team, do not like working with her. I heard her side of the story, and my answer was instant.

I was harsh, but Nikol needed to hear this.

She came to me asking why her manager, and her own team, do not like working with her. I heard her side of the story, and my answer was instant.

I have seen this exact pattern too many times. I lived it myself.

How The Trap Sets In

You are the best engineer on the team. Every hard task goes to you, and nothing feels impossible with enough time. You help the junior engineers get unstuck before they even ask their manager.

So you get promoted. Not because anyone taught you how to lead, but because you are clearly the strongest engineer in the room, and the company had to put that title somewhere.

You don’t get any handover, training or mentor.

Everybody expects you to figure it out, because obviously you are the best engineer they have.

And while we are on the topic, bookmark Your First 60 Days as a Data Engineering Lead. I did this for leads who are never going to get a mentor handed to them.

The Promotion Changes The Job

Suddenly you own every project timeline on the team. You inherit:

• roadmap conversations

• headcount requests

• standups you used to sit through half listening

You join stakeholder meetings and one to ones that used to belong to someone else. And underneath all of it, you are still trying to do the engineering work you did before, because that is where the real output is.

Nikol told me she was still reviewing every pull request on the team personally, the same way she did as a senior engineer.

I used to do the same. When a project slipped, the instinct kicked in, I got nervous, and made a comment about someone’s code quality or their speed, in front of everyone, in the standup.

Then I took the ticket back and finished it myself, faster than they would have.

That feels like leadership, but isn’t.

What This Looks Like Week To Week

It rarely looks dramatic from the outside. It looks like a string of small decisions that add up.

• You rewrite a teammate’s PR instead of leaving a comment explaining what to change

• You pull the ambiguous stakeholder request into your own inbox instead of pushing your team to scope it themselves

• You stay the on call escalation point months after you should have handed it off

• You measure a good week by how much you personally shipped, not by whether your team unblocked itself without you

Together, they tell your team you don’t trust them with the job you were supposedly promoted away from.

From Engineer To Operator

As an engineer, your output is your code, but as an operator, your output is your team’s code, and most of your day goes into things that do not look like work: conversations, decisions, unblocking, and deciding what not to build.

The instinct that made you a great engineer, write more, ship more, prove it yourself, is the instinct that now holds your team back. Here is what actually has to change, one reflex at a time.

Your Output Stops Being Your Own Work

The fastest way to check where you actually are: look at last week’s calendar. If most of it happened inside a codebase and not inside a conversation, you are still operating like an engineer with a bigger title.

The new measure is not how much you personally shipped. It is whether the work landed, whether your team is better than it was last quarter, and whether your manager can stop checking on your project without you noticing.

Stop Being The First Responder

You are the fastest debugger on the team, so you jump in and fix most incidents before anyone else even opens the alert.

That instinct solves the incident once. It also teaches your team to wait for you every single time, because you keep proving that waiting is faster than trying.

Nikol was the fastest responder on every incident her team had. That is not why her team resented her, but was one of the reasons nobody else on the team ever got faster.

Stop Translating Requests Into Tickets Yourself

A stakeholder asks for something vague, and you are the person who turns it into a scoped, technical ask in five minutes.

This is not your job anymore.

Every time you do the translation yourself, your team gets instructions instead of context. They stay one layer removed from the actual business problem, and you become the one person who cannot take a vacation without something breaking.

Sit in the stakeholder meeting. Let a senior engineer take the first pass at scoping the request. Correct it after, not before.

My team hates this, because this actually requires more effort than writing code. But this is how they grow.

Stop Avoiding The Conversation

Nikol never told her team their scope was wrong or their pace was too slow. She took the ticket back and fixed it herself, every time it happened.

The conversation is uncomfortable exactly once. Doing the work quietly and resenting your team for needing you to repeats every single sprint.

I recently read a post by asking if you can be a good manager if you hate conflict. An yes, you absolutely can. But you can’t avoid it.

As a leader you can’t avoid hard conversations. Not with stakeholders, not with your manager, and definitely not with team members. You are getting paid to have these conversations.

The more you avoid them, the deeper problems get. Bu do it with respect.

What Actually Makes You Effective Now

Here is the part that took me the longest to accept. The more senior you get as a leader, the less your technical skill is the thing carrying you.

At some point the job stops rewarding the person who solves the hardest technical problem personally, and starts rewarding the person whose team solves problems without them.

This is a completely different job.

Nikol is still leading with the one skill she trusts, because nobody told her the other skill exists. Nobody told her that her team was slow because she had never once told them, directly, what she actually needed from them.

What Making The Shift Actually Looks Like

The next time a project slips, do not open the ticket.

Ask what got in the way, not to assign blame, but to find the actual blocker. It might be unclear scope, a missing dependency, or a skill gap you can now train for instead of working around.

Then say the thing you were avoiding. Something close to:

I expected this by Thursday. It is Monday and it is not done. Walk me through what happened.

That sentence takes ten seconds, and does more for your team than three hours of rewriting their code ever will.

The Operator Audit

Nikol did not fix this in the one conversation where I called it out. This change doesn’t happen overnight. But in the 2 months we worked together she got significantly better. She has the tools to deal with it now.

Here are the five points Nikol and I were working on. Run this audit occasionally and go wild. Score yourself if needed.

1. Your output

Did you write code this week to prove a point, not because nobody else could do it?

Before you open your IDE (I use neovim, btw), ask whether a teammate could do the task with the right context. If yes, give them the context instead of doing it yourself.

2. First responder

Were you the first person on the last incident, even when someone else was on call?

Next time, wait ten minutes before touching anything. Let the on-call engineer make the first move. Step in only if they ask.

3. Translating requests

Did you personally turn a vague stakeholder ask into a scoped ticket?

Next request, bring a senior engineer into the conversation and let them draft the first pass of the scope. Correct it after, not before.

4. Avoiding the conversation

Did you skip a hard conversation and hope the problem would resolve itself?

Book fifteen minutes this week with the person whose work you have been quietly redoing. Name the specific gap you have noticed. Ask what is actually blocking them.

5. Measuring effectiveness

Do you know if your team is better this quarter than last, without checking their commit history?

Write down one thing your team now does without you that they could not do three months ago. If you cannot name one, that is this month’s actual project.

Answer honestly on all five. Then pick one action and do it this week. Not all five. One.

—

Until next time,

Yordan

PS: Reading about the operator shift is free, but the guides, scripts, and frameworks that make it happen live in the paid tier.

PPS: If you are living Nikol’s exact situation right now, that is what coaching is for. Book a call and let’s build your plan instead of guessing your way through it.

Let’s Connect

Connect on LinkedIn: https://www.linkedin.com/in/ivanovyordan

Work with me: https://www.ivanovyordan.com/coaching

Start journaling: https://www.dearself.ai

More on the Topic

• Why Senior Data Engineers Lose Their Velocity

• Two Ways to Disagree With Your Lead But Only One Is Right

• How To Stop Waiting For Permission To Lead

Most data professionals I talk to work incredibly hard at their craft. They go deep on Python, on dbt, on Spark, on whatever the stack is.

They take courses, they build side projects, they read documentation on weekends. And then they watch someone else get promoted. Someone who, honestly, is probably less technically sharp than them.

Introducing David Langer

Dave Langer has been in technology for almost 30 years. He started on the help desk, wrote COBOL on a mainframe at an insurance company, moved into software engineering, and lived through the dot-com crash in the early 2000s.

The last 15 of those years have been in analytics, starting with traditional BI and data warehousing, Kimball star schemas, the whole thing, and then moving into more advanced analytics.

Dave worked at Microsoft leading a data and analytics team, then at a startup called Schedulicity as VP of Analytics, and at Data Science Dojo. He is a Microsoft Excel MVP, not for writing formulas, but for evangelizing Python inside Excel, which he got early NDA access to in 2023.

David wrote a book on Python and Excel published this year, and his second book on SQL for Excel users is coming in 2027.

These days Dave is an independent consultant and trainer. He runs a Substack called The DIY Data Scientist, which is exactly what it says: practical tutorials on data analysis for professionals who want to use data better, regardless of their background.

We had a live conversation, and this is everything worth taking from it.

The Value Ladder Is Not What You Think It Is

Every organization has a value ladder. A rough hierarchy of what it perceives as worth paying for, promoting people for, and building strategy around.

The problem is that your version of the value ladder and your organization’s version are often completely different things.

When you are early in your career, this does not matter much. You are rewarded for technical output:

• Write good code.

• Ship clean pipelines.

• Produce accurate models.

That is the ladder, and being technically excellent gets you up it.

But at some point the ladder changes. And nobody tells you.

What the organization starts rewarding is perceived strategic value.

That could mean designing systems instead of building them. It could mean being able to translate between the data team and the C-suite. It could mean understanding enough about the business to push back on a requirement intelligently, not just implement it.

If you keep climbing the old ladder while the organization has already moved to a new one, you plateau. Only because you are solving for the wrong problem.

Subscribe now

The Outsourcing Lesson

Think about what happened to a lot of senior engineers in the early 2000s. They had gone deep on C++ (Qt will stay in my heart forever), and read all the right books, so they could architect complex object-oriented systems.

And then globalization happened, and suddenly all of that specialized knowledge became a commodity. You could hire it for a fraction of the cost from an outsourcing firm.

The people who survived that shift asked: what is the value ladder actually rewarding right now? And the answer was architecture, system design, coordination, the things that required judgment and organizational context. Not just technical execution.

The same question applies to you today. Not “how do I get better at this tool?“ but “what is my organization actually paying for?“

If you want a concrete way to map where you actually sit on your organization’s value ladder, that is exactly what the Career Progression Matrix in the premium library does. Paid subscribers get access to it on day one. Upgrade here.

Data Problems Are Not Technology Problems

Here is what nobody told me early in my career:

The hardest part of working in data is not the data itself, but people

If you’ve been in data long enough, you’ve probably been in meetings about the definition of a customer for way too longs. And all of that, just because four departments each had a different answer, and each answer made a different executive look better or worse.

This is what senior data professionals figure out. And it is what a lot of talented junior and mid-level people never fully internalize.

If your goal is to do technically excellent work in a well-scoped problem, you can stay in your lane and deliver. That is a legitimate career.

But if you want to actually drive decisions, if you want your work to change how an organization operates, you are going to spend a significant amount of your time as a mediator.

• Between engineering and business.

• Between what the data says and what someone wants the data to say.

• Between this year’s urgent request and the foundational investment that would make next year’s requests 10x easier to answer.

The people who succeed in that space are not always the most technically gifted. But they are always the ones who learned to sit in an uncomfortable call, name the real disagreement out loud, and not flinch when someone gets defensive.

Why Teaching Is a Career Asset

One of the underrated ways to build this skill is to teach.

Not necessarily a course or YouTube tutorial. Just the act of explaining complex technical concepts to someone who does not share your context.

It forces you to leave your own head and think from the other person’s perspective. You cannot fall back on jargon. You have to find the analogy that lands for this specific person, not the definition that would satisfy a technical reviewer.

That skill transfers directly to stakeholder work. Every time you explain a data model to a VP, every time you scope a request with a product manager who is not sure what they are asking for, every time you push back on a dashboard request by asking “what decision are you trying to make?“ you are doing the same cognitive work.

Data professionals who build that muscle early tend to accelerate later. The ones who skip it hit a ceiling.

Show & Tell is where I work through exactly this kind of thing live. One paid subscriber brings a real situation, and we work through it together in front of the group. Every session is recorded, and included in a paid subscription. Join here.

Executives Are Running the Same Hype Cycle Again

A few years ago, the mandate was “we need to be data-driven“. Companies hired data teams, built dashboards, set up data warehouses. Most of them never actually became data-driven. But “data-driven“ was the thing, so they built it anyway.

Now the mandate is “we need to be AI-first“.

The structure is identical:

• An executive reads something.

• The pressure cascades down.

• Someone, often the data team, gets handed the brief: implement AI.

The business problem that AI is supposed to solve is treated as secondary. Sometimes it is not even identified.

I have seen a company run a churn model comparison: a basic logistic regression built by a data scientist, versus an LLM-based approach. The logistic regression hit 80% accuracy. The LLM hit 50%, which is coin-flip territory. The traditional model also cost a fraction as much to build and run.

And that is before you account for what happens when AI pricing moves from flat subscription to usage-based. When your CFO is looking at a bill where the token cost exceeds the salary of the team that could have built a simpler model, the conversation changes fast.

Most organizations are not thinking about this right now. They are still in the “we need an AI use case“ phase. The ROI calculation comes later, usually painfully.

The Hadoop Lesson Is Already Written

During the Hadoop hype cycle, an executive asked Dave whether the company could move their SQL Server databases to run on top of Hadoop instead of fast disks. Because Hadoop storage was cheaper. The executive had read about it in a trade magazine.

The answer was no. Obviously. But the fact that the question was asked at all tells you something about how technology hype filters through organizations.

It’s all about the anxiety of not being part of the trend.

• MongoDB was supposed to kill relational databases.

• Excel has been declared dead every year for 20 years.

• SQL was going to be replaced by query builders.

None of that happened.

Some things change. Programming languages come and go. Platforms rise and fall. But the underlying problems stay the same: organizations need to make better decisions using data.

The data needs to be clean, structured, and understandable. The people consuming it need context.

The Two Questions Worth Asking

Here is what I would take from all of this.

First, be honest about your organization’s value ladder. Not the one you wish existed, but the one that actually exists.

• What gets people promoted here?

• What gets people noticed by leadership?

• Is it technical depth, or is it business impact, or is it visibility, or is it the ability to manage up

Because the answer changes your strategy.

Second, ask whether that ladder aligns with what you are actually trying to build. If it does, optimize hard for it. If it does not, that is useful information too. The organization where the ladder aligns with your goals exists. Sometimes you have to find it.

The data professionals who stall out tend to do one of two things: they optimize for a ladder that their organization does not value, or they assume the ladder they are on right now is the one they will always be on.

Neither is true.

Find More From Dave

Dave’s Substack is The DIY Data Scientist. Every issue is a hands-on tutorial with workbooks, code, and data included, free. If you work with data in any capacity and want to actually understand what you are doing with it, it is worth subscribing.

He is also active on LinkedIn, where his content leans more toward the enterprise and organizational side of analytics.

His book, Python for Excel, is available on Amazon and in Barnes & Noble stores.

—

Until next time,

Yordan

PS: Paid subscribers get the Career Progression Matrix, the full Show & Tell archive, and the entire premium resource library. Upgrade here.

PPS: The next Show & Tell session is coming up. A paid subscriber and I will be working on a long on a long therm-career plan. Join here.

Let’s Connect

Connect on LinkedIn: https://www.linkedin.com/in/ivanovyordan

Work with me: https://www.ivanovyordan.com/coaching

Start journaling: https://www.dearself.ai

More on the Topic

• The Other Half Of The Job

• Why Senior Data Engineers Lose Their Velocity (Even When Their Skills Improve)

• Your Best Work Is Invisible Because You Describe It Wrong

If you’ve been in data for a while, something feels different now.

The job you spent years getting good at has quietly changed around you. Everything is different, now. Not just tools and pacing. What leadership expects from you barely resembles the job description you, barely resembles the job you signed for.

You are watching a job you love change in front of you, without feeling ready for what it’s becoming.

Here’s what I think is happening, and why most of what you’re reading about it is making it worse.

What’s your leadership experience

The Truth About AI in Data

You know how the famous saying goes.

AI is like teenage sex. Everybody talks about it. Everybody thinks everybody else is doing it. Nobody really knows how to do it. And if you are actually engaged in it, you do it really badly.

You just need to open LinkedIn, or read any of the big data engineering blogs, and you’d think you’re dangerously late. You’d think every other company already has autonomous agents in production, a fully deployed semantic layer, and AI-generated pipelines shipping while the team goes for coffee.

So you shift roadmap, add a tool or two for evaluation, and somebody ships a workflow nobody asked for against a deadline nobody set. When it goes badly, nobody understands why, because the whole thing is about catching up to a benchmark that doesn’t even exist.

And you know what? Nobody is ready for this. Everybody is doing stupid shit and calling it strategy.

Companies used to have one goal: make more money. This goal is tangible and measurable. You knew if you were winning. Now the goal is “adopt AI“. And when you cannot define success, you cannot recognize failure either.

This issue is brought to you by Cube.

The best way to support me and this Newsletter is to check out the Sponsor below and click the link.

Cube Is the AI-Native BI Platform for Your Databricks Lakehouse

Cube is the agentic analytics platform for teams on Databricks. One platform for internal BI, embedded analytics inside your product, and an AI Analyst that answers questions in plain English.

All of it runs on a governed semantic layer, so a metric defined once means the same thing in a dashboard, a customer-facing app, and an AI query. Databricks holds your data. Cube is what your team, your customers, and your agents actually ask.

The Cube team is running a free 45-minute webinar on June 30 (1:00 PM ET / 10:00 AM PT) with a live demo connecting Cube to Databricks and exploring the same governed metrics through a dashboard, an embedded app, and the AI Analyst.

If you’re on Databricks and deciding what to standardize on, this is the session to watch.

Register free for the webinar

What the Race Produces

Data democratization

Every platform now enables agentic workflows. And since building is trivially easy, everyone builds their own thing. Your CEO probably has a workflow that pulls data from your BI tool. And your head of finance has one pulling from a different source, with a different definition of the same metric.

Call it democratization if you want. To me that’s decision surface area with no shared truth underneath it.

Give people access without defining what the metrics mean, and your org starts running on wrong numbers. At some point everyone’s dashboard shows something, and something feels like progress.

Don’t get me wrong, this problem existed forever. Now, it’s just appears at scale.

Staling innovation

There’s a deeper problem though, and this one bothers me more.

AI learns from existing code. It reproduces what the industry already built. AI can’t invent what nobody has built yet.

Things like DuckDB, Polars and Arrow came from people doing non-consensus work. A model trained on yesterday’s patterns would never be able to produce these.

If you outsource thinking to AI, the field converges. You get well-executed mediocrity, forever. I don’t think enough people are saying that out loud.

One thing that held though is code. Whatever AI generates, you can read, version, and test. The craft is the same.

Subscribe now

The Mess Is Your Backlog

Here’s what I’m watching happen.

The more carelessly people adopt AI, the more broken processes pile up. All these AI-generated assets that I told you a second ago are the natural output of an industry that moved fast without thinking.

And they are your backlog. Try this process just once:

1. Go talk to someone who isn’t on the data team.

2. Find the process that makes them want to throw their laptop.

3. Ask what they’re doing with AI and watch what happens.

I find something worth fixing almost every week.

And you know what? The more people use AI carelessly, the more stupid shit piles up for someone like you to fix.

The Raise Of Governance

That’s also why governance matters more right now than it has in years. Keeping your org from drowning in wrong numbers is the work.

Semantic layers used to feel like a gimmick to me. A nice idea that never justified the overhead.

I’ve changed my thinking on that. When your CEO, your CFO, and your VP of Product are each pulling “revenue“ from three different sources, a semantic layer is the only thing that gives you a shared definition both humans and machines use.

One source of truth that doesn’t depend on who built the dashboard.

I’m working on a project that helps us solve this in my org, and I’ll post a proper write-up when I have results.

Final Thoughts

I don’t know where this is going. Nobody does, and I’m skeptical of anyone who says otherwise. But I’m not worried, and I don’t think you should be either.

I believe you are fine as long as you stay current, solve real problems and skip the hype.

Every new type of work has something interesting in it if you go looking, and the people who adapt are always the ones who go looking. The job title might change, and the tools will definitely change. But the skill of finding a real problem and fixing it doesn’t go anywhere.

If this whole thing goes sideways, I’m genuinely good at building, electricity, and plumbing, although that last one is not my favorite. That was never the plan, but it’s a real option in case of disaster.

—

Until next time,

Yordan

PS: If you’re an experienced engineer trying to figure out the next move, the premium library is where I put everything on navigating the leadership ceiling, stakeholder politics, and getting paid what you’re worth. Chek it here.

PPS: If you’re figuring out your next move, a promotion, a raise, or what your career actually looks like in this market, I do one-on-one coaching with data people. Learn more when you are ready.

Let’s Connect

Connect on LinkedIn: https://www.linkedin.com/in/ivanovyordan

Work with me: https://www.ivanovyordan.com/coaching

Start journaling: https://www.dearself.ai

More on the Topic

• Intro to MetadataOps with Alejandro Aboy

• Why projects die

• Self-Service BI Is a Lie: 3 Problems You Can Resolve Today And Improve It

Every time something breaks in our data platform, somebody posts an announcement in the 6 Slack channel. Every single time, we get the same questions:

• “Is the Bookings dashboard affected?“

• “What about the ML report?“

• “Are the numbers in the product usage report going to be wrong?“

Three to five variations of the same question, from different people, about different things, all asking me to do the work they should do themselves.

Not that the announcements are unclear about what’s wrong, but we can’t list everything that’s fine. So they do the only thing available to them and asked.

And that’s only a part of the story. When a failure happens at the ingestion layer, we do not always know immediately what downstream data it affects. The lineage from a broken pipeline to a broken dashboard is not always obvious without the right tooling.

I decided to fix both at once. What I built is a data platform status page. The result was zero questions per incident. It has been that way since we launched it.

Here’s how you can do the same.

Data Teams Love the Word “Product” But Rarely Treat Their Platforms Like One

The data community has spent years arguing that data should be treated as a product. We now have data product managers, data product roadmaps, data product thinking and what not.

The common understanding is that data teams should operate with the same rigor, user focus, and accountability as software product teams.

And then those same teams don’t even post a Slack message when something breaks.

Every SaaS product you use has a status page. When something goes wrong with Stripe or GitHub or Claude (yesterday anybody?), there is a single place you go to see what is affected, what the team is doing about it, and how long the incident has been running.

You do not send a message to the Dear Self support team (that’s just me) asking if your journal processing is working. You go to status.dearself.ai.

Data teams do not do this.

They:

• sometimes announce incidents in Slack

• generate a thread of panicked questions

• answer those questions individually

and repeat the process next time.

The data as a product positioning is real in the planning meetings, but always disappears the moment something breaks.

The gap is in mindset, and the tooling to close it has been available for years.

Status pages are a basic operational practice that product teams figured out long ago, and somehow you don’t have them for your data “products”.

What You Need to Build This

If you’re running a modern data stack, you likely have most of the pieces already. Here’s what our setup looks like:

• An ELT process that lands raw data in the warehouse with a schema that mirrors the source. Everything in the lineage graph starts here.

• Two dbt repositories. One handles data engineering transformations. The other handles business-level transformations. This is has nothing to to d

• A data observability tool with column-level lineage. We use SYNQ. Without column-level lineage, the question “what downstream assets does this failure affect?” has no reliable answer.

• A BI tool. We use Omni. The specific tool matters less than whether it exposes an API.

And a few notes on that.

First, the process obviously works with ETL, but it’s slightly easier with ELT. We switched to ELT long ago, and our reasons may not be true for you. Don’t change your process if it works well.

Second, you absolutely don’t need 2 dbt projects. I decided to keep them separate, so I don’t mix skills, standards and workflows for our 2 subteams, DataOps and BI.

Last, but not least, you don’t need SYNQ specifically. Any observability or cataloging tool with column-level lineage works:

• Elementary is a solid open-source option.

• OpenLineage works if you’re building from scratch.

The principle is the same: you need a system that knows how data moves from source to consumption layer and exposes that through an API.

The Mapping Problem (And How AI Solved the Boring Part)

And here’s what makes this harder than it looks. SQNQ knows plan_id in the accounts table flows through some transformation int the bookings model. But it doesn’t know is which dashboard tile queries bookings.

Without that mapping, the status page tells stakeholders a dbt model failed and nothing about whether their report is affected.

SYNQ doesn’t support Omni out of the box. So we built the bridge. I wrote a script that:

• pulls a selected set of official dashboards from Omni’s API

• extracts every tile in those dashboards

• retrieves the query each tile executes

That gives us a map between Omni tiles and the underlying dbt models. Combined with SYNQ’s lineage graph, we trace a failure at the ingestion layer all the way through to specific dashboard tiles and surface exactly which reports are affected.

How an Incident Works

When a test fails or a pipeline breaks, we get a notification. So we run a quick triage: is this a real incident affecting stakeholders, or a transient failure we resolve quickly and silently? If it’s real, we go to SYNQ and declare an incident.

That declaration triggers a Slack notification to the channel. The notification includes a link to the status page, where stakeholders see exactly which data products and reports are affected. They don’t need to ask. The answer is already there.

As we work through the incident, we add comments and updates directly in SYNQ. Each comment triggers another Slack notification. Stakeholders get a running update without anyone on the team writing a separate message or copying the same information across threads.

Stakeholders follow the link, see the state of their data, and go back to their work.

What the Status Page Shows

The page is structured around data products, instead of incidents. The whole point is to give them access to something user-friendly that’s structured around how they think about data.

Here’s what they see:

• Selected data products. These are groups of dashboards. They have no idea about tables or pipelines.

• Reports per product. A list of reports associated with each data product.

• Current status, which we pull in real time from SYNQ.

• Status history showing the track record over time.

• Individual incident detail pages, including affected reports and a full timeline of updates.

Scoping the page to the products stakeholders care about is what makes it useful. Everything else would be noise.

The page has no database. It makes API calls to SYNQ at request time and renders the current state. Less infrastructure to maintain, and nothing to keep in sync.

Why Showing Failures Builds More Trust Than Hiding Them

The instinct when something breaks is to fix it quietly and move on. You might post a minimal announcement, keep the details sparse, resolve it fast, and say as little as possible.

I know that every incident announcement feels like an admission, and the more you say, the worse it looks.

That instinct is backwards.

Stakeholders lose trust when the team goes quiet during an incident and offers nothing useful until it’s resolved. The team that declares incidents publicly, posts running updates, maintains a visible history, and posts postmortem reports earns a different kind of reputation.

When a random manager raises data reliability as a concern, most data leads respond defensively. They explain the complexity, list the improvements they’re making, and leave the room without having changed anyone’s mind.

With a status page and twelve months of history, the conversation is very different. You pull it up, and show incident count, average resolution time, and trend. The conversation moves from subjective complaint to documented fact, and the documented fact works in your favor.

The status history is the part people underestimate. When you have a year of publicly visible incidents with response times and resolution notes, nobody tells you the data is always broken.

There’s also a simpler mechanism at work. When stakeholders know you’ll tell them when something goes wrong, they stop checking.

They stop asking “is this dashboard up to date?“ as a routine question, because they know that if it weren’t, they’d have heard from you.

That baseline confidence is worth more than any data quality initiative or SLA document you write. A status page operationalizes it so it happens automatically, every time, without anyone on the team having to think about it.

Data teams that earn strategic partnerships with the business are the ones whose stakeholders trust they’ll hear about problems before they have to ask.

Final Thoughts

None of that code was written by me. I read the API documentation for both tools, described what I needed, and handed it to an AI coding tool. The status page, the mapping script, the full integration. The whole project was complete in a matter of days.

The truth is that you don’t need to be a strong engineer to build this.

You need to recognize the problem, read some documentation, and spend a couple of days on it.

Every senior data professional has the ability to make this kind of impact. Most haven’t built it because they’ve never thought to look at what software product teams do and ask what they’re missing.

The skill here was identifying the gap.

AI allows you to go give life to your ideas faster than ever.

—

Until next time,

Yordan

Subscribe to Data Gibberish: https://www.datagibberish.com

Connect on LinkedIn: https://www.linkedin.com/in/ivanovyordan

Work with me: https://www.ivanovyordan.com/coaching

Start journaling: https://www.dearself.ai

PS: Paid subscribers get monthly Show & Tell sessions where you bring a real career, leadership or stakeholder problem and we work through it live together. Upgrade here, when you are ready.

PPS: Also, remember to check my Premium Content Library here.

More on the Topic

• Why trust disappears faster than servers crash when you hide bad news

• Bad news

• Your Best Work Is Invisible Because You Describe It Wrong

You are better at filtering other people’s ideas than your own.

When a stakeholder comes to you with a request, you slow down. You ask what decision this changes, who will use it, what happens if you don’t build it. Most requests don’t survive those questions. You know this. It’s part of the job.

But when the idea is yours, something shifts. The filter goes quiet.

• The shiny new initiative you want to take but that would pull you away from the work that matters.

• The cross-functional opportunity that flatters your ego but doesn’t serve your team.

• The tool you want to introduce but won’t help.

• The migration that’s interesting to do but doesn’t move anything.

• The architectural rewrite that’s been sitting in your brain for two years.

These feel different because they’re yours. You built the case in your head. You refined the idea in isolation, without the friction of someone else needing to fund it, sponsor it, or use it. That’s the problem.

Your Ideas Deserve the Same Scrutiny

The same questions you ask the head of sales when they come to you with a request apply to your own backlog. “What decision does this change? Who’ll use it? What happens if we don’t?“ Most of your ideas won’t survive the filter either.

The filter needs to start earlier than the idea itself.

I run bi-weekly calls with a few selected people across the org. My jov in those calls is to listen. They tell me about their plans and frustrations, what’s slowing them down, what they’re worried about heading into the next quarter. I don’t need to invent problems. People tell me about them.

This matters because the filter only works if the pain is real and externally verified. If the idea starts inside your own head and you go looking for evidence that the pain exists, you’re just looking for a confirmation.

The calls flip the sequence. The pain comes first, from the source, and then you decide whether it’s worth solving.

Once you have a real pain, run it through four gates before you commit a single hour of your team’s time.

Here’s the framework

1. Start with the pain

Describe it precisely:

• Does it have a measurable cost when it goes unsolved, or is it mostly an irritant?

• Is it something others in the org feel, or is it your version of the problem?

2. Research

• What’s the standard outside your org?

• Is this the most important pain to be solving right now, or is there something more load-bearing sitting ahead of it?

Most engineers skip this step. They feel the pain, design a solution, and ship. The research gate is where you find out whether you’re solving the right thing or the familiar thing.

3. Think of different solutions

If you go into solution mode with the one you already want to build, you’re defending an idea, not evaluating a problem. Generate at least two alternatives before committing to an approach.

4. Talk to the people it affects

• What do they think about the pain?

• What do they think about your solution?

• Are they willing to sponsor this, or do they have other initiatives that matter more to them right now?

This last question kills the most ideas, and rightfully so. A pain that nobody will prioritize is a pain the org has decided to live with.

If all four come back green, ship.

Subscribe now

Remember to download the complete audit at the end of the article.

Read more

He wrote the playbook down in his book Your Fractional CDO, after watching the same failure pattern repeat across years of client work.

A data team builds perfectly modeled tables, clean naming conventions, a finished warehouse, and still fails at the one job that matters.

The failure starts in the hiring order, months before anyone touches SQL.

The Pattern Behind Almost Every Broken Data Team

A company grows, raises money or bootstraps itself, and builds product and marketing first. Data waits. By the time anyone notices, the gaps have already turned into blind spots in real decisions.

The fix usually starts with a data analyst, hired to both analyze the data and build the warehouse underneath it. He patches things together with whatever is closest: ad hoc imports from spreadsheets, quick pipelines wired through whatever tool is available. It works until it doesn’t.

So the company hires a data engineer to rebuild the plumbing properly. Pipelines get rebuilt, structure improves, the technical layer finally looks right.

It still doesn’t work. Nobody ever agreed on what the numbers mean.

A clean warehouse built on an unresolved disagreement only makes the disagreement faster.

This is the gap a fractional Chief Data Officer gets hired to close. The job is the translation layer between what the business is trying to decide and what the data team is building. Head of Data, VP of Data, Chief Data Officer, the title changes. The job underneath stays the same: turn business priorities into a data roadmap, then close the distance between the two.

Three People, Three Numbers, One Metric

When the role gets done right, the first move rarely touches the warehouse.

On one project, the opening round of conversations with C-level stakeholders surfaced a single metric, active subscriber, calculated three different ways by three different executives. Each one trusted their own version. Nobody had flagged it as a problem, because nobody had compared notes.

The fix starts with a written catalog of every metric that matters: what it counts, what it excludes, who owns the definition. It lives somewhere simple, a Notion page, an Obsidian vault, plain markdown files, and every stakeholder signs off on a definition before a single pipeline gets touched.

Skip that step and the technical work inherits the argument anyway. A warehouse with perfect naming conventions and a properly modeled transformation layer still produces three different numbers for active subscriber, because the disagreement never lived in the tables. It lived in the room full of people who never sat down to agree on what the word meant.

What AI Changed

The Audit That Used To Take A Month

When Nick onboarded onto a new client’s stack, he inherited a Metabase instance holding around 2,000 saved questions, each one a SQL query sitting on top of the warehouse. Reading through that by hand to figure out which tables and metrics mattered used to eat him three to four weeks.

He connected an LLM to Metabase’s API and turned the same task into a question-asking exercise: which tables get used most, which metrics get calculated more than one way. The 2,000 saved queries referenced around 90 tables in the warehouse. Only 12 of them turned out to matter.

The same pass produced a first list of conflicting metric definitions, the kind of list that used to take weeks of stakeholder interviews to assemble by hand. It became the opening line of the metric catalog conversation. The conversation with stakeholders still had to happen.

A Personal Operating System

Outside of any single audit, Nick’s daily workflow runs through a stack of connected tools: email, Slack, Telegram, WhatsApp, Jira, Google Calendar, a meeting note-taker, a task planner, a personal fitness tracker. Each one is wired in through its own API, feeding into folders organized by client, by teammate, by personal project, written up as markdown files an AI assistant reads for context.

A new lead from the website used to mean reading an email and booking a call. Research on the prospect now happens automatically, the call transcript gets analyzed for next steps, and a draft proposal and CRM update follow without him typing either one by hand.

He still types every prompt himself. Roughly 90 percent of his working day still happens inside an editor, reading something, asking a question about it, deciding what happens next.

The Rules That Don’t Bend

Speed doesn’t extend to trust. A short list of rules stays fixed regardless of how good the tools get:

• Every SQL statement an AI writes gets verified by a human before it touches anything live or anything tied to budget.

• Raw client data never goes into a public AI tool. Only metadata about the data does, table names and query structure, never the rows underneath.

• Sensitive processing happens through local models inside the client’s own environment instead of commercial platforms.

Delivery got faster. The job grew, because speed opens room to take on more work.

Judgment Became The Scarce Skill

As AI absorbs more of the execution, headcount on this kind of team is shrinking on purpose, driven by a bet: enable each remaining data analyst and engineer with AI rather than hiring more people to do routine work that AI now does faster, the documentation nobody wanted to write, the boilerplate code, the first draft of a pipeline.

Not everyone takes that bet. Some experienced data engineers distrust the output outright or keep working the old way out of habit.

That resistance is becoming the dividing line. Once anyone is able to write a working SQL query or stand up a dashboard with AI’s help, calculation stops being the skill that earns a senior title. Reasoning about why a stakeholder cares about a specific number takes its place. The valuable hire thinks strategically about the business and catches the cases where the fast answer is the wrong one.

Reporting Lines Decide Whether The Job Works

Where this role sits on the org chart changes what it’s able to do, regardless of the title on the business card.

Put the data function inside engineering and it drifts toward whatever a chief technical officer recognizes as good work: pipelines, infrastructure, technical correctness, thinly connected to what the business is trying to decide.

Put it inside finance and a different bias shows up. Finance’s own questions get solved first simply because of who the team reports to, while marketing or product priorities wait their turn even when they matter more to the business right now.

Neither placement is wrong on paper. Both pick a winner without saying so.

The function works best as a peer to product, marketing, and finance rather than a report inside any one of them, sitting at whatever level makes the final call on what the company builds next: a CEO in most structures, whoever arbitrates between general managers in flatter ones. The job stays the same either way. Listen across every department, then build the data foundation that unblocks all of them instead of only the one holding the budget.

None of this requires the word “Chief” in the title. It requires someone willing to do the unglamorous work of getting people to agree on what a number means before anyone touches a pipeline, and the discipline to use AI as a way to do that work faster rather than as a reason to skip it.

Find Nick Online

You can find Nick on a number of places. I strongly recommend following Nick on LinkedIn and subscribing to his Substack.

• Valiotti Data, the agency Nick runs as a fractional CDO for growing companies.

• LinkedIn, where most of this thinking shows up first.

• Substack, where Nick writes long-form content.

• Your Fractional CDO, the book written for executives building a data function for the first time.

—

Yordan

More on the Topic

• Where Your Data Team Sits Is a Strategic Decision

• Data Platforms Exist to Enforce Consistency, Not Enable Insight

• The Operating System Every Data Engineering Leader Needs

I was in my early twenties, still in university university, and responsible for a handful of client projects at my first real job. I had been coding since high school and I thought I knew what good work looked like.

I added key bindings to a jov board website. The client ran a small recruitment agency. Nobody on their team had asked for it, and nobody ever used it but I spent a week on it.

Projects ran late, budgets inflated, and our clients got angry. My boss, instead of firing me, handed me a small stack of books. In one of these books, The Pragmatic Programmer, I read about a concept I had never considered: shipping code that is good enough.

That idea rewired how I work. It changed how I ship, how I grow, and how I think about what it means to be a good engineer. I have spent years since then watching talented senior engineers repeat the same pattern I had at 21, and lose the same things I lost: time, trust, and the opportunity to work on harder problems.

Perfectionism Is Not About Bugs

You should never deliver buggy work intentionally. That is not the argument here, and I want to be clear about it before going further.

Most of the work engineers do past the point of “done“ falls into one of two buckets.

The first is rough edges. All sorts of imperfections that exist in the product but that no real user will ever notice or care about fall in this bucket. Think about a column name that is slightly inconsistent with the naming convention elsewhere, or query that runs in 800ms when it could run in 200ms, on a report that gets opened twice a week.

The second bucket is design preferences. These are things you want differently that your stakeholder never asked for, like a cleaner folder structure in the repo, a more elegant solution to a problem the current solution already solves.

The Distinction That Changes Everything

Rough edges and design preferences get presented as quality standards. That is the trap. An engineer tells themselves they are maintaining high standards when they are spending an entile afternoon on a naming convention nobody will audit.

Intentional restraint is a design decision. Choosing not to smooth a rough edge because no user will ever hit it is not laziness, and choosing not to build the abstraction because the requirement does not exist yet is not cutting corners. These are scoping decisions, and good engineers make them on purpose.

The engineers who struggle to stop are not the ones with low standards. They are the ones who have never separated “complete in my head“ from “complete for the purpose it serves“.

Perfectionism Costs More Than You Think

Shipping on time builds something you do not think about in the moment: a track record of reliability.

Your stakeholder remembers two things about you:

• whether your work was good

• whether you delivered when you said you would.

The second one compounds faster than the first.

When you ship clean, scoped work on time, you get the next project. And, you get pulled into the room where the decision is being made, because someone already knows you can be trusted to close things out.

That is the return on shipping. It is not glamorous, but it is how careers actually move.

The Week You Think You’re Investing

When you spend an extra week polishing past the agreed scope you lose:

• the relationship capital that week would have built with a different project

• the harder problem waiting on the other side of this one closing

• the visibility that comes from being someone who finishes

The cost disappears into the background noise of a slow quarter, and you never connect it to the week you spent on a naming convention nobody audited.

What You Actually Buy With That Extra Week

Polish on a finished product does not teach you anything new. It does not stretch your judgment or expose you to problems you have not solved before. It has the texture of diligence without the substance of it.

Perfectionism Is an Identity Problem

Here is what I have never heard a perfectionist engineer say: “I know this is done, I just cannot stop“.

They do not say it because they do not believe it. Every rough edge they are smoothing feels critical, and every design preference they are implementing feels like a quality standard.

The rationalization runs underneath the decision, and it sounds like judgment.

Where the Identity Comes From

At some point early in your career, you built an equation: good engineer equals engineer who finishes everything.

You saw thoroughness rewarded. You also got praised for catching edge cases, for going deeper than required, for the extra mile. The identity formed around completion.

That equation made sense then. In the early stages of a career, thoroughness is how you signal competence. You do not have a track record yet, so you demonstrate care through the detail of your work.

The problem is that the equation does not update. Ten years later, you are still running the same calculation, in a context where finishing the right things matters more than finishing everything.

What the Research Says

Psychologists separate perfectionism into two types.

Adaptive perfectionism: high standards, strong execution, healthy relationship with finishing.

Maladaptive perfectionism: the same high standards, but fused with avoidance, procrastination, and an inability to call something done.

The difference is what happens in your head when the work is good enough but not perfect.

A 2021 meta-analysis found that adaptive perfectionism is associated with better performance and wellbeing, while maladaptive perfec tionism is associated with worse wellbeing, more procrastination, and avoidant coping.

Most engineers I know with this pattern are maladaptive perfectionists. I was one, too.

The Gap Between Knowing and Shipping

There is a moment every over-polishing engineer knows but rarely names. The product works. The stakeholder is happy with what they have seen. The agreed scope is complete. And you are still in the code.

You know you are done. You stay anyway.

What fills that gap is the discomfort of letting something exist in the world that is not complete in your head. Shipping means accepting that the version in production will always be slightly less than the version you imagined.

For an engineer whose identity is built around completeness, that feels like becoming a lesser version of yourself.

That is the thing a productivity tip cannot fix. You need a different answer to the question of what a good engineer actually is.

The Stakeholder Already Told You It’s Done

Most engineers treat “done“ as a technical state. The data product reaches some internal threshold of completeness and then it ships. The problem with that definition is that you set the threshold, and you will keep moving it.

Done is an agreement.

You and your stakeholder defined the scope before the work started. They tell you what they need, and you build it. At some point in the review process, the nature of their feedback changes.

Early feedback sounds like “can we change how this filter works“ or “the numbers on this chart do not match what I expected“. That is active problem-solving.

Then the feedback shifts:

• “This looks great, when can we roll it out?“

• “When I can get access to this?”

• “Can the we also give access to my team?”

This shows you that your stakeholder has stopped looking for gaps and started thinking about using the product. The agreement has been reached.

The Closing Question

Questions like “is there anything you’d like to improve“ or “are there any other features you’d find useful invite scope expansion and put you back at the beginning of the loop. Stop asking them.

Before you hand anything over, ask one question: is there anything critical missing?

The word critical does the work. It filters out preferences and surfaces only the things that would prevent the product from serving its purpose.

If the answer is no, you are done. Write it down, send it in a message, create a paper trail. That record is for you as much as for them.

When You Talk Yourself Out of the Signal

• “They said it looks great, but they have not stress-tested it yet.“

• “The feedback was positive, but that was on the easy use case.“

• “I should add one more thing before it goes live, so the first impression is better.“

Each of these sounds reasonable. None of them are about the product. They are about the discomfort of letting go, but the stakeholder told you it is done.

Bugs, Rough Edges, and Design Preferences Are Not the Same Thing

The “one more thing“ feeling always presents itself as necessary. That is what makes it hard to dismiss, and that is exactly why you need a framework for categorising what you are looking at before you act on it.

Bugs

A bug prevents the product from doing what it was agreed to do:

• A pipeline that drops rows under certain conditions

• A report that breaks when the date range crosses a month boundary

• A metric that calculates correctly in the dev environment and incorrectly in production.

Defects are non-negotiable. Fix them before you ship, and if you find them after you ship, fix them immediately. This is the category that justifies stopping the clock.

Rough Edges

A rough edge is an imperfection that exists outside any real user’s path:

• A column name that deviates from the naming convention in a table nobody joins to directly

• A query that runs in 900ms when it could run in 150ms, on a dashboard opened twice a week by one analyst

• A tooltip with slightly imprecise wording on a filter that experienced users skip entirely.

Rough edges are invisible in practice. Leaving them is a scoping decision made on purpose, based on actual usage.

“Ugly by design“ is a legitimate engineering position. It means you looked at the imperfection, assessed its real-world impact, and decided the cost of fixing it outweighs the value.

Design Preferences

Design preferences are the expensive category, because they are the hardest to recognise in the moment.

A design preference is something you want differently that your stakeholder never asked for:

• A cleaner abstraction that would make future changes easier, for a future with no confirmed date

• A folder structure that better reflects your mental model of the system

• A more elegant solution to a problem the current solution already solves.

The test is one question: did anyone ask for this, or does it exist because it is incomplete in your head?

If the answer is the second one, you are looking at a design preference.

The Post-Launch Contract

Shipping is not abandonment. Your stakeholder needs to know what happens after the product goes live, and that conversation is your responsibility to initiate.

Most engineers skip it. The product ships, the Slack message goes out, and everyone moves on with an implicit assumption that the engineer is available for whatever comes next. That assumption becomes a maintenance burden with no defined edges, and it will pull you back into the product indefinitely.

Define the edges before you ship.

What You Commit To

Two things:

First, if something breaks in a way that prevents the product from doing its job, you fix it. A bug that surfaces post-launch is still a bug.

Second, you will check in proactively after two weeks to ask whether the product is working for them and whether anything critical is missing.

That check-in matters more than most engineers realise. It signals that you care about the outcome, keeps the relationship warm, and gives your stakeholder a defined moment to surface real issues. It also gives you closure.

What You Do Not Commit To

Everything outside those two things goes into a backlog. Feature requests, aesthetic preferences, and optimisations that would be nice to have are legitimate inputs for a future iteration, and you should capture them.

They are worth doing if the product sees real usage and the stakeholder comes back with them after living with the tool for a while.

A short message after launch that outlines what you are monitoring, when you will check in, and how to flag something urgent. Two paragraphs are enough.

The goal is a shared understanding, not a formal document, slides or any other corporate bullshit.

Why This Protects the Relationship

A stakeholder who knows what to expect from you after launch trusts you more. The engineer who disappears after shipping and the engineer who commits to everything indefinitely both damage the relationship in different ways.

Defining the post-launch contract is what makes you someone worth working with on the next hard problem.

You Will Know If It Works

Two weeks after launch, you have access to something more reliable than your own anxiety: data.

Data products leave traces:

• Every query run against your pipeline shows up in query history

• Every dashboard opened appears in your BI tool’s usage logs

• Every tab viewed in a shared Google Sheet is recorded in the Activity Dashboard

You do not need your stakeholder to tell you whether the product is being used, because you can look.

This matters because the fear that drives post-launch anxiety is usually vague:

• “What if something is wrong?“

• “What if they are not using it?“

• “What if I missed something?“

These are discomfort dressed up as diligence. The data answers them directly, and the answer is almost always that the product is running, people are using it, and nothing is on fire.

What to Look For

Query history tells you whether the pipeline is being hit and at what frequency. A product that was supposed to run daily and shows seven consecutive days of successful runs is a product that works

If the usage logs show nothing, that is information too. A product nobody uses is a product worth a conversation with your stakeholder, and that conversation is more useful than another week of polish would have been.

The Check-In

At the two-week mark, send a short message. Ask two things: is the product working for them, and is there anything critical missing.

The same question you asked before launch, asked again with two weeks of real usage behind it.

Most of the time the answer is positive. Occasionally something surfaces that genuinely needed addressing, and you address it. Either way, you now have a closed loop.

The product is no longer an open question sitting in the back of your head. You can move on to the next problem without carrying it with you.

Final Thoughts

The way you write code has changed. AI writes a significant portion of it now, which means the craft argument for over-polishing has collapsed.

Spending an extra week perfecting namming conventions and folder structures in a codebase where most of the code was generated is a poor use of the one thing AI cannot replicate: your judgment about what to build and whether it worked.

Your value as an engineer was always moving in this direction, but AI accelerated it. The engineer who ships, checks whether the outcome landed, and moves on to the next hard problem is worth more than the engineer who produces immaculate code that took three weeks longer than agreed.

Stop asking whether the code is beautiful. Start asking whether the outcome is what you and your stakeholder agreed on. That is the question that maps to real value, and it is the question that will matter more with every year that passes.

Ship it. Check if it works. Move on.

—

Yordan

More on the Topic

• Stop Starting Data Projects

• The Customer Service Mindset Is The Fastest Way To Destroy Your Data Team

• How ruthless prioritization got me a 40% raise and a head of data title I didn’t even want

As a hiring manager and coach, I have interviewed hundreds of candidates over the years. The pattern is almost identical every time.

The interviewer asks “Tell me about yourself“. You take a breath and start at the beginning.

You walk through your education, then your first role, then every move after that, including the reorg nobody asked about and the project you are clearly proud of. 5 minutes into your monologue, the interviewer is nodding, but their eyes have gone somewhere else.

I get it, you have a lot to say. The problem is that you are answering a different question than the one being asked.

“Tell me about yourself” is not an invitation to walk through your resume. It is the one moment in the entire interview where you hold the controls. You decide what they ask you next. Most candidates hand that back immediately and spend the next five or even ten minutes reading from a document the interviewer already read.

You will not do that.

They Already Read Your CV

The interviewer already read your resume. That’s why they called you for an interview.

They did not call you to hear it again. They called you because something on that one-pager was interesting enough to warrant a conversation. The resume did its job. Your job now is different.

Walking through it line by line does not add information. It repeats it, slower, with more words, to someone who is already familiar with the content. Worse, it signals that you do not know the difference between a document and a conversation.

You have not idea how to be effective communicator. And you don’t know what matters the most.

The candidates who impress in the first two minutes are not the ones with the longest history. Hiring managers remember candidates who show up knowing exactly what is relevant for this role and lead with that. Everything else can come out later, in the questions, where it lands with context.

You Only Have 90 Seconds

Spend no more than 90 seconds introducing yourself.

That limit is the difference between giving the interviewer something to grab onto and overwhelming them with everything at once. A short intro forces you to choose what matters. That choice is what signals seniority.

There is another reason to keep it tight: Interviewers follow up on what they heard last. Whatever you end your intro with is where the conversation goes next.

That means your 90 seconds are a steering mechanism. You are deciding, right now, which parts of your experience get explored in depth and which ones stay in the background.

Most candidates never realize this.

They treat the opener as a formality to get through. It is the opposite. It is the most leverage you have in the entire conversation.

How to Build Your 90-Second Intro

Read more

I started applying the same processes I use at work to my own career and life. Logging decisions and why I made them. Running retros on myself, tracking where I wanted to go and whether I was actually moving in that direction.

At some point I noticed I was getting better faster. Writing forced me to actually process the information around me.

That’s the discipline most engineers never apply to themselves. They track everything at work and nothing about their own career. Years of decisions, hard lessons, pivots all live in memory, which means most of it is already gone.

We are going to fix that today.

Your Brain Is a Terrible Data Store

You carry an enormous amount in your head at any given moment:

• Technical tradeoffs you’ve made

• Political context that shaped a decision

• Conversation that changed how you thought about a problem

• A pattern you noticed but never wrote down

All of it evaporates.

This is the thing nobody talks about at the senior level. You get better at your job, build better systems, and make faster decisions. But you have no record of any of it. No way to know whether you are actually growing or just staying busy.

The Feeling of Falling Behind Is a Data Problem

By not recording your own progress, you consistently underestimate how far you have come. The wins feel smaller in hindsight and the mistakes loom larger. Your brain rewrites the timeline to fit whatever story feels true in the moment.

You would never run a data platform with no observability or let a production system generate events you never capture.

Your career is generating signal every single day. So why you won’t capture it?

What Writing Actually Does

Most people treat writing as output, a way to communicate something they already know. That framing misses the point entirely.

Writing is how you think. The act of putting something into words forces your brain to resolve ambiguity it was happy to leave floating. You jsut can’t write too vague vague thoughts.

Subscribe now

The Pennebaker Experiment

In 1986, psychologists James Pennebaker and Sandra Beall ran one of the most replicated experiments in psychological research at Southern Methodist University. They split healthy undergraduate students into three groups and had each group write for 15 minutes a day across four consecutive days.

The first group wrote about boring topics like describing their dorm room, their shoes, or a tree outside the window. Pure surface-level observation with no emotional content whatsoever.

The second group wrote about a traumatic or stressful event from their lives, but with strict instructions: facts only, with no mention of how it felt or what it meant.

The third group wrote about both the facts and their deepest thoughts and feelings about the experience, the reasoning behind what happened, the emotional weight of it, and what it meant to them.

What the Data Showed

The differences in long-term physical and mental health outcomes across the three groups were significant. The first two groups showed minimal lasting benefit. The third group showed measurably better health outcomes months later.

You already know this distinction at work. A log entry that says “pipeline failed” is useless, but a post-mortem that captures what failed, why it failed, what assumptions were wrong, and what you would change is what actually improves the system.

The same principle applies to you.

At the senior level, the events worth processing are rarely technical. Instead, they are the messy, ambiguous, political moments that shape how you lead.

Writing about what happened is the easy part. But writing about why you made the call you made, what you were afraid of, and what you would do differently is where the actual learning happens.

How to Write

The blank page is where most people quit. The habit dies before it starts because writing “whatever comes to mind“ produces nothing useful and feels like a waste of time.

Proper structure fixes this. A small set of questions you return to consistently is all you need.

The Weekly Review

Once a week, write about three things:

• A decision you made and the reasoning behind it

• Something that frustrated you and what you think is underneath it

• Where you are relative to where you want to be

You need fifteen minutes at most. The goal is to get your thinking out of your head and into a place where you can actually look at it.

This is the layer that catches the things you would otherwise carry forward unresolved. Most engineers have three to five open loops in their head at any given time, a conversation that did not land, a call they second-guess, a relationship they are unsure about.

The weekly review is where those loops get closed, or at least named. Named problems are manageable, while unnamed ones accumulate.

The question that does the most work is the second one. Frustration points at something you care about, something that conflicts with how you think things should work, or something you have been avoiding. Writing about what is underneath it is where the actual insight lives.

The Event Log

Whenever something significant happens, write about it the same day. A hard conversation, a project that went sideways, a moment where you handled something better than you expected.

The detail fades faster than you think, and the reasoning fades even faster.

Two weeks after a difficult stakeholder conversation, you will remember the outcome but lose the texture of what you were thinking, what you were afraid of, and what you chose to say versus what you held back. That texture makes the entry useful when you read it six months later.

The event log is also where you catch your own patterns before they become problems. The same frustration appearing three times in four weeks is information. The same type of conversation going badly repeatedly is a skill gap worth addressing. It’s very hard to see any of that without a record.

The Career Retro

Once a quarter, zoom out. What did you ship, what did you avoid, where did you grow, where did you stall. This is the layer that makes the weekly entries useful because patterns only become visible when you read across time.

The career retro is the closest thing to running a proper retrospective on yourself. Most engineers run retros on their projects and never on their own trajectory. A quarterly review of your own writing surfaces the things your brain was too close to see in the moment.

Read back through the last three months of entries before you write the retro. Look for recurring themes, recurring frustrations, recurring avoidance. The goal is to walk away with one or two things you want to do differently in the next quarter, leaving the long list of resolutions behind.

Subscribe now

Write Like You Talk

Specificity beats polish every time. What happened, what you decided, why you made that call, what you would do differently. You are creating a record your future self can actually use, and that record is only useful if it is honest.

The trap most people fall into is writing for an imaginary audience. They tidy up their reasoning, smooth out the contradictions, and produce something that reads well but captures nothing real.

The entry that says “I approved the hire because I was under pressure and did not do the reference check properly“ is worth ten times more than one that says “we moved quickly on this one“.

Dictation changed how I write entirely. I dictate most of my entries now (the same way do with this article). Speaking removes the friction of the blank page and keeps the tone honest. You do not perform for a voice memo the way you perform for a text editor. The words come out closer to how you actually think.

Start Slow

Write when you have something to write about. The weekly review, the event log, the quarterly retro are all things you build toward, not things you set up on day one. Pick one thing that happened this week and write about it for ten minutes. That is the whole system for now.

The mistake most people make is treating the habit like a project. You design the perfect framework, pick the perfect tool, and quit two weeks later because the system is too heavy to maintain.

And here’s one more principle to mention: Consistency over completeness. A messy entry you actually wrote beats a perfect template you opened and closed. The value builds through repetition, and repetition only happens when the bar to start is low enough that you do it even on the days you do not feel like it.

With time, when you start enjoying it (because you will), set a system. I wrote about that on the Dear Self blog.

Why Email and AI Changed Everything

Paper journals have a beautiful aesthetic and a fatal flaw. You cannot search what you wrote. The insight you wrote down eighteen months ago stays buried unless you happen to find the right book and flip to the right page.

Most apps solve the search problem but introduce a different one. They add friction by locking your data into a format you do not control, and they tend to disappear or pivot into something you never asked for. You have probably abandoned at least one app and lost everything inside it.

Why Bullet Journaling Eventually Breaks Down

Bullet journaling worked well for me for years. The structure kept the habit alive and the format was fast enough to not feel like a chore.

However, Bullet journaling is optimised for capture, and capture alone produces the same result as the second group in Pennebaker’s experiment. You get the facts and skip the reflection. The entries become a log of events rather than a record of your thinking, and a log of events without reasoning is not much more useful than a calendar.

Why Email Works

Email is the one tool you will never abandon and never lose access to. There is no new interface to learn, no blank page anxiety, and no proprietary format trapping your entries. You open a compose window, write, and send it to yourself. The archive is searchable from day one.

Writing to yourself by email also changes the tone in a useful way. You are writing a message to your future self, and that framing naturally produces the kind of specificity and honesty that makes entries worth reading later.

It is close enough to a conversation that the words come out more naturally than they do in a dedicated journaling app.

Where AI Changes the Game

Search is the foundation, but AI is what turns a collection of entries into a system that actively works for you.

The difference is between a graveyard of notes and something that talks back:

• Proactive prompts surface the questions worth asking based on what you have already written.

• Reflections get surfaced back to you at the right moment, connecting something you wrote six months ago to something you are dealing with today.

• Patterns you would never spot manually become visible across hundreds of entries.

This is the layer that makes the whole practice qualitatively different from anything that existed five years ago. The ability to query valuable writing, reflect on it, and have it reflected back to you is what makes it a system.

That is why I built Dear Self.

I actually built it for myself, and after a few months I can say it is the best journaling experience I have ever had. I want to share that with you. It is free and if you are a paid Data Gibberish subscriber, DM me and I will get you your first year of premium features on me.

Final Thoughts

The best engineers I know learn faster than everyone else, because they are better at extracting signal from their own experience.

You have been generating that signal for years. Every hard decision, every difficult conversation, every project that went sideways and every one that worked better than expected, contains information about how you think, where you are strong, and where you have room to grow.

Writing is how you stop letting that information evaporate.

The engineers who compound fastest are the ones who treat their own career with the same rigour they bring to their systems. They log, they retro, they reflect. They build a record of their own reasoning and they actually use it.

You already know how to do this work. You do it every day for your team, your stakeholders, your infrastructure. The only thing left is to point the same discipline at yourself.

Start this week with one entry in less than ten minutes. Write about something that is still sitting in your head from the past few days and what you actually think about it.

That is the whole system for now.

—

Yordan

PS: The medium doesn’t matter and you don’t need to use my service. But if you do, I’d love to hear your constructive feedback. Start by writing an email to me@dearself.ai.

More on the Topic

• How to Plan a Year of Content as a Data Engineering Leader

• This Engineer Teaches Their Audience How To Think The Right Way

• Stop Feeling Behind: How Journaling Heals Productivity Shame

There is a moment you hit as a senior engineer and almost nobody warns you about. The learning that used to feel automatic starts to slow down. The gaps between “I learned something real today“ get longer. Weeks, sometimes months.

Most people read this as a personal failure. You assume you have gotten lazy, or comfortable, or that the hunger that made you good is fading. You start chasing certifications or side projects to feel the velocity again.

The velocity does not come back. And chasing it is the wrong move.

What actually happened is that your horizon changed. When you were building things, feedback arrived fast. You wrote the code, the tests ran, the pipeline moved. The learning was continuous because the signals were continuous.

When you operate, you make a decision in January and find out in August whether it was right. A different kind of learning, with a different rhythm, and almost nothing prepares you for it.

This piece is about how to keep growing when the old mechanisms stop working, because you outgrew them.

The Horizon Shifted

As an engineer, your feedback loop is tight. A PR merges, a pipeline runs, a test passes. You know within hours whether you did the right thing. That tightness is what makes learning feel fast. You get a signal, adjust, and move.

When your role grow, the signals take months:

• You change a hiring process and find out a year later whether you hired well.

• You restructure how your team runs projects and see the results two quarters down the line.

• You invest in a stakeholder relationship and the payoff arrives slowly and quietly, instead of in a notification or a merge.

This is the job now.

If you still measure yourself by what you personally shipped this week, you will always feel behind. Not because you are behind, but because you are using the wrong ruler.

The New Ruler

So what do you measure yourself by instead?

Three things:

• The team ships.

• Stakeholders are happy.

• The people on your team feel like they grow from working with you.

That is it. Those three are your new ruler. If all three are true, you are doing your job well. If any of them are consistently off, something needs your attention, regardless of how much you personally produce.

This feels uncomfortable at first, especially if you are used to clearing a todo list and knowing exactly where you stand.

These three things are slower to read and harder to fake. A stakeholder who tells you everything is fine in the meeting and escalates to your manager the next day is a signal you missed something. A team member who looks busy but stops bringing you hard problems is a signal too.

You learn to read these, but it takes time.

It also feels, like you are doing less at first. You are not shipping the pipelines, not writing the models, and not closing the tickets. The work is less visible and the feedback is slower. You are running a system instead of building a component. That is harder, and the leverage is orders of magnitude larger.

Own it as seriously as you owned every PR you ever merged. Even more.

Block the Time

All of this happens by design.

If you do not protect time for learning, the week fills up and learning is always the thing that moves to next week. Next week becomes next month. A year passes and you realise you have been reacting to the job rather than growing inside it.

Block two to three hours a week.

That’s non-negotiable. Put it in your calendar as a recurring block and treat it with the same respect you give your most important meetings. Because that is a meeting with your future.

You don’t need to do it in a single block. Thirty minutes at the end of each day works just as well. What matters is that it is regular and protected. The moment you start treating it as optional, it disappears.

What you do with the time doesn’t matter that much. Take a course, read a book, build something small, or even talk with a peer at another company. The medium is less important than the habit. The point is to keep a part of your attention pointed outward, at something that is not your current backlog, your current team, your current problems.

Subscribe now

One thing worth noting:

Learning at this level is not mostly about new tools or frameworks.

A new tool is learnable in a day when you have enough context. What takes real time is developing judgment:

• How to read an organisation.

• How to run a hard conversation.

• How to make a call with sixty percent of the information you want.

You build that slowly, through exposure and reflection. Tutorials won’t help you here.

The two to three hours a week is where that reflection happens.

The Learning Workspace

Learning without a strong manager who can really teach you the skills you need is hard. So, I packed some of my best tools to share them with you so you can design your career growth and become the professional you want to.

Read more

Presenting Peter Mukherjee

Peter built a London business from scratch in 1992, grew it into an international franchise network with a public listing, and lost most of it to the 2008 crash. He reinvented himself as a professional architectural photographer and worked at it for fifteen years before retiring in 2023 to write full-time.

Subscribe to Peter's Publication

I’ve known for about an year now, and he’s one of the wisest people I’ve ever talked to.

The whole time he was talking during our conversation, I kept hearing the same diagnosis for why most senior engineers hit a ceiling.

You are managing your career like an employee.

The people who break through manage it like a business. They use a different vocabulary, run a different operating system, and make different decisions. None of it requires another certification, and all of it requires a mindset shift most senior engineers never make.

Forced Pivots Surface The Rest Of You

The default for a generation was 30-40 years at one employer, a stable pension, and a CV that looked sensible. The world does not work that way anymore. Multiple careers across one working life is now the norm.

The hardest version is the forced pivot. Income stops, status stops, and everything you built is on hold while you scramble to establish something new.

Getting through it requires confidence in your own abilities, because most people are more capable than they give themselves credit for, and the work is digging deep, finding the talents you have, and pulling yourself through.

Resilience is built mostly through failure. There is no shortcut. You learn the most during the times you fail, because failure forces you to dig in and recover. The worst response is going negative, looking backwards, sliding into “what have I done“. That posture stops the recovery before it starts.

The deeper insight is that most people never find out what their real talents are. They pick the first thing they think they are good at and develop only that.

There might be tens of thousands of people with that level of athletic talent who never find out they run, because they never explored their full range. When things go wrong, you are forced to ask “what else am I good at?“ That question, under pressure, surfaces the talents the comfortable version of you never bothered to look for.

The Old Leadership Model Is Finished

The safe pair of hands is done. Steady the ship, grow dividends 5-8% a year, stay around for ten years. That mindset takes the business backwards now, because the world moves too fast for it. Standing still is moving backwards.

The replacement is the transformational leader. Transformation involves risk-taking, which only works if the culture supports it, which means giving your people permission to take risks too, without making them fear for their jobs when they get something wrong.

Apple and Google run open plans where managers sit among their people. Senior managers might have an office, but it is glass-sided so people see them and they see their people. Any transformational leader has to be able to carry their people through change, and you only carry them properly if you are visibly involved with them.

A successful US business leader who talked about being vulnerable in front of her people in the 90s was told “that’s because you’re a woman“. Now every leadership conversation centres on empathy and vulnerability. COVID accelerated it and AI will accelerate it further.

Where Leaders Lose People

The biggest source of failure is arrogance and ego, and the biggest organisational consequence is losing the people you invested time and money into developing. You will always be outbid on salary. There is always someone offering more money.

But the one thing people put ahead of the bigger paycheque is enjoying where they work. The manager who is arrogant, ego-driven, and disconnected from their team creates an environment people are eager to leave, and that manager loses good people on a clock.

Humility Is The Hardest Lesson

The kind of humility forced on you by having to let go of people who built the business with you. Telling someone who has been with you for 15 years that the business is closing is one of the hardest things a leader does.

There are two ways to handle it. The first is matter-of-fact, because that is what organisations do. The second is to treat the moment with the weight it carries for the person on the other side of the desk, with empathy and active care for what happens to them next, including helping them frame what they learned.

The relationships survive the second version.

The same principle runs through every other team interaction. Make people feel included, valued, recognised.

A specific mechanic that works: team-level bonuses where the team itself decides who the top contributors are. Everyone shares in the reward, but the team picks the largest pieces. It does not have to be financial. A day out, a big dinner, share options. The point is structuring recognition so it incentivises working together instead of competing.

The 51/100 Rule

The mythology of the successful CEO making lots of right decisions is wrong. Every CEO is making decisions every day, and some they get wrong, sometimes catastrophically.

The batting average, from Ursula Burns (former CEO of Xerox, started as an intern): out of every 100 decisions, you make 51 good ones and 49 bad ones. The job is making sure the 49 bad ones are not too bad.

You afford one or two howlers, but the rest need to be recoverable. If you have an immaculate record of getting everything right, you are not taking enough risk, and your career is moving backwards while you congratulate yourself for being right.

The mechanic underneath is reversibility. Good risk-taking is calculated, not suicidal. Sort every decision into reversible and irreversible. The reversible ones get volume. The irreversible ones get contingency planning.

The downside in the reversible bucket is embarrassment and a week of recovery. The downside in the irreversible bucket is structural and measured in years.

Most senior engineers treat every decision as if it were in the irreversible bucket. They spend two weeks deciding whether to submit a conference talk. They draft a LinkedIn post and never publish it. They workshop an internal proposal until the moment to make it has passed. The cost is invisible because nothing went wrong. Nothing happened at all.

Curiosity Is The Response To Uncertainty

For individuals, the best response to uncertainty is curiosity. Nobody becomes successful without it. Learning does not stop when you finish university. Curious people are the ones who do something big.

Invest in emotional skills. Degrees matter less. The capability to communicate, manage people, be visionary, be inspirational, and bring creativity is what employers will look for. Those are the skills that differentiate you in three years’ time, and they are the ones currently at risk from the AI-as-co-pilot trap.

The AI Entropy Warning

Entropy is a real thing. Systems decay toward their lowest energy state when nothing pushes back. Brains are a system. The muscle you stop using is the one that atrophies. People who used their brains for the work are now using AI for the work. The output looks fine. The brain underneath is going to sleep.

The fix is using the tools while keeping the thinking. Let AI draft, then edit by hand, let it summarise, then read the source or let it suggest options, then make the decision yourself and write down why.

The skills that cannot be outsourced are the ones the technical job market will pay for in three years, and right now the industry is volunteering for neglect.

Load The Dice Yourself

Every successful person credits luck. A specific phone call, a meeting, a door that opened at the right moment. A single phone call was the pivot for a multi-billion-dollar career, where the answer was 50-50 right up until it was given. The mistake is to hear that and conclude luck is random.

Luck is a numbers game with a rigged distribution. Go to one networking event a year and talk to three people you already know, you have three lottery tickets. Go to five events and talk to ten new people each, you have fifty. Same person, same skills, sixteen times the surface area for a lucky break.

For data engineers, the equivalent is volume of visible work. Internal documents that travel. Brown bags and lightning talks. External writing. Conference submissions. Every one of them is in the reversible bucket. The hit rate on any individual piece does not matter. The volume across all of them does.

The wider pattern shows up in every successful career: luck plus passion. Everyone talks about their passion for what they do, and there is a direct correlation between success and finding it. “Follow your passion” only works if you have one. If you have not found one, the work is exploration. Keep looking until you find the thing you love. Until then, every step is a stepping stone.

The cost of getting this wrong is what makes the rest of it urgent. The greatest risk is that you never find success and fulfilment because you stuck with something you did not love.

Sir Clive Woodward, the England rugby head coach who won the 2003 World Cup, put it this way:

The worst thing that happens in life is getting older and looking back thinking “I wish I’d taken a chance on that“

Treat Yourself As A Business Inside The Organisation

Change your perception of your role. You are no longer “the programming manager.” Look at yourself like a one-person business inside the organisation you work for. Promote yourself. Acquire the skills. Develop yourself the way a business develops.

The language change shows it:

Saying “I am an admin manager, I have been doing this for seven years“ tells the listener nothing.

Saying “in the last seven years I have organised X, I have done Y, these are the successes, these are my strengths, this is where I add value, these are the things I will carry forward to help your organisation“ is a different conversation.

That is how you communicate inside the organisation. That is how you describe yourself outside it.

Life is like snakes and ladders. You roll the dice, you move forward. Lucky, you climb a ladder. Unlucky, you slide down a snake. The mistake is to think your future is dependent on the roll of the dice. Most people are more in control of their destiny than they think.

Too many expect someone else to design their career. The alternative is to go out, meet people, decide your own projects, decide your own career.

Final Thoughts

The senior engineer ceiling is not built out of skill gaps. It is built out of an operating model imported from the codebase: optimise for being right, ration the decisions, stay invisible until you are sure. That model breaks the moment the job stops being primarily technical, which is the moment the ceiling appears.

The people on the other side run a different model. They aim for 51 out of 100, sort decisions before they spend time on them, and surround themselves with people better than they are.

They take humility seriously and ego personally. They keep their brains on while the industry switches theirs off. They describe themselves like a business that knows what it sells.

None of it requires a certification. All of it requires the mindset shift most senior engineers never make.

Where To Find Peter

Peter writes on Substack. The book is available in hardback, paperback, Kindle, and audiobook, the audiobook read with 12 different voices, one per contributor.

—

Yordan

Subscribe now

A colleague of mine was one of the most talented engineers I have ever worked with. Sharp, fast, technically thorough. When a stakeholder came to him with a project, he would disappear for six weeks and come back with something impressive.

The stakeholder would look at it, go quiet, and say something like “This is not quite what I had in mind“.

And every single time, his response was the same. “They are stupid. I know what they need. They should use it“.

The project would either get quietly cancelled, or the stakeholder would ask someone else to redo it. I watched this happen twice. Same engineer, same pattern, different projects.

The thing is, he was not wrong about the technical side. The code was clean, the logic was sound, and the solution was, in his view, objectively correct.

He just never learned the business process. He never asked the stakeholder to show him how they actually worked. He heard the request, translated it into a technical problem, and solved that problem in isolation. Then he was genuinely confused why nobody wanted what he built.

The truth is that nobody gets fired for writing good code. They just stop getting the interesting projects. The stakeholder quietly routes around them. And they never quite understand why.

It is a process problem. And process can be fixed.

Here is the exact process I use, from the moment a vague Slack message lands to the moment I close a project out.

The Scripts That Keep Projects on Track

Know exactly what to type at every stage, from the first vague request to closing the project out.

Download Now

Reply to the Slack Message Before You Touch a Keyboard

The vague request is not a problem at all. It is the starting point.

Most engineers read a message like “Hey, can we get a dashboard for the sales team?“ and immediately start thinking about the data model. That is the mistake.

Instead of a spec, the request is an opening. Your job at this moment is to open a conversation without making it feel like a process.

Do not schedule a meeting. Do not send a list of questions. Send one message back.

Here is what I type:

This kind of a message signals you are taking it seriously without turning it into a project before you know what the project is.

When the call happens, most engineers ask “what do you want the dashboard to show?“

But that is the wrong question. It gets you a list of metrics that may have nothing to do with how the stakeholder actually works. Instead, ask them to show you their current process.

Here are the three questions I ask on every first call:

• Can you walk me through how you’re doing this currently?

• What does a good week look like for you, in terms of this data?

• If this was done and working perfectly, what would you do differently?

The third question is the important one. It forces them to think about behaviour, not features. The answer tells you what the project is actually for.

The Definition of Done Is a Contract

Before you leave that call, you need one sentence that both of you agree on.

Here is how I get it:

Let me make sure I’ve got this right. What we’re building is X, and we’ll know it’s done when Y. Does that sound right to you?

If they say yes, you have your definition of done. If they hesitate, you have just saved yourself six weeks of building the wrong thing. Keep asking until the sentence lands cleanly.

Write it down and send it in Slack after the call. Something like:

Ideally, you create a channel for that project. Now both of you, and everybody interested in that project have it in writing.

Know Who Can Kill This Before You Build Anything

Before you write a single line of code, you need to know two things:

• who can kill this project

• who will fight for it

Most engineers talk to the person who sent the Slack message and assume that is the whole picture. Sometimes it is.

Often there is a manager who needs to sign off, a team that will be affected, or a senior stakeholder with strong opinions who has not been in the conversation yet. Finding out about them after you have built something is expensive.

You do not need a stakeholder mapping workshop, but you need to ask one question on that first call:

Is there anyone else I should loop in, or anyone whose sign-off we’ll need before this goes live?

That one question will surface 90% of what you need to know. From the answer, you are looking for three types of people.

• The decision maker: The person who can say yes or no to the final output. Everything you build needs to satisfy them, even if they are not the one who sent the request.

• The champion: The person who wants this to exist and will push for it internally. Usually the person you are already talking to. Keep them close.

• The informed: People who need to know this is happening but have no power over the outcome. A quick message every now and then is enough.

Once you know who you are dealing with, agree on delivery stages. You just need two or three rough checkpoints and an expected delivery date. Something you can type in Slack in four lines.

Here is what I send:

That message does three things. It sets expectations, commits you to a timeline, and signals that you will be showing progress before the end. Your stakeholder now knows they will not be waiting in silence for six weeks.

The RACI Nobody Will Ever See

You do not need to show anyone a RACI diagram. But you need to have done the thinking.

Take five minutes, on your own, after the first call. Write down who is responsible for building this, who is accountable for the outcome, who needs to be consulted, and who needs to be kept informed. No template, no meeting, and definitely no slides.

When something goes wrong mid-project, and something always does, you will know exactly who to call and what to ask. That clarity is worth more than any amount of documentation.

Build the MVP in Under Five Days

McKinsey research on 5,400 IT projects found that the average large project delivers 56% less value than predicted. The number one reason is building in the wrong direction for too long before anyone checks. The fix is embarrassingly simple:

The rule I follow is five working days from the moment I actually start building. Not from the first call. From the day I sit down and open Neovim (and mostly Claude Code nowadays).

It will have hardcoded values, missing edge cases, and placeholder logic. That is fine. The goal of the MVP is to confirm you are moving in the right direction before you invest more time going in the wrong one.

Here is how I introduce it:

That framing removes the pressure from both sides. The stakeholder is helping you steer.

Don’t look for the “I love it” responses. This doesn’t matter. Listen for whether they are reacting to the right problem.

If their feedback is about polish, you are on track. If their feedback reveals a fundamental misunderstanding about what they needed, you have just found that out in week one instead of week six.

How to Handle “This Isn’t Quite What I Had in Mind”

This is the moment most engineers dread. It does not have to be.

If the stakeholder sees the MVP and tells you it is not what they expected, that is the process working. Here is what I say:

That’s really useful, thank you. Can you explain me what you had in mind? A rough sketch or an example would help me get there faster.

Then you go back, adjust, and show them again. The loop is fast because you have not built much yet.

The other outcome is rarer but worth being ready for.

Sometimes a stakeholder sees the MVP and realises they do not need the project at all. Seeing something concrete makes them understand that what they actually wanted was a process change, or a conversation with another team, or something that has nothing to do with data.

When that happens, you have saved everyone a significant amount of time. Close it and move on.

Show Progress Every Three Days or Lose the Project

Once the MVP is signed off and you are building toward the real thing, most engineers go quiet. They are heads down, making progress, and they assume the stakeholder knows that.

The stakeholder does not know that.

• After three or four days of silence, they start wondering.

• After a week, they are telling their manager the project might be stalled.

• After two weeks, they have mentally moved on and started looking for alternatives.

You are still building, but the project is already dying.

The rule I follow is simple.

No more than three days without sending something.

It does not need to be a status report, or a meeting. It can be one Slack message.

Here is what I send msot times:

This takes less than 30 seconds to write. It resets the clock on stakeholder anxiety and signals that the project is moving.

Over time, the stakeholder starts getting invested. They follow the progress, and start telling colleagues about it. By the time you deliver, they are already its biggest advocate.

That is not a small thing. A stakeholder who is invested in a project will push for its success, defend it in conversations you are not part of, and use it after delivery.

That kind of adoption happens only if you keep them close while you are building.

When Your Stakeholder Goes Ghost

Sometimes the silence goes the other way.

You send an update and hear nothing back. You share a progress demo and get no response. You follow up once and still nothing. In my course, I call these stakeholders Ghosts.

Silence at different stages means different things:

• Early in the project, no response usually means competing priorities. They are busy, the project is not urgent, and they will come back.

• Mid-project silence is a warning sign. It often means they have lost confidence, or something has changed internally that they have not told you about.

• Post-MVP silence, after they have seen something concrete and gone quiet, almost always means they did not know how to say “this is not what I wanted“.

In all three cases, the move is the same. One message that forces a yes or no without making it feel like a confrontation.

Here is what I send:

That message gives them an easy out if something has changed, and it gives you the information you need to decide whether to keep building.

Most of the time, you will get a response within an hour. Either they apologise for being slow and re-engage, or they tell you something has shifted. Both outcomes are better than building in silence for another two weeks.

Plus, having this in the dedicated channel means you have a documented go/no-go decision.

Stop When It’s Good Enough

At some point, the project is done. Not perfect, but done.

This is harder than it sounds. Engineers are wired to improve things. You finish the main deliverable and you start noticing rough edges:

• The query could be faster

• The dashboard could be cleaner

• The logic could handle three more edge cases

So you keep going, and your stakeholder is sitting there wondering why they still do not have the thing you promised them.

The signal that you have hit good enough comes from the stakeholder. When their feedback shifts from “can we change X“ to “this is great, when can we roll it out“, you are done.

Here is how I close a project:

I think we’re at a good place. Here’s what’s been delivered, here’s how to use it, and here’s who to contact if something breaks. Is there anything critical missing before we call this done?

That last question matters. It gives them one final chance to raise something real. If they say no, the project is closed. If they raise something small, you make a call on whether it belongs in this version or the next one.

And this brings you to scope creep. Every project attracts it. The stakeholder sees what you built, gets excited, and starts asking for more. New reports, new dimensions, new use cases.

These are all reasonable requests, and all belonging in a different conversation. I have a full article on how to handle that without losing the relationship, linked at the end of this one.

The other exit is the kill decision. Sometimes the right call is stopping entirely.

Final Thoughts

The engineers I have seen succeed long term are not always the most technically gifted. But they are the ones whose work gets used. Their dashboards are open every week, their pipelines run without anyone filing a ticket, and their stakeholders mention them by name in meetings they are not part of.

That happens because they treat the non-technical part of the job with the same rigour they bring to the technical part:

• They ask the right questions before they build anything

• They show their work early and often

• They keep people close instead of going silent

The process in this article is the minimum required to make sure your technical work actually lands. Most of it happens in Slack, and none of it requires a bureaucracy.

Even a project that looks doomed can be turned around with the right moves at the right time. I have seen it.

The engineer who gets that call, who steps in late and still manages to deliver something the stakeholder trusts, is never the one who worked the hardest. It is always the one who communicated the best.

—

Yordan

PS: Paid members consistently share they got promoted or praised because they apply my guides. When ready, upgrade here.

More on the Topic

• The Customer Service Mindset Is The Fastest Way To Destroy Your Data Team

• Workshop: Cross-team data project management with RACI

• How I Scope Minimum Viable Data Products That Prove Value Fast

• Why trust disappears faster than servers crash when you hide bad news

• Crush Scope Creep: Data Engineer’s Blueprint for Bulletproof Data Product Plans

A year and a half later, they quit. Frustrated, and feeling like the company did not value them.

In fact, The company did value them. They valued their execution. What they never got was somebody to listen to their ideas and judgment. Because this person never voiced any of it.

They worked on everyone else’s ideas, delivered brilliantly, and waited for recognition that was never going to come through technical performance alone.

I have seen this story play out dozens of times. And you probably recognise it too.

The Problem Is Not Performance

Brilliant engineers get great performance review, but no promotions. They are never picked for the exciting projects.

You do what you are told, you do it well, and somehow the people who get the opportunities are not the ones who write the best code.

And this is an onboarding problem.

The first 30 to 90 days in a new role set a reputation that is almost impossible to shake. The team forms an opinion of you before you have a chance to demonstrate your full capability.

That opinion determines which projects you get offered, whether your proposals get heard, and whether your manager sees you as someone to develop or someone to assign tasks to.

Most engineers get this period exactly backwards.

They arrive with strong opinions, a full toolkit, and a mental list of everything they would do differently. And then they start doing it. Immediately. Without asking why anything is the way it is.

One person I worked with joined as a new Head of Data and kicked off a full warehouse migration in their first weeks. The context they were missing could have told them why that decision was catastrophic for this specific organisation. They were gone within a few months.

Subscribe now

Three Habits That Change How You Land

This is not a framework. These are behaviours I have observed from every person I have coached or worked with who had a strong first year. They do not require a senior title to execute. They work whether you are joining a new company, a new team, or picking up a new stakeholder group at your current org.

Habit 1: Empty the Glass Before You Fill It

There is an old story about a student who goes to a master and asks to be taught. The student has already read everything. They already have opinions. The master pours tea into their cup until it overflows, then says: you cannot fill a glass that is already full.

When you walk into a new data environment, you are going to see:

• broken dbt lineage

• tests that take 12 minutes

• an #analytics-questions Slack channel full of questions whose answers exist in a Wiki nobody reads

• an infrastructure that looks like nobody ever cared about it

Your instinct is to fix it. But that instinct is wrong.

Why do those tests take 12 minutes? One engineer I worked with assumed it was negligence. After asking, they found out those tests were keeping watch on a set of records in the most critical table in the warehouse because of a recurring data integrity issue nobody had yet figured out how to solve properly.

Their first plan was to filter the lookback window to 24 hours. The right answer, after understanding the context, was a full remodelling of the upstream logic.

One of those solutions fixes the problem and one of them creates a production incident that makes you the person everyone watches carefully from then on.

Here is what you are mapping in your first 30 days:

• How code gets shipped. Pull request process, number of reviewers required, merge versus rebase conventions, who actually needs to approve before you merge. This is not about following rules. It is about understanding the norms before you violate them accidentally.

• How decisions get made. In the team I lead, nothing gets committed without a design review first. You document your proposed approach and get confirmation from someone experienced before writing a line. Some teams have nothing like this. Some have something entirely different. Ask before you assume your way is the standard.

• How stakeholder updates work. Some teams run a dedicated project Slack channel and push daily updates. Some expect nothing until the thing ships. Do not guess. Ask.

• What the actual team pain is. The problems you experienced in your last job are not automatically the problems here. Some teams struggle with data literacy across the business. Some have a perfectly functional data team sitting next to a shadow analytics function that is quietly doing parallel work and creating slow-burning turf conflict. You want to know that before you accidentally step into it.

The engineers who build trust fastest in the first month are not the ones who ship the most. They are the ones who break nothing and ask the most questions. That reputation is worth more than any quick win.

Habit 2: Form Opinions Before You Are Expected To

Asking questions without forming opinions is just data collection. The second habit is what you do with what you learn.

This is where a lot of technically excellent people stall. They go quiet. They process. They wait until they feel they have enough context to say something worth saying. By then, the window has closed and they are already seen as someone who executes rather than someone who thinks.

You do not need a lead title to have an opinion. You need observations and a reason.

Here is the method. Open a spreadsheet, a Notion board, anything. Start a running log:

• What I observed. Something specific. Not “the pipelines are slow” but “every pipeline runs once a day except this one, which runs every three hours.”

• Why I think this is the case. Based on the questions you asked. Not a guess. An informed hypothesis.

• What I would do to improve the situation. One idea. Sometimes one bullet. Sometimes a few steps.

That third column is where your reputation gets made. It turns observations into proposals. And proposals, shared at the right moment, are what shift how your teammates see you from month one.

There is an important distinction here. This is not the same as the previous habit telling you not to presume. Presuming means acting without context. Forming opinions means building a point of view with context. One breaks things. The other earns you a seat in the conversation.

The engineers who are seen as strong leaders after 90 days are almost never the ones who delivered the most. They are the ones who showed they understood the environment and had ideas about how to make it better.

You can still get it even if you prefer to remain a free subscriber. Grab the field kit here.

Premium subscribers get this FOR FREE here.

Read more

Nobody tells you when your growth stops. You ship things, solve problems, and really get better at the craft. And at some point you look up and realize the bottleneck is no longer the code. It is you. The way you think, communicate, make decisions, understand the systems around you.

That realization is uncomfortable. Most people ignore it. They go deeper into tooling because that is safe and measurable. But the engineers and leads who keep growing past that ceiling are almost always consuming differently. Not just reading docs. Reading things that change how they think.

This is that list.

Newsletters

Technical Depth

1. Data Engineering Central by — If you are looking for opinionated writing on the data landscape from a really experienced guy, this is the newsletter for you. Daniel writes a lot about Databricks, DuckDB, Polars, data quality and overall data architecture.

2. Pipeline To Insights by — Erfan shares real-world experiences, technical tutorials, and personal reflections to inspire growth and continuous learning in the world of data and AI. Plus, Erfan is a really great guy.

3. SeattleDataGuy’s Newsletter by — All things data: end-to-end data flows, MLOps, practical architecture and data leadership. Popular for a reason and consistently useful. Ben has always been nice to me.

4. The Pipe & The Line by — Hands-on guides, tools, and experiments to sharpen your Data & AI Engineering skills from someone who learned it all in the wild. Loads of practical insights on data and AI here.

5. VuTrinh by — Mostly fundamentals here. Vu reads a lot about big corp data architectures and distills it into clean, focused writing. A great place to go deep on the foundations.

6. From Data to Product by — Eric writes about bridging the gap between data and product thinking. He brings 15+ years of experience, including as Head of Experimentation at Yelp, to questions every data professional should be asking: what does this data work actually enable, and for whom?

7. Learn Analytics Engineering by — Practical analytics engineering skills, made accessible. Madison writes clearly about dbt, data modeling, and the craft of building reliable data systems. One of the most consistently useful newsletters in the analytics engineering space.

8. The Data Governance Playbook by — Charlotte writes about data governance from the inside. Ten years of consulting experience, real client stories, and no patience for governance theater. If you are trying to make data governance actually land inside an organization, this is where you start.

Engineering Leadership & Management

9. Engineering Leadership by — One of the most consistent writers on what it actually means to go from engineer to CTO. Gregor went through the whole journey himself, and it shows. I keep recommending this one to people on my team who are starting to think about the leadership path.

10. Manager.dev by — Written exclusively for engineering managers. No career advice for junior devs, no motivation content. Just practical, sometimes uncomfortable takes on building and leading teams. Anton is the kind of writer who calls out what most people are too polite to say.

11. High Growth Engineer by — Jordan writes for engineers who want to grow fast and deliberately. The articles are dense with actual frameworks, not vague advice. If you are somewhere between senior and lead and feel stuck, this is where you should be spending your reading time.

12. The Caring Techie by — Irina writes about influence, leadership presence, and the people skills that no one teaches you in a tech job. This is about leading with emotional intelligence without losing technical credibility.

13. Leadership in Change by — Joel writes for leaders trying to figure out AI without losing their voice or values. Practical frameworks, no hype. Useful if you are managing teams that are already using AI and trying to lead through that shift rather than around it.

Communication & Influence

14. Wes Kao’s Newsletter by — Wes writes about executive communication, managing up, and standing out as a high-performer. The frameworks she shares are sharp enough that I have used them directly in conversations with stakeholders. This is the newsletter I recommend most. One of my most favourite of all times.

15. Strategize Your Career by — Francisco writes about building leverage in your career, or otherwise said, how to stop drowning in the day-to-day and actually make progress on the things that matter. He talks a lot about how you present your work, manage your own visibility, and make the right people notice you.

16. Hungry Minds by — A weekly newsletter on learning, growing, and communicating as a developer. Alexandre writes with a lot of honesty about what actually works when you are trying to level up and be heard.

Strategy & Systems Thinking

17. The Data Ecosystem by — Dylan bridges the gap between data and business strategy in a way that most data engineering content does not bother to do. If you want to understand how data decisions connect to organizational decisions, this is a good place to start.

18. The Breaking Point by — Advice for leaders on how to make better business decisions. Sean is a founder who has been through the full arc and writes with the kind of hard-won clarity that you only get from having made expensive mistakes. I’ve been a dedicated reader for a few years now and I really this publication.

19. Data Operations by — Greg writes about integration, automation, and product challenges through an operational lens. Each issue is a short essay plus one thing that looks like a toy today but might matter a lot tomorrow. Absolutely worth following if you want to think more broadly about how data systems connect to product decisions.

20. The Pragmatic Engineer by — One of the most widely read engineering newsletters for a reason. Gergely covers Big Tech and high-growth company internal. Useful for anyone trying to understand the system they are operating in.

21. Wondering About AI by — Karen writes about AI tools with mixed feelings — she builds them and critiques them in equal measure. Useful for leaders trying to cut through the noise and understand what AI actually changes versus what it just hypes.

22. Product Management IRL by — Amy writes weekly about real product management challenges — the messy, stakeholder-heavy, judgment-dependent kind. Useful for data leads because the overlap between PM and data leadership is larger than most people admit.

Career Growth & Self-Awareness

23. Level Up by — Former Amazon VP writing about what it actually takes to reach executive levels in tech. No generic inspiration. Specific, earned perspective on how promotions really work, how careers stall, and what to do about it.

24. Coding Challenges by — Learn by building real tools from scratch. John gives you a challenge each week, and the deliberate practice compounds fast. This is for people who want to stay technically sharp while everything else in their role becomes more people-focused.

LinkedIn Profiles

Technical Depth

25. Joe Reis — I don’t really need to tell you anything about . He’s one of the OGs in the modern data world. Posts on architecture, the state of the industry, and the mental models behind good DE decisions. I love Joe’s contrarian takes because they have substance.

26. Zach Wilson — Distilled hyperscale DE experience from Meta, Netflix and Airbnb level problems. Skews toward fundamentals and data modeling depth. I disagree with some of his opinions, but knows his stuff.

27. Simon Späti — Deep open-source DE content. Curates the ecosystem better than almost anyone. ’s site ssp.sh is an underrated resource.

28. Maxime Beauchemin — Creator of Airflow and Superset. Less active than he used to be, but when he writes it lands. His 2018 essay on functional data engineering is still one of the best things written about pipeline design. Worth reading even if you think you already know the ideas.

29. Andreas Kretz — He posts consistently about pipeline architecture, data platform design, and the day-to-day of building data systems. Good for staying calibrated on what practitioners are actually dealing with. I’ve never seen a paid video or post from .

30. Julien Hurault — Sharp perspective on the data ecosystem from someone who lives in it. Strong voice in the lakehouse space. You should follow if you want to stay current on where the tooling is heading.

31. Dawn Choo — Data scientist (ex-Meta, ex-Amazon) who posts about career development, AI, and the data profession with unusual directness. is one of the clearer voices on what it actually takes to grow in a data career without the usual hand-waving.

Engineering Leadership & Management

32. Addy Osmani — Posts a lot about AI-assisted engineering lately, but his older content on team effectiveness and technical leadership is where the real gold is. is one of the few people at this level who writes openly and accessibly.

33. Alex Ewerlöf — Staff-plus engineering perspective from someone who has thought very carefully about what technical leadership means beyond the title. also writes at blog.alexewerlof.com and is worth bookmarking separately.

34. Anemari Fiser — Writes about the transition from senior IC to leader in a way that feels grounded and specific, not generic. is one of the more underrated voices in the engineering leadership space.

35. Adrian Stanek — Practical takes on the day-to-day of engineering management. ’s content has a great signal-to-noise ratio, no fluff.

36. Steven Claes — Leadership through the lens of engineering culture. Posts that make you think about your team differently, not just your own career.

37. Karl Ivo Sokolov — Managing Director building the Data & AI service line at Specific-Group Austria. talks about data and AI leadership with a practitioner’s frame, not a consultant’s deck.

Communication & Influence

38. Shachar Meir — Engineering leadership meets communication strategy. Posts about how to get technical decisions across to non-technical stakeholders in a way that lands. is a great guy.

39. Carmine Gallo — Communication coach for executives at Intel, Coca-Cola, and Chevron, and author of Talk Like TED. His LinkedIn content distills decades of studying what makes ideas stick in presentations and conversations. If you present technical work to leadership, follow him.

40. Matt Abrahams — Stanford lecturer and host of the Think Fast Talk Smart podcast. Writes about spontaneous communication. One of the clearest thinkers on how to communicate under pressure.

41. Clare Kitching — Data and AI strategy consultant with a decade at McKinsey and QuantumBlack. Posts about AI adoption, data foundations, and what it actually takes to make transformation work. Sharp on the gap between what organizations say they’re doing with data and what they’re actually doing.

Strategy & Systems Thinking

42. Taha Hussain — Systems thinking applied to data and engineering organizations. posts about how platforms, teams, and architectures interact in ways that most practitioners miss until it is too late.

43. Vin Vashishta — Data and AI strategy at the organizational level. thinks about data differently from most engineers, through the lens of business value, competitive positioning, and organizational readiness. Follow him to develop a more strategic frame for your own work.

44. Cassie Kozyrkov — Google’s first Chief Decision Scientist, founder of the Decision Intelligence field, now CEO of Data Scientific. has over half a million followers for good reason. Posts that make statistical thinking feel urgent and applicable. Required follow for anyone making data-backed decisions at scale.

Career Growth & Self-Awareness

45. Ryan Peterman — Staff engineer at Instagram who writes about career progression with unusual honesty. talks about the parts of growing that are uncomfortable, not just the milestones.

46. Neo Kim — Author of the System Design Newsletter. has a sharp perspective on what it means to be known for something, how to build a reputation in the field, and how to make technical work visible. Plus, he’s really kind!

47. Daria Rudnik — Career growth and self-awareness content for data professionals. covers the stuff that technical training ignores: knowing your own patterns, managing energy, making better decisions about your own path.

Books

Technical Depth

48. Designing Data-Intensive Applications by — The foundational text for understanding distributed systems at the infrastructure level. Not DE-specific but irreplaceable for anyone who wants to understand why their stack makes the choices it does.

49. Fundamentals of Data Engineering by Joe Reis & Matt Housley — The clearest mental model for the full DE lifecycle. Read it to find the gaps in your own mental model, not to learn syntax. Also, the data modelling book is shaping very well.

50. The Pragmatic Programmer by David Thomas & Andrew Hunt — A classic for a reason. But what makes it relevant here is not just the technical advice. It’s actually is the philosophy of craftsmanship and continuous self-improvement that runs through the whole book. The engineers I respect most have internalized this way of thinking, whether or not they have read it.

51. Your AI Survival Guide by Sol Rashidi — helped IBM launch Watson in 2011 and has spent the years since doing real-world AI deployments across industries. This is not a book about AI theory. It is a book about what breaks when you try to ship AI in an organization and how to fix it.

52. The Data Hero Playbook by Malcolm Hawker — spent over 25 years working with CDOs and data leaders and got fed up watching the same mistakes repeat. This book is about the mindset problems that sink data initiatives, and how to fix them.

Engineering Leadership & Management

53. High Output Management by Andrew Grove — Written by the former CEO of Intel, this is still the most rigorous book on management ever written for people who came up through engineering. Grove writes about meetings, decisions, performance reviews, and organizational leverage with the precision of someone who built one of the most operationally excellent companies in history. This book was mandatory for every manager in my org.

54. The New One Minute Manager by Ken Blanchard & Spencer Johnson — Short enough to read in an afternoon. The core ideas, sound almost too simple until you realise how rarely anyone actually does them. A good early read before you have your first direct report.

55. The Manager’s Path by Camille Fournier — The foundational text for the engineering management track. Camille mapped out the whole journey from tech lead to VP in a way that actually prepared me for what was coming. Required reading before your first direct report, not after.

56. An Elegant Puzzle by Will Larson — Systems thinking applied to engineering organizations. Will Larson writes about team sizing, migration strategies, and organizational design with a level of rigor you rarely see outside of academic papers. The org-chart chapter alone is worth the price.

57. Staff Engineer by Will Larson — If you are approaching the staff level or managing people who are, this is the clearest map of what that role actually looks like across different companies and archetypes.

58. Resilient Management by Lara Hogan — Short, dense, and written by someone who spent years as VP of Engineering at Kickstarter and Engineering Director at Etsy. writes about the fundamentals of managing engineers: how to build trust, give feedback, navigate team stages, and grow people without burning them out or yourself. This book shaped a big part of my management philosophy.

Communication & Influence

59. How to Win Friends and Influence People by Dale Carnegie — Published in 1936 and still one of the most practically useful books on human dynamics ever written. The title makes it sound manipulative. The content is mostly about listening, showing genuine interest, and making people feel valued.

60. Captivate: The Science of Succeeding with People by Vanessa Van Edwards — Not a tech book. That is exactly why it belongs here. is a behavioral researcher who has spent years studying how people actually communicate, build rapport, and earn trust. The frameworks in this book are more applicable to one-on-ones and stakeholder conversations than most “communication for engineers” content I have read.

61. Cues: Master the Secret Language of Charismatic Communication by Vanessa Van Edwards — The follow-up to Captivate. More focused on the nonverbal and micro-behavioral signals that shape how you are perceived in meetings, presentations, and conversations. Read this one twice. Oh, also, check her YouTub channel!

62. Never Split the Difference by Chris Voss — A former FBI hostage negotiator writes about negotiation and influence. It sounds like it has nothing to do with data & engineering leadership. It has everything to do with it. This book prepares you for difficult stakeholder conversations, budget discussions, or pushback on scope.

63. On Writing Well by William Zinsser — A book about writing clearly and with purpose. The kind of writing that gets your ideas across in a design doc, a Slack thread, or a post-mortem without anyone having to read it twice. Most technical writers benefit more from this than from any course on technical writing.

64. Talk Like TED by Carmine Gallo — Gallo broke down hundreds of TED talks to find what made the best ones land. The result is a practical framework for any presentation where you need to make a complex idea stick with a non-technical audience. Useful every time you walk into a room to justify a platform decision or a team investment.

Strategy & Systems Thinking

65. Good Strategy / Bad Strategy by — The clearest book I have read on what strategy actually is and why most of what gets called strategy in organizations is not strategy at all. Every data lead who has ever sat in a planning meeting wondering why nothing sticks should read this.

66. The Goal by Eliyahu Goldratt — A novel about a manufacturing plant manager trying to save his factory. Goldratt’s theory of constraints is the most useful mental model I have applied to pipeline architecture and team throughput. Read it once and you will never look at a slow pipeline the same way again.

67. Playing to Win by & A.G. Lafley — Strategy as a set of explicit choices: where to play and how to win. Martin strips out all the planning theater and gets to what strategy actually requires. More rigorous than Good Strategy Bad Strategy and more actionable for people who need to make real platform or team decisions.

68. The Effective Executive by Peter Drucker — Written in 1966 and still more relevant than most modern management content. Drucker writes about time, priorities, and decisions in ways that apply directly to anyone in a senior technical role. I re-read sections of this every year.

69. Upstream by Dan Heath — About solving problems before they happen rather than reacting to them after. The mental model maps directly onto data platform design, incident prevention, and technical strategy.

Career Growth & Self-Awareness

70. So Good They Can’t Ignore You by Cal Newport — The antidote to “follow your passion” career advice. Cal argues that career capital is what gives you leverage, options, and eventually, work you care about. I think about this book every time I talk to someone who is unsatisfied with where they are.

71. A Few Wise Words by Peter Mukherjee — A collection of success stories from accomplished people across different fields, each sharing in their own words what the journey actually looked like. is one of the smartest and humble people I know.

72. Man’s Search for Meaning by Viktor Frankl — Written by a psychiatrist who survived the Nazi concentration camps and spent the rest of his life studying what keeps people going when everything is taken from them. The answer is purpose. It’s not a career book. But every engineer I know who has read it has come back with a clearer sense of what they are actually working toward.

73. The Subtle Art of Not Giving a F*ck by — A blunt, practical argument for choosing your problems deliberately rather than chasing the illusion of a problem-free life. Useful for anyone who finds themselves grinding on things that do not actually matter, or avoiding the hard conversations that do. This book was a blog post, btw.

74. What Got You Here Won’t Get You There by Marshall Goldsmith — About the specific behaviors that hold successful people back as they move into leadership. Not the obvious stuff. The subtle habits that made you effective as an IC and will quietly undermine you as a lead.

75. Radical Candor by Kim Scott — How to care about people and challenge them directly at the same time. Most managers do one or the other. Kim Scott makes the case that doing both is not a contradiction, but the job. Read by almost every engineering manager at some point. Worth reading before you need it rather than after.

76. Thinking, Fast and Slow by Daniel Kahneman — The science of how we make decisions and where we reliably go wrong. Long, dense, and worth every page. Once you understand the two systems Kahneman describes, you start recognizing their fingerprints on every estimation, every post-mortem, every hiring decision you have ever made.

77. The 5 Elements of Effective Thinking by Edward Burger & Michael Starbird — A short, dense book on how to think better. The book applies directly to how you approach problem-solving, architecture decisions, and career growth. I return to this one more than most.

Podcasts

78. Data Engineering Podcast by — The most technically rigorous DE podcast running. Petabyte-scale systems, Iceberg table management, orchestration failures. Consistent quality since 2017.

79. Soft Skills Engineering by Dave Smith & Jamison Dance — They answer engineering career questions in a format that is somehow both funny and genuinely useful. It covers everything from how to handle a difficult manager to whether you should take a promotion. This podcast actually taught me that it takes more than great code to be a great engineer.

80. Think Fast Talk Smart by Matt Abrahams — Stanford podcast on communication under pressure. Short episodes, high density. The focus on spontaneous speaking makes it more practical for working engineers than most public speaking content.

81. Acquired by & David Rosenthal — Long-form deep dives into how great companies were built. Not a strategy framework podcast, but something better. You walk away understanding how specific organizations made specific decisions over time. The compounding effect of listening to a dozen episodes is a permanently better sense of how businesses actually work.

82. The Knowledge Project by Shane Parrish — Farnam Street’s podcast on mental models, decision-making, and the habits of people who think clearly. Every episode is a masterclass in how to reason better under uncertainty. Slow to binge but the kind of content that compounds over years.

Final Thoughts

Do not follow everybody on this list. Do not subscribe to every newsletter. That is not how this works.

Pick two or three sources per category that actually match where you are right now. Follow them consistently. Read them when they land. Let them compound.

Most of the people on this list are not just newsletter writers or LinkedIn creators. They have books, courses, podcasts, YouTube channels, and communities built around the same ideas they post about for free. Once someone’s thinking starts landing for you, go deeper. Find them on every platform they are active on. Buy the book. Take the course. Join the community. The newsletter is usually the smallest part of what they offer.

Two or three resources you actually read will do more for you than twenty you subscribed to and forgot about. The goal is not to consume more. The goal is to think better.

—

Yordan

More on the Topic

• The Smart Engineer’s Framework to Staying Current Without Noise

• The Certifications Scam

• 5 Red Flags of Mediocre Data Engineers

Presenting Yuki Kakegawa

Yuki is a Staff Data Engineer, author of the Polars Cookbook, and the founder of Orem Data. He writes about data tools and independent consulting on Substack and LinkedIn.

I asked , Staff Data Engineer and author of the Polars Cookbook, what the biggest adjustment was when he became a team of one.

He skipped tooling. He skipped workload. He said this:

It’s a shift in mindset. You’re not a data engineer anymore. You’re a data person helping the business with data and analytics

That sentence landed harder than I expected, because most people assume the challenge is the technical breadth: wearing all the hats, being the analyst, the engineer, and the data scientist rolled into one.

That part is visible. What nobody talks about is the identity part.

You Were Hired as a Data Engineer. That Is the Problem.

At a large company, your identity is your scope. You do pipeline work, pass it to the analyst, the analyst makes the report, the report goes to the business. You are one node in a chain, and that chain insulates you from the messiness of what the business needs.

When you’re a team of one, that chain is you.

Yuki described his current day-to-day:

• talking to the business to understand what they care about

• translating that into projects

• building the pipelines

• building the models,

• building the reporting layer

• delivering insights directly to stakeholders

He skips the handoff entirely.

That scope describes a different job, built around a different identity.

Subscribe now

The Selfishness That Kills You

He used a word that surprised me: selfish.

When I was first getting started, I wanted to work on pipelines and I didn’t want to work on reporting. That selfishness kills you in this role. You have to be flexible and you have to care about the business first.

Most data engineers have strong opinions about which work is worth doing. Pipelines are interesting. Reporting is boring. Data modeling is craft. Dashboards are noise.

Those opinions are how you survive in a large org where you can negotiate your scope.

A team of one has no scope to negotiate. The business has no interest in which part of the stack you find meaningful.

The work that needs doing is the work you do, and if you have not made peace with that before taking the role, the first six months will feel like a slow grind against a situation you agreed to.

What Is Broken at Every Startup You Walk Into

I asked Yuki what he finds broken almost every time he walks into a young company.

Tooling is rarely the issue. It’s the processes around the processes that produce the data used for reporting and analytics downstream. And the alignment on what’s important, the definition of metrics, what we want to build.

Two things. The upstream processes that generate data, and whether the business has agreed on what the numbers mean. The stack is almost never the problem.

The Metric Definition Problem

This is the one that is hardest to fix and most commonly ignored. Yuki described the ideal state: every metric the business cares about is written down, defined, and agreed on before the data team touches a model.

When that is true, the data team’s job is implementation. The hard part is already done for you.

The real state at most companies is three departments with three definitions of the same number.

If you use one team’s definition, you put another team in a bad spot. If you try to reconcile them, you are suddenly in a political conversation nobody hired you to have.

I’ve been in that position. That’s the part I hated the most.

Defining core metrics early, before the company grows, before each department builds its own reporting layer, before everyone has opinions baked into their own numbers, is more valuable than any pipeline you will build.

If you’re walking into a company where this work is undone, do it first.

How to Handle the Stakeholder Who Thinks Your Work Takes Two Weeks

Yuki described a pattern that every team-of-one data person will recognise. A stakeholder requests a report, thinks it can be done in two weeks, and the data behind it is messy enough that you know it will take four.

I can try, but I can’t promise, because I think these things will be bottlenecks.

Two things are happening in that exchange. Setting expectations is the obvious one. Making the complexity visible is the one that changes the relationship long-term.

When you’re the only data person, the rest of the company defaults to assuming data work is fast. Pull the numbers, build the report, done.

They genuinely have no frame of reference for what it takes to model messy source data into something accurate enough to make decisions with. Your job is to narrate the work before you do it, and skip the explanation after you miss the deadline.

The Transparency Stack

Yuki added something that makes the whole prioritisation problem easier: making your priority stack visible to the entire business.

Being transparent about what you’re working on and what the priorities are is important. Because if one stakeholder thinks his project is the top priority, but I’m working on a priority item requested by the CEO, then that stakeholder will understand that his project is not at the top.

This works because it shifts who does the priority negotiation. When everyone can see the queue, the conversation moves from “why isn’t my thing done“ to “is my thing in the right place in the queue“. The second conversation is faster and involves fewer emotions.

Accuracy vs. Speed Is a Trade-Off to Name, Not a Choice to Make Alone

I pushed Yuki on a question that sounds simple: accuracy or speed?

Definitely accuracy. But that’s where your skill comes in. How do you deliver projects fast while ensuring accuracy?

The real answer lives in the communication around the trade-off, not the trade-off itself. If you can deliver in one week at 80% data quality, or in two weeks at 100%, the stakeholder should make that call. They can only make it if you put the options in front of them explicitly.

If you wait two weeks, I can ensure 100% accuracy. If you want it in one week, there might be things that still need to be pinned down.

Yuki’s underlying point is worth sitting with: making decisions on low-quality data defeats the entire point of data-driven decision making. Speed that produces bad numbers is a liability that takes three times longer to fix than the shortcut saved.

On Using AI When You Have Nobody to Think With

One thing Yuki said is worth pulling out directly, because it is the most honest framing of AI use I have heard from anyone working in the data space:

I use AI especially when I want to bounce ideas around. I use it because I don’t have anybody else to talk to, because I’m the only data person.

AI as a rubber duck rather than a replacement for judgment. At a startup where you’re the only data person, you have no peer to sanity check your modeling approach, your architecture choices, or whether your prioritisation call makes sense. AI fills a small part of that gap.

But Yuki also admitted he is not sure AI is making him faster overall. The time he used to spend building the solution, he now spends validating what the AI built. The total may net to zero, with the shape of the work changed and the volume unchanged.

That matches what I have seen. AI handles narrowly scoped, well-defined tasks well: understanding what a table column means by parsing the codebase, generating boilerplate inside a framework you have already defined, drafting initial SQL that you then refactor.

For architectural decisions and anything that requires sustained judgment, the model is an input, and the judgment stays with you.

Connect With Yuki

Yuki is very active online. Here are some of the best ways to connect with Yuki:

• Substack profile

• The Data Toolbox

• The Independent Insight

• LinkedIn profile

• The Polars Cookbook

• Consultancy website

—

Yordan

I was coaching Hamza last year.

He was one of the best engineers I had worked with outside my team. Sharp, reliable, the kind of person who fixes the production incident at 2am and never makes it anyone else’s problem. He had delivered more that year than most people deliver in two.

He got a 3% raise in January.

When we talked about it, the pattern was immediate:

• He had done everything right technically and nothing right strategically.

• He never flagged his expectations.

• He had no record of what he had delivered.

• He walked into the review meeting with memories and feeling instead of a number.

His manager liked him, but had no ammunition.

Getting a raise has almost nothing to do with whether you deserve one. It has everything to do with whether your manager can build a case for you in a discussion you are not in. The people who get raises are the people whose managers have the data to justify it, the evidence to present it, and the confidence the conversation will stay professional.

The research backs this up. According to Pew, 38% of workers who did not negotiate said they felt uncomfortable asking. The same study found that 66% of workers who did ask got what they asked for. Two thirds. The ask works.

This article is the system Hamza did not have before we worked together.

Download the complete manual at the end of the article.

Why Good Engineers Stay Underpaid

Most of the data engineers I know who are underpaid are genuinely strong. They ship, fix things before anyone notices they are broken, and mentor the junior engineers without being asked. They are the people their companies cannot afford to lose.

And yet they lose money every single year.

The problem is structural. Salary decisions do not get made by the person who sees your work every day. They get made in budget meetings, by people who know you mostly by reputation, based on recommendations from managers who have five other people to advocate for at the same time.

The engineer who gets the raise is the one whose manager walks into that room with the strongest case.

Everything else is secondary.

The math your company will never show you

Here is something your HR department knows and hopes you never think about:

According to SHRM, replacing an employee costs between 50% and 200% of their annual salary. For senior roles, the Center for American Progress puts that number at up to 213%. Recruiting fees, onboarding time, lost productivity, the six months before a new hire operates at full speed.

Giving you a 10% raise costs them 10% of your salary.

The math is on your side. The problem is you are not using it.

Subscribe now

Why the conversation never happens

The Pew Research Center found that 38% of workers who did not negotiate said they simply felt uncomfortable asking. Not a bad manager. Not bad timing. Discomfort is the real enemy.

So engineers stay quiet. They assume good work speaks for itself. They wait for their manager to bring it up. And their manager, who is busy and conflict-averse and assumes you would say something if you were unhappy, never does.

Nobody brings it up, and another year passes.

The 90-Day System

A raise is a system you run across the whole year.

The engineers who get what they ask for flag their expectations early, build a running record of their impact, research their market value before they need it, and invest in the right relationships long before budget season.

By the time the review meeting happens, the outcome is mostly already decided.

The rest of this article walks through each part of that system in full:

• How to flag your expectations early enough to actually influence the outcome

• How to build a brag list that gives your manager something to fight with

• How to find your market number before the conversation

• How to win the sponsors who have real budget influence

• How to run the conversation itself, including what to do when they say no

This is the system I walked Hamza through. He got a 13k salary increase. Keep reading and download the ebook at the end.

The content below is for paid subscribers. If you are not subscribed yet, this is the part worth paying for.

Read more

Presenting Prajakta Yerpude

Prajakta spent a decade climbing every rung of the data engineering ladder, intern, engineer, senior, lead, manager, and somewhere along the way realized her biggest impact was no longer in the code she wrote but in the people she unblocked.

Connect with Prajakta

Getting promoted into management felt like winning. I had the title, the 1:1s, the org chart with my name at the top. What nobody told me was that everything I had spent years getting good at had just become irrelevant.

The skills that got me the promotion were the wrong ones for the job. Speed, ownership, technical control, those are IC superpowers. In management, they become liabilities.

figured this out the hard way too, after a decade climbing every rung of the data engineering ladder. What she learned on the other side is clearer than anything I’ve read on the topic, and it maps almost exactly to where most senior data engineers quietly stall

The data teams that are ahead right now aren’t just experimenting with AI. They’re shipping agentic analytics systems that reason, recommend, and explain their work. And they’re learning things the rest of the industry hasn’t caught up to yet.

Cube’s Agentic Analytics Summit on April 29 is where some of those teams share what they’ve found. Dan Meshkov from Brex will talk about what a data foundation looks like for a mission-driven company. Gabe Romero from Jobber will cover how financial reporting changes when you move beyond charts. Joe Reis will dig into data engineering in the agentic era.

If you lead a data team or build data infrastructure, the insights here could save you months of trial and error.

Grab your free seat here

You Are Already a Manager Before the Title

Most people wait for the promotion to start operating differently. That is the wrong order.

The data professionals who get promoted into leadership are not the ones who wanted the title. They are the ones who were already doing the work before anyone gave it to them. Clarifying requirements with stakeholders when nobody asked. Helping teammates get unblocked when they had their own deliverables to ship. Asking the why behind the problem instead of just solving the what.

The Week Your Code Sits Untouched

There is a specific moment that signals the shift. You get to the end of a week and realize you made almost no progress on your own work — but the team moved forward because of the conversations you had.

That discomfort is the job changing underneath you.

If you wait until the title to start operating at the next level, you are asking your manager to take a bet on future behavior. That is a harder sell than showing them behavior that is already there.

Speaking Up Is a Skill

Early in any leadership journey, the most uncomfortable moment is the same for almost everyone: being in a room of more experienced people, knowing something is wrong, and deciding whether to say it.

The imposter syndrome is real. The difference is learning not to let it make the decision for you.

Facts Remove the Politics

In data, you have a specific advantage: you can ground a challenge in numbers and evidence. When you do that, you shift the conversation from opinion to information. The most experienced person in the room does not win by default anymore.

Leadership is about stepping up when something needs to be said. That is available to you at any level. An intern can flag a bad assumption. A senior IC can challenge a decision made three levels above them.

The cost of staying quiet is harder to measure than the cost of speaking up. But it compounds over time.

Empathy Is Leverage.

As engineers, we are trained to solve technical problems. The higher you go, the more you realize the hardest problems are people problems.

Misalignment. Stress. Lack of clarity. Someone having an off day. These do not have a clean solution. Some of them do not need a solution at all. Instead, they need listening.

The Problem-Solver Trap

The instinct to fix things immediately is exactly what gets senior ICs into leadership. It is also the thing that makes early management hard.

Not every situation calls for a solution. Sometimes the person in front of you just needs to feel heard. When you learn to read that distinction — when you can tell the difference between a problem that needs solving and a person who needs to know you are paying attention — the team responds differently.

People who feel understood and supported perform better without you pushing harder. You do not manufacture that with a process or a framework. It is built through consistent, genuine attention to what is actually going on beneath the surface.

Technical skills get you to the role. Empathy and emotional intelligence determine how far you go inside it.

Subscribe now

Your Impact Becomes Invisible. That Is the Whole Point.

As an IC, your output is visible. Code you wrote. Pipelines you fixed. A cost number you moved. You can point to it directly.

As a manager, your impact shows up through other people. Better decisions. Stronger execution. A team delivering consistently and growing in the process. None of that has your name on it.

Unlearning the Need to Own the Outcome

The hardest thing to unlearn is tying your personal value to what you directly produce.

Andy Grove put it clearly in High Output Management: your output as a manager is your team’s output. That is the measure. You do not have a separate output anymore. The team speaks on your behalf.

That means impact can look like:

• Creating the right direction when things are ambiguous

• Removing friction before it slows the team down

• Helping someone succeed in a way they would not have without your support

• Building the environment in which good work happens consistently

You stop building systems. You start building the conditions in which good systems get built. That is more scalable than anything you could ship yourself.

The Myth That Needs to Die

The most persistent management myth: a good manager always has to be in control of everything.

New managers fall into it. Experienced managers fall into it. They believe that effectiveness means knowing every detail, monitoring closely, and constantly stepping in. What that actually creates is dependency. It slows people down. It makes you a single point of failure.

The Real Measure

The actual measure of good leadership is when your team can collaborate, problem-solve, and resolve issues without you needing to be in the room. When something gets fixed before you even know it happened. That is the job done well.

High standards work better when trust already exists. When people know you genuinely support them, they are more open to feedback, more willing to stretch, more willing to take on work that makes them uncomfortable.

You can be kind and still be clear. You can be supportive and still hold people accountable. The balance is a more effective version of both.

Final Thoughts

The move from IC to manager is a step sideways into a completely different job that happens to sit inside the same organization.

The data professionals who stall at senior are almost always technically strong. They can build. What they have not yet built is the capacity to make others better at building.

If you are waiting until you feel ready to step into that, here is the honest version: you will never feel fully ready.

Growth happens because you stepped forward before you did.

—

Yordan

More on the Topic

These are some of the articles Prajakta mentioned in our chat:

• The Operating System Every Data Engineering Leader Needs

• Business value mapping playbook: How to translate data work into executive language

• How To Stop Waiting For Permission To Lead Data Engineering

• Why Senior Data Engineers Lose Their Velocity (Even When Their Skills Improve)