Chapter 5 Tools, but not about tools

Everyone has the tools now.

In 2015, "being technical" as a PM meant you could write SQL, read a Jira ticket, build a wireframe in Balsamiq. Those were scarce skills. They signaled something about your seriousness and your range. Hiring managers used them as proxies for rigor.

In 2026, a new grad with a browser can generate a PRD, prototype three variants, run a cohort analysis, and draft the launch email before noon. The skills aren't scarce. The judgment about what to do with the output is.

So this chapter is not about which tool to use. It's about what the job looks like when the tools stop being the point — and what failure modes appear when PMs don't notice that the tools changed faster than their habits did.

There's a subtler version of this problem. The tools didn't just get easier — they got easier in ways that make bad habits feel like productivity. You can produce more artifacts per hour than ever before. Artifacts are not impact. The PM who has learned to distinguish between the two is the one who builds real things. The PM who hasn't is the one who ships roadmaps nobody follows and documents nobody reads.

5.1 The artifact trap — a deeper look

The artifact trap is measuring your day by output. Docs written. Decks completed. Dashboards built. Tickets closed. It's seductive because output is visible. Impact is delayed and often ambiguous. When you've shipped three polished docs by noon, you feel like you've worked. When you've spent the morning in one difficult conversation that changed a decision, you feel like you haven't.

One of those is more valuable. You probably know which one.

AI made artifacts infinite. You can generate twenty PRDs by 10am. None of them matter if you picked the wrong problem. The trap is feeling productive because output is high, while impact stays flat. This is a new version of an old trap — before AI, it was the PM who spent three weeks in a spreadsheet building a prioritization model instead of talking to the three customers who could have told them what mattered. AI didn't create the artifact trap. It made it faster.

Here's the specific failure mode by context:

At a startup, the artifact trap looks like velocity. You can ship a new landing page, a new onboarding flow, a new pricing experiment every week. You will. Your users will be confused, your data will be too thin to read, and your team will burn out sprinting at targets that keep moving. The PM's job at this stage is saying "stop" more than "go." The artifact trap at a startup is not about documentation — it's about building the wrong things at high speed. The startup PM who can say "we're learning from this one before we build the next" is more valuable than the one who ships three things before anyone can learn from the first.

At a scale-up, the trap is process documentation. A spec for every ticket. A review for every spec. A deck for every review. A stakeholder update for every deck. AI makes the artifacts easy to generate, which means more of them get generated, which means more reviews are needed, which means the team is producing documentation about documentation. The org has accidentally built a documentation factory. Your job: ask what you can delete. Not defer — delete. "What do we stop doing to create capacity for this?" is a PM question AI will never ask. It will only help you do more of what you're already doing.

At a mega-corp, the trap is documentation theater. The twelve-page memo with appendices. The pre-read that people read on the way to the meeting and then set aside. Executives will skim and ask one question you didn't answer. AI can generate the twelve pages in twenty minutes. That does not make twelve pages the right answer. The PM who can write the one-page version — and defend why the other eleven pages are scaffolding, not substance — is the one executives trust to make decisions. Length is not rigor. Precision is rigor.

How to test yourself: Open your document folder. Count how many docs produced in the last quarter were read by more than three people. Count how many of those changed a decision. If the second number is much smaller than the first, you're in the artifact trap. The fix is not to write better docs. It's to stop writing the ones that don't change anything.

5.2 Prototyping: speed as a forcing function, not a substitute for clarity

The shift here is clean. Before AI-assisted prototyping, building a testable version of an idea was expensive enough that you debated before you built. The debate was the work. You had to be clear enough about the idea to justify the cost of prototyping it.

Now prototyping is cheap. One afternoon in v0 or Replit or Claude gets you something testable. The risk is using that speed to defer clarity rather than achieve it. You prototype five things because you haven't decided which one is worth the team's time, and each prototype generates three new questions instead of answering the one you started with.

The discipline is the same as it's always been — write the success criteria before you build. "If three of ten users complete this flow without asking for help, we proceed." If you don't write that before you prototype, you'll iterate forever on things that feel promising but never cross a threshold that warrants shipping.

What changed: the failure mode is no longer "we didn't prototype because it was too expensive." The failure mode is now "we prototyped twelve things and learned nothing because we never defined what learning looked like." The tool got cheaper. The discipline requirement got higher.

There's also a subtler shift in what prototyping does to team dynamics. When prototyping is expensive, the PM's job is to enable it — to get design and engineering aligned enough to invest in a prototype. When prototyping is cheap, the PM's job is to constrain it — to prevent the team from spending three weeks refining a prototype for an idea that shouldn't survive a two-hour user test. Speed is a forcing function. It only works if you've defined the force.

At a startup: You don't have a designer. You have the tools and a browser. You can get to testable in an afternoon. Use it to kill bad ideas fast — not to polish them into things that look like products before you've proven they solve anything. The prototype that goes to users on day three beats the polished demo that never leaves the internal review. Speed is your advantage. Don't waste it on polish.

At a scale-up: You have design, you have research, you have a process that exists for good reasons. AI makes that process look slow. A PM with Claude can produce a testable prototype while the design brief is still being written. Use that speed to filter: run the rough test to kill the ideas that aren't worth Design's time, then bring the survivors to the team for craft. Pre-filtering for design is a legitimate and high-value use of AI prototyping. Replacing the design process entirely is not.

At a mega-corp: You have a design system, accessibility requirements, brand guidelines, and a legal review process. You cannot just ship a test. But everyone else in the building can prototype on your surface — marketing, growth, other PMs, third-party partners. Your job is knowing which experiments are running without you, which ones matter, and which ones violate a constraint the person who built the prototype didn't know to check. Governance of prototyping is a PM function at this scale, not just execution of it.

5.3 Self-service data: the danger of confident wrongness

Self-service analytics used to be a PM superpower. Before Looker, before Amplitude, before the modern data stack, you needed a data analyst to run any query that wasn't already in a prebuilt dashboard. PMs who could write SQL were rare. They had a genuine advantage: they could answer their own questions, iterate faster, and arrive at stakeholder conversations with evidence instead of intuition.

By 2020, most mid-sized tech companies had self-service analytics. By 2026, you have AI-assisted query generation. You describe the question in plain English and the tool writes the SQL. The advantage that used to accrue to PM-who-can-query is gone. Everyone can query.

What didn't get easier: knowing whether the query is right.

The risk isn't that AI gives you wrong numbers. It's that it gives you wrong numbers confidently, with clean formatting, in a chart that looks like it came from your data team. The LLM doesn't know that event_timestamp is UTC while your user interface displays Pacific. It doesn't know the table you're querying was deprecated in Q4 and the new one has a different schema. It doesn't know about the sampling decision your data team made that excluded a specific cohort from the default dataset because of a data quality issue they're still resolving.

The confident wrongness problem is real and it's new. Before AI query generation, if you wrote a wrong query, you usually got an error or obviously nonsensical results. AI query generation produces plausible-looking results even when the underlying query is wrong. The number looks right. The chart looks clean. The conclusion feels supported. The data is misleading.

The discipline: treat self-service data as exploration, not as evidence. Use it to generate hypotheses, to understand direction, to figure out what question you actually want to ask. Don't use it as the source you cite in the exec review without someone who knows the schema confirming the join was right. The data analyst's value is not running queries — it's knowing when a query will lie to you.

At a startup: Learn the schema physically. Sit with the engineer who built it and have them walk you through the tables. The model will confidently query the wrong thing if you don't catch it. Being the PM who knows what the data actually means — not just what it says — is a compounding advantage that pays off for years.

At a scale-up: The data team is backlogged. You're going to self-serve. That's fine and right. Build the habit of tagging the data team before you act on a finding, not after you've shared it with stakeholders. "Here's what I found, here's what I'm planning to do with it, please confirm the query is right" is one message. It saves you from presenting a broken number in the quarterly business review and spending three weeks recovering from the credibility hit.

At a mega-corp: You have experimentation infrastructure. Use it. The rigor exists for a reason — the sample size requirements, the holdout period, the guardrail metrics. When PMs run informal tests outside the infrastructure, they produce results they cannot trust and cannot defend. The infrastructure is slow on purpose. If the speed of the infrastructure is your bottleneck, the answer is to push to improve the infrastructure, not to route around it.

5.4 Writing: AI drafts, you write the subtext

Before AI, a clean, concise, well-structured PRD signaled PM ability. Writing was the visible craft. An interviewer could read your PRD and know a lot about how you thought, how clearly you understood the problem, whether your priorities were in the right order, whether you'd anticipated the objections. The document was the test.

After AI, every PM can produce a clean, well-structured PRD in twenty minutes. The document no longer signals what it used to signal. Interviewers know this, which is why the PM interview is shifting toward real-time exercises — "here's a problem, you have one hour" — and away from "walk me through a product you built."

What AI doesn't write: the subtext.

The subtext is the part that makes a PM document useful rather than just presentable. It's the paragraph that explains why Sales is going to push back on this and how you'd handle it. It's the footnote that acknowledges the competing initiative from the other team and why you believe the two aren't in conflict. It's the framing that makes the tradeoff visible — "we're choosing to optimize for new user activation at the cost of power user retention in this release, and here's why that's the right call for this quarter." That sentence requires you to have actually made the tradeoff consciously, to have thought through both sides, and to be willing to put your name on the choice.

AI writes the first draft. You write the second draft — the one with the subtext. The ratio of time you spend on the first draft to the second draft is a reasonable proxy for how much PM value you're adding to the document.

Here's a specific example of the subtext problem. A PM uses AI to write a spec for a new notification feature. The AI produces a clean spec: user story, acceptance criteria, edge cases, success metrics. It's good. But it's missing three things only the PM knows: (1) Engineering flagged a concern about notification fatigue six months ago and that concern was never fully resolved, (2) Marketing is planning a campaign that will generate high notification volume at the same time this ships, and (3) the success metric (open rate) is measured differently in iOS than Android due to a privacy change last year. None of those things are in the AI draft. All of them will bite the team if they're not in the spec. The PM who adds them writes the spec that prevents two incidents. The PM who ships the AI draft writes the spec that causes them.

At a startup: The doc is a Notion page nobody will read if it's more than one screen. AI will generate a beautiful six-section PRD. Delete four sections. Then delete the section headers. Then read it out loud. If you can explain it that way, ship the doc. If you can't, the doc is hiding a thinking problem, not a formatting problem.

At a scale-up: The doc needs to survive a reviewer who has twelve of them in their queue and will spend four minutes on yours. AI will make yours look identical to the other eleven. What makes yours legible is the thing only you know: what failed in the previous attempt, what the data says that isn't in the standard dashboard, what the customer said that changed the problem statement. Put that in the first paragraph. Not the sixth.

At a mega-corp: The doc enters a process. Legal, policy, comms, executives with context you don't have about concerns you didn't anticipate. The PM who writes for the reviewer — not just for the process — gets through faster. "I flagged the privacy implication of section 3 with legal and here's their input" is the sentence that saves you two weeks of back-and-forth.

5.5 What remains when all the tools are free

Three things. They've always been three things. AI just made them more visible by removing everything around them.

Problem selection. AI will optimize whatever you give it. If you give it the wrong problem, it will optimize the wrong problem with impressive speed, beautiful documentation, and high confidence. Every PM tool gets better at execution. None of them get better at knowing which problems are worth executing on. That selection — the choosing of what to work on from the infinite space of things that could be worked on — is still entirely yours. It always was. The tools just used to obscure this by making execution feel like the hard part.

Narrative. Why now, why us, why this is the thing to do when everything is hard and the metric is flat and two stakeholders have competing priorities and the engineer you most trust just told you the timeline is longer than you thought. Teams don't follow charts. They follow the story they can retell. AI gives you words. Words are cheap now. Conviction isn't. The specific conviction that comes from having been in the customer interviews, from having made the call that turned out wrong and then the one that turned out right, from understanding the product's scars as well as its roadmap — that conviction is yours. It's what the AI draft is missing. It's what the team is following when they follow you.

Accountability. The model cannot get fired. It cannot be held responsible for a product decision that hurt users or missed the quarter. You can. That asymmetry is why the job exists and why it is, at its best, a serious job. The PM takes the accountability that the team cannot share evenly because most of them don't have the full context. In exchange, the PM has the authority — influence, not formal — to set the direction. Accountability and authority travel together. Both are still human.

5.6 Try this

The two-column test. Take your current project. In one column, list everything AI can help you do: generate the PRD, prototype a variant, run a metric query, draft the stakeholder update. In the other column, list the things AI cannot do: name the three people who will push back and predict exactly what they'll say; write the paragraph that acknowledges the previous failure and explains why this approach is different; define what "success" means in a way your CEO and your lead engineer would both agree with.

The second column is your job. If it's empty, either you haven't thought hard enough or the project doesn't need a PM.

The document audit. Look at the last three significant documents you produced. For each one: How many words did AI write? How many words did only you know? If the ratio is more than 80/20 in AI's favor, spend one hour adding the context only you have. Share that version instead of the AI draft. Notice the difference in how people respond.

Chapter 6 stops describing the job and asks you to test whether you like it. Not conceptually. Actually.