How I rolled out an AI automation stack for my Product Team and saved 30% of our working time

Building an AI Automation Stack for Your Product Team: Lessons From a Year of Trying

For most of my career as a product manager, I assumed the boring parts of the job were just the cost of doing it. Writing the same ticket templates. Drafting rollout documents in the same format. Producing weekly status updates. All of it necessary, none of it interesting

Then I spent a year building an automation stack for my team and learned that a meaningful chunk of this work can be handed off to AI tools. Not all of it, and not without tradeoffs. But enough that I want to share what worked, what failed, and what I would do differently if I started over

This article is for product leads thinking about rolling out AI tools to their teams. I will cover the four skills my team actually uses, the experiments that did not work, and the lessons I learned about building tools versus driving adoption

The Stack

Before going into specific tools, here is the high-level setup.

I use Claude Code as my main interface for building custom skills. These are small markdown-based configurations that teach Claude how my company does things: which ticket project to use, how our PRDs are structured, where our experiment documents live, and so on. For workflows that need to chain multiple steps or handle webhooks, I use n8n self-hosted via Docker. Several of my pipelines use the Claude API directly when I need different agents with different personalities.

For integrations, I rely on MCP connectors for our YouTrack ticket system and Confluence documentation. These let Claude read and write inside the actual tools my team uses every day. The total monthly cost across all of this is under fifty dollars.

Tool 1: Feature Breakdown Skill

This is the most-used skill on my team. The workflow is simple: a PM pastes a feature specification into Claude Code, and the skill creates the design, frontend, backend, A/B test, and analysis tickets in YouTrack with the correct parent-child relationships. It also posts a Confluence summary page with all task links and a one-paragraph problem statement.

Before this skill, creating the full ticket structure for a new feature took between ninety minutes and four hours, depending on the complexity of the spec and how many times I context-switched. After deploying the skill, the same task takes around ten minutes of review before assigning owners.

The skill also produces more consistent tickets. Before, ticket structure varied depending on who wrote them and when. Now every ticket follows the same template, which means engineers and designers can read them faster.

There were two challenges I underestimated when rolling this out to the team. First, my initial version was hardcoded to my specific YouTrack project and Confluence space. Making it work for other PMs required several rounds of parameterization. Second, even after the skill worked for everyone, adoption was not uniform. Some PMs used it from day one. Others took a month to try it. One never adopted it at all.

Looking back, this is a normal pattern for any internal tool. Optional tools get used by the people who needed them in the first place. Mandatory tools generate resistance.

Tool 2: A/B Rollout Skill

When an experiment ships with positive results, the next step is documenting the rollout: defining phases, creating tickets for the rollout work, and updating the experiment tracker. This is a structured but tedious process that takes about twenty minutes per experiment.

I built a Claude Code skill that reads the experiment design document from a fixed Confluence parent page, then generates the rollout plan using our standard template. The output goes into Confluence with the right tags and the right linked tickets.

The time savings here are modest, around fifteen minutes per rollout. But the consistency benefit is significant. Every rollout document now has the same structure, the same sections, and the same level of detail. New team members can read any rollout document in under a minute and find what they need.

This was the easiest skill to roll out across the team. I think this is because it did not ask anyone to learn a new tool. It just made an existing artifact look like everyone else’s.

Tool 3: Multi-Agent Landing Page Generator

This is the most ambitious tool I built, and the one that taught me the most about multi-agent systems.

The use case is generating quick landing pages for experiment hypotheses. When my team wants to test a new value proposition or pricing variant, we often need a simple landing page within a few days. Building these by hand involves a product brief, a copywriter, a designer, and a developer. Even a small page takes a week.

The pipeline runs in n8n with five agents in sequence:

1. Product Agent: takes a one-line product idea and expands it into positioning, target audience, and value propositions.
2. Copywriter Agent: writes the headline, subheadline, three feature blocks, and an FAQ.
3. UX Agent: proposes section order and layout.
4. Developer Agent: outputs the HTML and CSS.
5. Reviewer Agent: checks the result against the original brief and our brand voice document. Refuses to approve drafts that do not match.

The first version of this pipeline had four agents and no Reviewer. The output was unusable. The copy did not match the brand voice. The layout choices were inconsistent. The HTML had inline styles everywhere.

Adding the Reviewer was the key insight. By giving it a single job, comparing output to brief and rejecting if it does not match, I solved most of the quality problems without making the upstream agents more complex. The Reviewer typically rejects the first draft of the copy and requests revisions. The final output is usable after two or three iterations.

End to end, the pipeline produces a usable landing page in about ten minutes. The team uses this whenever a customer development call requires a quick page that did not exist the day before.

Tool 4: Skeptic Agent

This last tool is just for me. I keep a Notion page of startup ideas. Before I let myself get excited about a new idea, I send it to a skeptic agent.

The agent has one job: argue against the idea. It pushes back on market size, customer acquisition cost, whether the problem is actually painful enough to drive purchasing behavior, and what the unit economics look like at scale.

Getting this agent to actually push back required several iterations. Early versions just agreed with whatever I said, even when I explicitly told them to disagree. What worked was providing concrete examples of bad ideas paired with the kind of skeptic responses I wanted, plus a system prompt that named excessive agreement as a failure mode.

I have used this agent on roughly thirty ideas in the past year. Most of them did not survive the conversation. The ones that did emerged sharper and clearer because they had been stress-tested before I invested any real time.

I tried to roll out a similar tool for my team to evaluate product hypotheses, but it did not work. PMs did not use it. My best guess is that people accept a model challenging their personal startup ideas because the stakes are low, but they do not accept it challenging their work decisions because work decisions already have a real manager attached.

What Did Not Work

Not every experiment succeeded. A few examples of tools I built and then abandoned:

A status update generator that reads recent YouTrack activity and drafts the weekly Slack update. The output was technically correct but obviously machine-written. My team noticed within two weeks. I went back to writing the updates myself.

A planning agent that was supposed to look at my calendar, open tickets, and a priority list, then tell me what to work on next. It produced reasonable-sounding suggestions but I never trusted them. Every time I disagreed with the agent, I spent ten minutes reading its reasoning to find the flaw. That is more work than just deciding myself.

A roadmap prioritization helper for the team. Technically functional, but it ignored the organizational dynamics that actually drive prioritization decisions. The lesson here is that you cannot automate work that requires reading the room.

Lessons for Product Leads

If you are considering rolling out AI automation to your team, here are the patterns that held across all my experiments.

• Build for yourself first, then generalize. I lost a month trying to design a tool that would work for everyone from day one. The right pattern is to build a hardcoded version that works perfectly for you, use it for two weeks, then parameterize for the team.

• Start with the most boring task. Tickets are political. PRDs are personal. Find the dullest, most mechanical paperwork your team produces and automate that first. Adoption will be smooth because no one cares enough to resist.

• Do not measure success by adoption rate. Some PMs on my team use every skill I built. Others use none. Both outcomes are fine, as long as the team’s overall output is more consistent. The PMs who do not adopt are not failing. They have their own systems that work for them.

• Accept that you cannot automate political work. Anything that involves stakeholder dynamics, performance reviews, or strategic prioritization will produce technically correct but practically useless output. Save your effort for the work where the rules are clear.

• Adoption requires deadline pressure. Demos do not drive adoption. People adopt new tools when they have a deadline and the new tool is the fastest path to meeting it. Wait for the moment, then provide a clear link.

Conclusion

A year of building AI tools for my product team produced real productivity gains: roughly a day a week saved per PM, more consistent documentation, and faster onboarding for new hires. But the larger lesson was that the technical work of building tools is the easy half. The harder half is convincing people to use them.

The tools that succeeded were the ones that solved a clear pain point, integrated with existing workflows, and did not require anyone to change their habits. The tools that failed either tried to replace human judgment or made the user feel like they were being managed by software.

If you are a product lead considering this work, start small, build for yourself first, and measure success by team consistency rather than adoption rate. The rest will follow.