AI Team or AI in Every Team? The Org Design Decision
Do you build a dedicated AI team, or push AI into every existing team? The wrong answer costs you a quarter. Here is the decision matrix and the three patterns that actually work.
Bottom line. The question "should we have an AI team?" has three defensible answers depending on the size and structure of your organisation, and exactly one wrong answer — "a committee." Smaller organisations (under ~150 people) usually win by pushing AI into every team with one named champion. Larger organisations usually need a shared-services AI platform team plus embedded roles. The middle path — a central AI team that owns features end-to-end — works in specific cases but fails more often than it succeeds. This briefing gives you the matrix for picking, the specific org patterns that work in 2026, and the failure mode that eats the most calendar time when leaders get this wrong.
Module L2 opens here because org design is downstream of the Module L1 exposure work. Once you know your quadrant and your budget envelope, the next question is structural: who inside your organisation is doing the AI work, and how are they connected to the teams shipping the product? This is a one-time decision with long consequences — changing it later costs months — so it deserves the 20 minutes of careful thinking this briefing asks for.
Why the choice matters more than it sounds
Start with the failure modes on either extreme, because naming them clarifies why the middle is hard.
The "dedicated AI team" extreme: a separate team owns all AI features. It has its own tech stack, its own vendor relationships, its own eval sets, its own roadmap. The rest of the product organisation sends requests to this team and waits in the queue. The team builds features that they find interesting, not what the business needs most. Integration with the rest of the product is brittle because the team doesn't share context with the teams they depend on. Six months in, there's a long backlog, slow shipping, and frustration in both directions.
The "AI in every team" extreme: no dedicated AI team at all. Every product team is expected to adopt AI capabilities as part of their own roadmap. In theory, this distributes AI work close to the problem. In practice, without shared infrastructure and patterns, every team reinvents the wheel — each builds their own prompt management, their own eval set, their own vendor integration, each re-discovers the same failure modes independently, and nobody owns the cross-team AI platform work (observability, compliance, cost optimisation) because it isn't anyone's explicit responsibility. Six months in, there are twenty half-shipped AI features, no shared infrastructure, and nobody who can answer "what's our AI strategy?"
The "committee" extreme: leadership declares AI strategy important, creates an AI Steering Committee or AI Council with cross-functional representation, holds biweekly meetings, produces decks, and ships nothing. This is the worst outcome and it is the most common default because it requires the least commitment. Committees can discuss AI strategy indefinitely without ever producing a decision. If your current org pattern is "an AI committee," your current org pattern is strategy theatre — see L1.1. Kill it.
The correct structures sit between the two extremes, and the right choice depends on your size and exposure.
Three patterns by size, one pattern to avoid. Let me walk through each working pattern with the specific roles, responsibilities, and failure modes, because the size bucket is only the starting point.
Pattern 1: AI everywhere, one named champion (under ~150 people)
The shape: no dedicated AI team. AI is a capability every engineer, PM, and designer is expected to use and apply to their own work. One named senior person — typically a senior engineer or staff engineer, sometimes a VP or CTO — serves as the organisation's AI champion: the person who maintains shared patterns, reviews cross-team AI decisions, keeps the eval-set conventions current, and owns the vendor relationships.
Why it works at this size: a company of 80 engineers and 30 other staff can coordinate informally. The champion knows everyone's work. Teams ask the champion directly when they have questions. The cost of a dedicated AI team — extra headcount, coordination overhead, siloing — outweighs the benefit at this scale. The champion's part-time work is sufficient to maintain shared patterns without bureaucratising them.
Specific roles and responsibilities:
- The AI champion (one person, 30-50% of their time): reviews significant AI changes across teams, maintains shared eval conventions, owns vendor contracts, briefs leadership quarterly on AI capability shifts, runs internal training sessions every 4-6 weeks.
- Every product team: owns its own AI features within its product surface area. Uses shared patterns and infrastructure from the champion. Escalates ambiguous decisions to the champion.
- Finance partner (roughly 10% of a finance lead's time): tracks AI inference spend across teams, flags outliers, produces monthly cost reports.
- Legal/compliance partner (roughly 10% of a legal lead's time): maintains the DPA templates and vendor compliance language.
No dedicated AI team. Four partial-FTEs at most. Everyone else does AI work as part of their day job.
When this works: the company has a single product or a tight product line. Engineering teams are small enough that coordination is informal. Leadership is prepared to have one senior person step up as the named owner.
When this breaks: the company grows past ~150-200 people, the product expands into multiple surface areas, or the champion leaves and no one inherits the role cleanly. All three trigger a transition to Pattern 2.
The specific failure mode: no champion is named. Leadership agrees AI is important, doesn't pick a specific person, and assumes "everyone will do their part." Everyone does their part on the narrow slice in front of them; nobody owns the cross-cutting patterns; the organisation ends up in the "twenty half-shipped features" state within a year. The fix is simple: name the champion in writing, with explicit allocation of their time and authority, within the first month of committing to this pattern.
Pattern 2: central platform, embedded roles (~150-800 people)
The shape: a small central AI platform team (typically 4-10 people) plus embedded AI roles in each major product team. The platform team owns shared infrastructure: prompt management, eval frameworks, observability, vendor integration, cost monitoring, guardrails. The embedded roles — usually a senior engineer or a tech lead who has AI as part of their charter — own the product-specific AI features and work directly inside the product team they belong to.
Why it works at this size: a company at this scale is too big for one champion to cover informally but not big enough to support multiple dedicated AI product teams. The platform team provides the shared infrastructure that keeps every feature from reinventing the wheel. The embedded roles keep the AI work close to the product context where it's needed. Together they balance centralisation (infrastructure, patterns, compliance) with decentralisation (speed, product context, team autonomy).
Specific roles:
- Platform team (4-10 people): 2-4 engineers building shared infrastructure, 1-2 ML/AI engineers maintaining eval frameworks and model selection, 1 product manager coordinating platform roadmap, 1 compliance/security partner, 1 tech lead.
- Embedded AI lead per product team (1 per major team, typically 4-12 teams): a senior engineer who owns the product team's AI features, consumes platform services, and escalates needs back to the platform team. These are existing senior engineers given AI as their explicit focus, not new hires.
- Central governance function (often owned by the platform team lead): quarterly AI strategy review with leadership, vendor relationship management, incident response coordination, compliance reporting.
When this works: the company has multiple product surfaces, the product teams are autonomous enough that decentralised AI work is possible, and there's enough shared infrastructure work to justify a dedicated platform team.
When this breaks: the platform team becomes a bottleneck (product teams wait weeks for platform services), the embedded roles are too junior to own their product's AI work, or governance becomes bureaucratic (the quarterly review becomes a status meeting instead of a decision forum). Each failure mode is addressable but requires active management.
Concrete 2026 examples: this is the most common pattern for mid-market SaaS and professional services firms with 300-800 people in 2026. Companies like Linear, Vercel, and many others have roughly this shape. The platform team is small and senior; the embedded roles are distributed; the coordination is lightweight.
The specific failure mode: the platform team builds for itself, not for the product teams. The platform team ships internal infrastructure that's technically impressive but not what product teams actually need, and product teams work around it with their own ad-hoc solutions. The fix is to measure platform adoption by product teams as the platform team's primary metric — not engineering velocity, not features shipped, but how many product teams are using the shared infrastructure and how satisfied they are with it. If the number is below 60% after six months, the platform is off-track and needs a re-scope.
Pattern 3: platform + product + governance (over ~800 people)
The shape: at larger scale, Pattern 2 expands into three distinct functions. The platform team grows to 10-30 people and owns infrastructure exclusively. Multiple AI product teams (or AI-focused pods within product teams) own specific customer-facing AI features end-to-end. A governance function — typically part of a broader risk/compliance/legal org — owns AI policy, vendor review, regulatory compliance, and incident response.
Why it works at this size: very large organisations have too many product surfaces and too many stakeholders for the platform-plus-embedded pattern to scale. You need dedicated AI product teams for the features complex enough to require full-time engineering. You need a governance function distinct from the builder function because the risks are too large to manage as part of another team's charter. The organisational complexity is the cost; the alternative is ad-hoc chaos.
Specific roles:
- Platform team (10-30 people): same charter as Pattern 2 but larger, with dedicated subteams for model operations, eval frameworks, cost management, and developer experience.
- AI product teams (1 per major AI feature, typically 3-10 teams): full-stack product teams that ship specific AI features end-to-end — engineering, PM, design, sometimes data science. Each team owns its features including their own eval sets, consuming platform services for infrastructure.
- Governance function (3-10 people across legal, security, compliance, and risk): policy writing, vendor risk review, incident response coordination, regulatory compliance (EU AI Act, sectoral rules), cross-team guardrail standards.
- Executive sponsor: at large scale, a specific VP or C-level person owns AI across the organisation, participates in quarterly strategic reviews, and serves as the escalation point for conflicts between platform, product, and governance.
When this works: very large enterprises, public companies, regulated industries, or companies where AI is core to the product strategy and multiple teams ship significant AI features.
The specific failure mode at this scale: governance and platform become adversaries. Governance says "slow down, we haven't reviewed this"; platform says "we need to ship." Product teams get caught in the middle. The fix is to have governance and platform report to the same executive sponsor, with a shared SLA on review turnaround times (governance reviews complete within N days of request) and a shared kill-switch authority (either can pause a launch, but both have to approve resumption). Without these mechanisms, the organisation ships slowly and takes more risk than it intends.
The decision matrix
Put your company through the two-axis matrix to figure out which pattern applies.
| Criterion | Pattern 1 (champion) | Pattern 2 (platform + embedded) | Pattern 3 (platform + product + governance) |
| Size | <150 people | 150-800 people | >800 people |
| Product surfaces | 1-2 | 2-6 | 5+ |
| AI investment | <$500K/year | $500K-$5M/year | >$5M/year |
| Regulatory exposure | Low to medium | Medium | High |
| Executive AI sponsor | Founder/CTO as champion | Designated VP | Dedicated C-level or SVP |
Most companies at a given size fit cleanly into one row. Edge cases — a 600-person highly regulated company, or a 200-person company with very heavy AI investment — can justifiably bend toward the next pattern up, but the bend should be deliberate and named. If you find yourself trying to fit into two patterns simultaneously, pick the larger one and over-invest slightly; it is cheaper than under-investing and catching up later.
A worked 2026 example: a 340-person B2B SaaS company with $65M ARR in the mid-market segment. They have 3 product surfaces (core product, admin dashboard, mobile companion) and are actively investing in AI features across all three. Their AI budget is ~$1.8M/year. They are not in a highly regulated sector.
Run the matrix: size says Pattern 2. Surfaces say Pattern 2. Investment says Pattern 2. Regulatory exposure is medium — still Pattern 2. Executive sponsor is a designated VP Engineering — Pattern 2.
The org design that fits: a 6-person platform team (2 platform engineers, 2 ML/AI engineers, 1 platform PM, 1 tech lead who doubles as executive partner). Embedded AI leads in each of the 3 product teams (existing senior engineers given AI as their focus). A compliance partner at 20% allocation. A quarterly AI strategy review owned by the VP Eng.
Total dedicated AI headcount: ~7 FTEs (the platform team) + 3 partial embedded roles + ~0.5 FTE in compliance + ~0.3 FTE in executive sponsorship ≈ 10-11 FTEs total. At a loaded cost of ~$200K/FTE, that's $2-$2.2M/year — slightly over the $1.8M AI budget, which means either the budget grows or the platform team starts smaller (4 instead of 6). Either answer is fine; what matters is the decision is explicit.
The failure mode: "AI Council"
The most common 2026 organisational failure mode, and it deserves its own naming: the AI Council / AI Steering Committee / AI Strategy Forum. A cross-functional group of senior leaders (eng, product, legal, sales, marketing, finance) meets biweekly to discuss AI strategy, vendor decisions, and prioritisation. The group reports to the CEO. Everyone on the council feels important. The council produces one deck per quarter with strategic themes and some high-level decisions.
This does not ship anything. Councils are decision-avoidance structures. They discuss; they don't build. They align; they don't ship. They satisfy leadership's instinct to "involve stakeholders" without actually owning the work. The companies I've seen run AI Councils for six months produced zero AI features directly attributable to the council's work. The companies that ship AI features have named owners (champion, platform team lead, or AI product team leads) who make decisions and are accountable for outcomes.
The specific defence: if your organisation has an AI Council, rename it or reorganise it. The council can survive as an advisory forum where the actual owners (from Patterns 1-3) update stakeholders and hear input. It cannot survive as a decision body because decision bodies without accountable owners don't decide.
The test: look at the last three decisions your council made. For each decision, ask "who is accountable for the outcome?" If the answer is "the council" or "leadership as a whole," the decision has no owner and will not ship. If the answer is a specific named person, the council is acting as a forum for a real owner, which is fine. The distinction is subtle in language and enormous in practice.
What to decide on Monday
- Pick your pattern this week using the size/surfaces/investment/regulatory matrix. 15 minutes.
- Name the specific people who own each role in the chosen pattern. For Pattern 1, name the champion. For Pattern 2, name the platform team lead and each embedded role. For Pattern 3, name the platform lead, the AI product team leads, and the governance owner.
- Write the ownership in a one-page org doc. Circulate it to your leadership team. Ask for objections. Resolve them within a week. Commit.
- If you currently have an AI Council, reorganise it into an advisory forum (not a decision body) in the same week. Decisions move to the named owners.
- Set quarterly review cadence with the executive sponsor. Not status meetings — decision meetings where the pattern's working metrics are reviewed and changes are committed.
- Budget for the pattern, not the aspiration. Pattern 2 at ~10 FTE costs real money; the budget has to fund it. Halfway-funded patterns are worse than fully-committed smaller ones.
- Revisit the pattern annually. Companies grow; org patterns have to grow with them. Moving from Pattern 1 to Pattern 2 or Pattern 2 to Pattern 3 is a deliberate decision, not an emergent one.
Next briefing, L2.2, gets deeper into the execution of these patterns: the three org archetypes that actually work in 2026, with specific named companies operating each one, the specific hiring plan that goes with each, and the one archetype (the committee) that fails by default.
Course navigation
| ⬅️ Previous | 📍 You are here | Next ➡️ |
| ⬅️ Previous L1.4 · Calibrating Your AI Exposure | L2.1 of L4.3 | Next ➡️ L2.2 · Three Org Patterns That Work |
📚 AI for Leaders · Course Home — 15 briefings, four modules.
Cover photo via Unsplash. This post is part of the AI for Leaders series.
