A bank I will keep anonymous had, at one point, three separate teams doing nearly identical work. Customer operations had built a chatbot to support call-center agents. Fraud analytics was experimenting with a model to summarize investigation reports. Digital banking had wired AI into the mobile app for personalized insights. Each initiative delivered localized value. None of them shared infrastructure, data pipelines, or governance. Multiply that pattern across a large enterprise and you get the signature condition of unmanaged AI adoption: plenty of AI activity, almost no AI capability.
The AI Center of Excellence exists to cure exactly this. Done well, it is the institutional mechanism that turns scattered experimentation into a coherent, scalable enterprise capability. Done badly, it becomes the most sophisticated bottleneck your organization has ever built. This article is about the difference.
What a CoE actually is
Strip away the branding and an AI CoE is a small central function with four jobs:
Platform. It provides the shared foundation: approved model access, security and privacy guardrails, reusable components, and monitoring capabilities that let teams track performance and catch risks. Teams build on this foundation instead of assembling their own stacks from scratch.
Standards. It defines how AI gets built here: evaluation requirements, documentation formats, deployment gates, and the risk classification that decides how much scrutiny a use case needs. Standards are what make fifty teams’ work legible as one program.
Enablement. It trains, advises, and accelerates. Office hours, patterns, playbooks, reference implementations, and embedded experts who parachute into business teams for the hard first mile.
Portfolio visibility. It maintains the enterprise view: what is being built where, what is working, what should be scaled, and what should be quietly retired. Mature programs review outcomes regularly rather than just reporting them; initiatives that work get scaled, patterns that prove reliable get reused rather than rebuilt.
Notice the verb pattern: provides, defines, trains, maintains. Not approves, not owns, not builds everything. The CoE is infrastructure for other people’s success.
The gatekeeper trap
The most common CoE failure is structural, and it is baked in at birth. Leadership, anxious about risk, gives the new center approval authority over every AI initiative. Every use case must pass through the CoE. Every model must be built or blessed by the CoE. It sounds like governance. It behaves like a tourniquet.
Here is the arithmetic that kills it. AI demand in a large enterprise runs to hundreds of use cases across every function. A CoE is typically fifteen to forty people. No plausible central team can gate that volume, so a queue forms, and the queue teaches the business a lesson: the official path is slow. Business units respond rationally. They buy SaaS tools with embedded AI, hire their own contractors, and route around the center. The CoE created to ensure quality becomes the reason AI happens without any oversight at all.
Figure 1 contrasts the two operating postures: the gatekeeper CoE, where all work flows through the center and stalls, and the enabler CoE, where the center pushes platform, standards, and skills outward while delivery stays in the business. The boxes look similar on a slide. The flow of work is the entire difference.
The enabler posture does not mean abandoning control. It means embedding control in the platform and the standards rather than in a human approval queue. When the paved road is genuinely faster than going off-road, teams choose it voluntarily, and your guardrails travel with them. Product teams retain ownership of their use cases while building on a shared foundation that ensures consistency, security, and efficiency. The CoE does not replace the teams; it makes them faster.
The charter: writing down what the CoE is not
Every CoE needs a charter, and the most valuable section is the one listing what the center explicitly does not do. A workable division for a hub-and-spoke enterprise looks like this.
The CoE owns: the platform and model gateway, enterprise AI standards and the risk-tiering scheme, the shared evaluation and monitoring tooling, literacy and enablement programs, and the portfolio view.
The business units own: use case selection and prioritisation within their domain, delivery teams and their funding, business outcomes and their measurement, and first-line operational accountability for the systems they run.
Shared, with named owners: risk acceptance for high-tier use cases (business owns the decision, CoE owns the assessment), talent standards (CoE defines role profiles, units hire), and incident response (unit runs point, CoE runs the pattern analysis).
Write it down, get it signed, and revisit it every six months, because the right boundary moves as the spokes mature. In year one, the CoE may genuinely need to build most solutions because nobody else can. By year three, a CoE still building everything is a warning sign, not a strength.
Staffing the hub
A CoE is a talent-dense function, and its composition tells you what it values. The core roster for a mid-sized enterprise hub:
A CoE lead who is a translator first and a technologist second, senior enough to hold the line against both over-eager business units and over-cautious risk functions. Platform engineers who treat internal teams as customers. A responsible AI lead who owns standards and risk-tiering. Solution architects who embed with business teams. An enablement lead who runs literacy programs and playbooks. And, increasingly in 2026, an agent operations specialist, because the shift from generative tools to autonomous agents is rewriting what “monitoring” means.
Keep it small and senior. A forty-person CoE doing delivery is a delivery team with a fancy name; the hub’s leverage comes from what it multiplies, not what it produces.
The metrics that keep a CoE honest
A gatekeeper measures itself by control: reviews completed, risks flagged. An enabler measures itself by adoption and velocity, and the difference shows up in the KPI list.
Time from idea to production for a standard-tier use case. Percentage of AI workloads running on the shared platform versus bespoke stacks. Number of teams shipping AI without CoE hand-holding. Reuse rate of shared components. Literacy program reach and proficiency lift. Incident rates on-platform versus off-platform, which is the number that justifies the platform’s existence to the CFO.
Figure 2 assembles these into the CoE flywheel: platform quality drives voluntary adoption, adoption concentrates workloads where guardrails live, concentration generates the usage data that improves the platform, and improvement drives further adoption. Every metric above is a sensor on one segment of that loop.
The flywheel framing also explains why the gatekeeper posture fails so predictably: approval queues break the loop at its first joint. Adoption under compulsion generates resentment instead of pull, and as Figure 2 implies, a flywheel with one seized bearing is just a heavy object.
Where CoEs go next
The CoE is not a permanent structure; it is scaffolding for a maturity journey. As we saw in the last article, the trajectory that leading enterprises are mapping runs from a strong central hub through domain-based hubs embedded in major business areas, toward an end state of AI-empowered teams operating independently on enterprise platforms. Along that path the CoE sheds delivery work, then advisory work, and concentrates into what never federates well: the platform, the standards, and the enterprise view.
Some organizations are also extending the CoE’s mandate upward, from managing models and platforms to orchestrating how AI integrates into enterprise workflows and decision-making, becoming, in effect, the architects of human-AI collaboration. That is a bigger idea than it first appears, and we will spend a whole article on it when we discuss managing a hybrid workforce later in this track.
For now, the practical takeaway is simpler, and Figure 1 is the test to apply. Look at your CoE, or the plan for one, and trace the flow of work. If everything flows through the center, you have built a gate and the queue is coming. If capability flows out from the center, you have built a hub, and the next question becomes what that hub actually serves up. That is the platform, and the platform deserves its own article. It is next.