An AI Center of Excellence (CoE) is a permanent organization within a company that consolidates AI-related expertise, standards, and governance, deploying them across the entire organization. While the Chief AI Officer (CAO) is an individual executive role and AgentOps is a specialized team for AI agent operations, the CoE refers to the cross-organizational promotion structure itself.

This article is intended for CIOs, DX promotion leads, corporate planning staff at headquarters, and local subsidiary heads of Japanese companies operating across multiple ASEAN locations. It explains the prerequisites for CoE design and four steps: mission definition, organizational structure, knowledge management, and governance. By the end of the article, readers will have a clear understanding of the key issues involved in defining the roles of headquarters versus local subsidiaries, clarifying relationships with existing IT/DX organizations, and launching a CoE in a multilingual, multi-regulatory environment.

A CoE is not "a department that gathers AI-savvy people" — it is a permanent infrastructure designed to make AI promotion reproducible as an organization across the entire company. Japanese companies operating across multiple ASEAN locations face structural challenges such as knowledge silos between sites, regulatory differences, and duplicated investments. Resolving these issues is the primary motivation for establishing a CoE.

CoE, CAO, and AgentOps are often confused as organizational models for AI promotion, but they differ in the granularity and nature of their roles.

All three coexist. The typical structure has the CAO acting as sponsor to establish the CoE, with the CoE overseeing AgentOps. If organizations move forward with "let's create a CoE" while still conflating these concepts, the CoE tends to be perceived on the ground as "yet another committee" or "an advisory body with no real execution power." The CoE must be designed as a substantive organization responsible for decision-making, execution, and governance.

The structural reasons why Japanese companies operating across multiple ASEAN locations need a CoE can be summarized in three points.

• Knowledge becomes siloed at each location — Operational know-how built around RAG implementations at a Thailand subsidiary never reaches Vietnam or Indonesia. The same mistakes are repeated in each country. Without a CoE, knowledge remains in individuals' heads and is lost when people resign or transfer.
• Governance becomes fragmented across locations — Personal data protection requirements differ by country: Thailand's PDPA, Vietnam's PDPL, Laos's Personal Data Protection Law, Indonesia's PDP Law, and Singapore's PDPA. When each location handles compliance independently, auditing and control from a headquarters perspective becomes extremely difficult.
• Investment duplication and gaps emerge — Multiple locations independently launch proofs of concept on the same business theme, or conversely, no one invests in critical areas. Both represent a double loss, caused by the absence of a central entity to manage the investment portfolio across locations.

The CoE is the organizational answer to these three structural challenges. The goal is not simply to "create a department strong in AI," but to build a platform across three axes: knowledge circulation, regulatory compliance, and investment allocation.

There are four typical patterns by which a CoE becomes a hollow formality after launch.

1. Headquarters-driven to the point of not resonating locally — Standards and templates decided at headquarters do not fit the operational context of local sites, so local subsidiaries end up submitting reports only as a formality. No meaningful adoption occurs.
2. Advisory function only, with no execution capability — The CoE only provides "advice" and "guideline presentations" without holding budget, personnel, or decision-making authority, making it unable to drive action on the ground.
3. No executive sponsor, leaving the CoE's organizational standing ambiguous — Because none of the CIO, CDO, or CEO has clearly taken on the sponsor role, the CoE is consistently at a disadvantage when negotiating with other departments.
4. No KPIs, leaving activities invisible — With no way to measure activity volume, business value, or standardization progress, the CoE is perceived as "a department no one understands," and its budget gets cut in the following year.

These issues do not stem from individual operational problems, but from a failure to address the key issues during the initial CoE design phase. The four steps covered in the following sections are intentionally designed to structurally avoid these four patterns.

Before embarking on CoE design, three constraints specific to Japanese companies must be clarified. The balance between headquarters-led direction and local autonomy, the relationship with existing IT/DX organizations, and the handling of multiple languages, currencies, and regulatory frameworks——without establishing these as prerequisites, applying a generic CoE framework as-is will not work.

Forcing the AI promotion activities that are beginning to take shape at a Thai subsidiary into a headquarters-driven framework instantly kills the local sense of speed——this is a failure pattern seen repeatedly on the ground at Japanese companies advancing into ASEAN with DX initiatives. Conversely, granting full local autonomy undermines governance and creates risk management issues for headquarters.

The balance between headquarters and local entities is easier to manage by dividing responsibilities as follows.

• Areas owned by the headquarters CoE — Company-wide standards (PoC design, vendor selection, common data handling clauses), cross-functional governance (investment review, risk review), global talent development, overall budget allocation.
• Areas owned by local subsidiaries — Use case selection (requiring an understanding of local operations), PoC execution speed, negotiations with local vendors, local-language content and knowledge.
• Areas owned jointly — Personal data protection compliance (headquarters policy + local regulations), go-live decisions, localization of training programs.

Rather than a binary opposition of "headquarters decides / local decides," responsibilities should be divided by topic. Explicitly documenting boundaries and deliberately reducing gray areas leads to fewer long-term frictions.

Most Japanese companies already have IT departments, DX promotion offices, information systems divisions, global IT strategy departments, and similar functions. Failing to clarify the division of roles between these entities and the CoE at the time of its establishment simultaneously creates redundancy on the org chart and confusion on the ground.

The key relationships to sort out are as follows.

• Delineation from the existing DX promotion office — If the DX promotion office handles overall business transformation, the CoE is positioned as a sub-organization specializing in the AI domain. Alternatively, CoE functions can be integrated within the DX promotion office.
• Coordination with the information systems department — A practical division is for the information systems department to operate shared infrastructure (cloud, data platforms, authentication), while the CoE is responsible for AI-specific infrastructure, model management, and experimentation environments.
• Alignment with global IT strategy — CoE standards must be aligned with the technology stack and security standards defined by the headquarters' global IT strategy. If the CoE operates on its own track, long-term system integration costs will balloon.
• Delineation from the R&D department — AI technology validation in the research and development phase belongs to R&D, while business application belongs to the CoE. Where the boundary is ambiguous, a collaborative project format should be used.

An org chart and a RACI (Responsible / Accountable / Consulted / Informed) matrix should be documented at the time of CoE launch and agreed upon with all relevant departments.

In multi-site ASEAN operations, each country's language, currency, and regulatory framework operate in parallel. CoE design must incorporate mechanisms to absorb these differences from the outset.

The key issues can be summarized as follows.

• Multilingual knowledge management — A practical approach is to maintain common documents with Japanese or English as the master language, supplemented by local-language summaries. If each site independently maintains knowledge in Thai, Vietnamese, or Indonesian, searchability from headquarters deteriorates significantly.
• Differences in personal data protection laws — Thailand's PDPA, Vietnam's PDPL, Laos's personal data protection law, Indonesia's PDP Law, Singapore's PDPA, the Philippines' Data Privacy Act, and Malaysia's PDPA each have different provisions regarding consent acquisition, cross-border data transfers, and data subject rights. The CoE should consolidate the liaison function with each country's legal team and manage common requirements and country-specific requirements separately.
• Investment management across multiple currencies — A mechanism is needed for headquarters to track PoCs conducted in local currencies such as the Thai Baht, Vietnamese Dong, and Indonesian Rupiah in Japanese Yen or US Dollar terms. The impact of exchange rate fluctuations should be visualized in monthly reports.
• Differences in time zones and public holidays — Cross-functional review meetings should be held on fixed days and times, and an annual schedule that accounts for each country's public holidays should be created at the outset.

These issues will break down if left to "local sites to handle as needed." They must be documented as operational rules at the time of CoE launch and embedded into tools and operational workflows.

In Step 1, the mission, scope, and executive sponsor are documented. By deciding upfront "what the CoE does and what it does not do," unnecessary friction with related departments and misaligned expectations within the organization can be avoided.

Setting the CoE's mission too broadly as "everything related to AI" disperses activities and leaves every initiative half-finished. Conversely, narrowing the scope too much causes the organization to become dysfunctional. By explicitly defining what the CoE does and does not cover, boundaries with related departments become clear.

Examples of areas the CoE covers:

• Company-wide standards (PoC design templates, data handling policies, vendor evaluation criteria)
• Cross-functional governance (investment reviews above a certain threshold, risk assessments, duplication checks)
• Talent development (internal training, external partnerships, internal community management)
• Direction-setting for shared infrastructure (model management, evaluation environments, knowledge DB)
• Aggregation and reporting of cross-functional KPIs

Examples of areas the CoE does not cover:

• Individual operational improvement proposals (handled by business unit and local subsidiary improvement teams)
• Full-scale system development and operations (handled by the IT department and AgentOps teams)
• Pure research and development (handled by the R&D division)
• Independent execution of individual PoCs (business units take the lead; the CoE plays a support role)

Explicitly defining what the CoE does not cover prevents it from becoming a "catch-all utility organization." Keeping the mission to a sustainable size is the most critical decision in the initial design.

A CoE without executive sponsorship will always be at a disadvantage in budget negotiations and decision-making. Given its inherently cross-functional nature spanning multiple business units, the organization cannot function without strong sponsorship.

Key points to confirm when designing sponsorship are as follows:

• Sponsor's title and authority — CEO, CIO, CDO, or COO is preferred. Without executive-level authority, the CoE is often outmaneuvered in negotiations with other business units.
• Time the sponsor commits to the CoE — Secure specific time commitments, such as a monthly review meeting, a quarterly executive committee report, and an annual policy review.
• Escalation path — Establish in advance a clear route for escalating to the sponsor issues the CoE cannot resolve on its own (inter-departmental conflicts of interest, major investment decisions, withdrawal decisions).
• Communication responsibility — The sponsor takes on the role of communicating the CoE's importance both internally and externally. Sponsor-driven messaging raises organizational awareness far more effectively than self-promotion by the CoE.

Launching a CoE without clearly defined sponsorship typically results in it being perceived as a "nominal advisory body" within six months to a year, necessitating a relaunch. Avoiding this pitfall at the initial design stage is what determines the CoE's long-term success or failure.

For multi-site operations across ASEAN, the central question in organizational design is whether to adopt a hub-and-spoke model or a federation model. Neither is universally correct; the optimal choice depends on company size, number of sites, and the strength of headquarters governance.

The following table summarizes the differences between the two representative models.

Selection guidelines:

• Small-to-medium site scale, HQ governance prioritized → Hub-and-spoke model
• Large site scale, local speed prioritized, significant variation in national regulations → Federation model
• A phased approach—starting with hub-and-spoke and transitioning to federation as the organization matures—is also a practical option

Japanese companies in the early-to-mid stages of ASEAN expansion frequently start with the hub-and-spoke model.

Regardless of whether the hub-and-spoke or federation model is chosen, the design of the interface between the HQ CoE and local subsidiaries is what determines the CoE's effectiveness.

The key interfaces to design are as follows:

• Regular communication — Monthly reviews (all sites participating, conducted in English or with Japanese-English interpretation), weekly operational meetings (internal to the CoE), and quarterly sponsor reports.
• Knowledge-sharing platform — An internal tool for centrally managing a use case registry, PoC reports, and template libraries. Searchability and access control settings are critical.
• Escalation path — A clear route for escalating from the HQ CoE to the sponsor for matters that local subsidiaries cannot decide on their own (major investments, vendor selection exceptions, regulatory risks), with indicative turnaround times specified.
• Consideration for language and time zones — Standardize on a common language (English or Japanese) and have the CoE provide interpretation and translation support. Schedule meetings during hours when core working times overlap across countries (e.g., 14:00–17:00 Thailand time for Thailand and Japan).
• Role definition for liaison staff — Document the mission and performance indicators for the CoE liaison assigned to each site. Also specify the recommended ratio of this role to primary responsibilities (recommended: 20–40%).

Leaving the interface design ambiguous makes it easy for HQ and local sites to end up "unknowingly heading in completely different directions."

Step 3 involves establishing knowledge and playbooks as core CoE assets. Making the insights held in individuals' minds reproducible at an organizational level is the very reason for the CoE's existence.

Use case management is the core function through which the CoE makes AI investments visible across the organization. By consolidating use cases progressing independently across individual sites and business units into a single ledger, it becomes possible to identify duplication, gaps, and opportunities to roll out successful initiatives horizontally.

Minimum required fields for the use case ledger:

• Basic information: ID, title, business domain, target site, start date
• Status: Under consideration / In PoC / In pilot / In production / Retired
• Ownership: Business owner, IT lead, vendor, CoE liaison
• Investment information: Cumulative investment, expected ROI, payback status
• Outcomes: Quantitative impact (processing time, cost reduction), qualitative impact, number of active users
• Risks: Legal risk, operational risk, external dependency risk
• Horizontal rollout recommendation: Whether replication at other sites is feasible, and the prerequisites for doing so

The ledger should be reviewed quarterly, and retirement decisions should be prompted for stalled use cases (those with no status change for six months or more). If the ledger becomes stale, the overall credibility of the CoE's governance function will be undermined.

The use case ledger is the most foundational and critical asset of the CoE. It is recommended that operational workflows be defined before tool selection.

For horizontal rollout to proceed effectively, it is a prerequisite that each site operates using the "same format." By having the CoE develop standard templates and making them reusable across sites, the cycle of knowledge sharing accelerates.

Templates to be developed:

• PoC design template — Business problem definition, scope, success criteria, evaluation data design, go/no-go gate for productionization (customized from the PoC design guide into an internal version)
• Vendor selection checklist — Technical evaluation, contract terms, data handling, support structure, local regulatory compliance
• Standard data handling clauses — Personal data protection clauses to be included in vendor contracts (covering Thailand PDPA, Vietnam PDPL, etc.)
• ROI evaluation template — Calculation logic for investment amount, expected impact, and payback period, including treatment of currency fluctuations
• Risk assessment template — Checklist items covering legal, operational, security, and ethical risks
• Productionization plan template — Architecture, operational structure, KPIs, retirement conditions
• Training program — Three-tier structure for executives, business owners, and frontline users

Templates should be developed to a level where "simply filling them in achieves a baseline level of quality." Attaching a usage guide and completed example to each template makes independent adoption by local sites easier.

Templates are living documents. Collect feedback from sites using them once per quarter and revise accordingly. Maintain a revision history so that differences from previous versions are clearly visible.

Step 4 involves designing a cross-functional review process and KPIs to make CoE activities measurable at an organizational level. A CoE in which governance and impact measurement are not functioning will, within six months to a year, come to be perceived as "a department no one understands," and its budget will be cut.

For the CoE to function effectively, the cross-functional review process must be formally documented, and a mechanism must be established through which AI projects at each site and business unit pass through defined checkpoints.

Key review processes:

• Investment review — AI investments above a certain threshold (e.g., ¥3,000,000 / USD 100,000) require mandatory CoE review. Duplication, alternatives, and ROI are assessed.
• Risk review — AI systems that handle personal information, sensitive data, or are exposed externally must pass a risk review in advance. Legal, security, and ethical perspectives are integrated into the review.
• Vendor review — A vendor evaluation is conducted when onboarding a new vendor. Existing contracted vendors undergo an annual review.
• Duplication check — When a new PoC is proposed, the CoE checks for overlap with existing entries in the use case ledger. Where duplication is found, participation in the existing project is recommended.
• Exception request process — A route for bypassing the standard process in urgent or special circumstances. Sponsor approval is required, and post-hoc reviews make the accumulation of exceptions visible.

The review process should be operated as "quality assurance," not as a "brake." Response times to approval should be stated explicitly as SLAs (e.g., investment reviews completed within two weeks), balancing speed with governance.

Accumulating review logs and analyzing frequently occurring reasons for rejection provides input for improving templates and training programs.

Operating a CoE without KPIs makes its activities invisible to the organization. KPIs should be designed in a four-layer structure, with each layer combined to provide an overall evaluation.

Common pitfalls in KPI design:

• Chasing activity volume alone — Evaluating solely on metrics like PoC count or training participants creates an incentive to mass-produce PoCs that never reach production. Always pair these with a business value layer.
• Overstating business value — Continuously reporting "projected hours saved" as actual results causes the gap with measured values to widen. Establish from the outset a principle of reporting based on actual measurements.
• Deprioritizing risk KPIs — A CoE with strong activity volume and business value metrics but frequent incidents is an organizational failure. Always include the risk layer in management reporting.

KPIs should be agreed upon with executive sponsors at the time of CoE launch and reviewed quarterly. A CoE still using the same KPIs after three years is likely stagnating as an organization.

The following is a structured overview of common failures that frequently occur from the early launch phase through the operational phase of a CoE, paired with corresponding countermeasures.

1. Headquarters-driven templates become a formality at local sites — Local staff in Thailand, for example, accumulate frustration when forced to use PoC design templates that do not fit their operations. Countermeasure: Include liaison representatives from multiple sites as founding members during template development, and conduct local reviews at the beta stage.
2. The CoE becomes a catch-all "convenience store" — Due to an ambiguous mission, "bring all AI-related problems to the CoE" becomes the norm, scattering resources. Countermeasure: Review the "in-scope / out-of-scope" boundaries defined in Step 1 quarterly and communicate them across the organization.
3. Overly strict headquarters governance drives local teams to use workarounds — Slow or stringent approval processes lead local teams to run small-scale initiatives without authorization, increasing "shadow AI." Countermeasure: Publish explicit review SLAs and establish a simplified review track for small-scale projects.
4. KPIs fail to resonate with executive leadership — Reporting only activity volume KPIs fails to communicate business value. Countermeasure: Always include business value KPIs and translate them into financial metrics that CFOs and CEOs care about—revenue, cost, and customer satisfaction.
5. Liaison staff burn out from concurrent responsibilities — Local liaison staff exceeding an 80% concurrent workload ratio begin deprioritizing CoE activities. Countermeasure: Agree on a maximum concurrent workload ratio (e.g., 40%) between headquarters and local sites, and monitor it regularly.
6. CoE positioning becomes unstable when sponsors change — When executive sponsors are replaced due to organizational changes, support for the CoE can be lost. Countermeasure: Prepare a sponsor transition package (past achievements, current challenges, one-year forward plan) in advance during normal operations.

Q1: What is a realistic team size for launching a CoE?

In the early launch phase, it is common to start with 3–5 dedicated members and 5–10 members including those in concurrent roles. The basic composition consists of a leader (full-time), a standards and knowledge officer, a governance officer, a training officer, and site liaison representatives from each location (20–40% concurrent). A large-scale CoE of 30 members is more realistically positioned as a target for a mature phase of AI advancement within the organization—typically 2–3 years after launch.

Q2: Which department should the CoE report to?

Reporting directly to the CIO, CDO, or CEO is preferable. Embedding the CoE within a business unit or IT department weakens its cross-functional coordination capacity. Placing it within an IT systems department causes it to be perceived as "IT work," making it harder to gain cooperation from business units. Positioning it directly under executive leadership ensures it can oversee multiple business units and local subsidiaries on an equal footing.

Q3: If AI initiatives are already underway at individual sites, is there still value in creating a CoE after the fact?

Yes, there is. In fact, when activities are already scattered across sites, the impact of consolidating cross-functional knowledge and eliminating duplication tends to be even greater. A practical approach for the early launch phase is to start with "cataloging existing projects" and "standardizing regulatory compliance," then gradually apply standards to new projects—a progressive rollout.

Q4: How should the CoE be designed when ASEAN sites such as Thailand, Vietnam, and Indonesia are at different stages of AI maturity?

Connect sites at different maturity levels using a hub-and-spoke model, and establish a mechanism for sharing knowledge from more advanced sites (often Thailand or Singapore) to less advanced ones. Prioritize providing PoC design templates and training programs to less advanced sites to shorten their initial ramp-up period. For advanced sites, explicitly include the development of horizontal rollout knowledge as part of their mission.

Q5: How many years are needed to judge whether a CoE has succeeded or failed?

While KPI evaluations are conducted quarterly, assessing the CoE as an organization requires a minimum span of 18–24 months. In the first 6 months after launch, focus on activity volume KPIs; at the 12-month mark, review trends in business value KPIs; and at 18–24 months, make an executive-level judgment on return on investment. Demanding results within less than a year tends to drive PoC mass-production behavior, obscuring the true value of standardization and knowledge consolidation.

For Japanese companies operating across multiple ASEAN locations to make their AI advancement structure reproducible as an organization, designing the CoE across four steps is effective. The key points covered in this article are summarized below.

• Step 1: Define mission and scope — Clearly document in-scope and out-of-scope areas, and designate an executive sponsor. Prevent the CoE from becoming a catch-all "convenience store."
• Step 2: Design organizational structure and role allocation — Select between a hub-and-spoke or federation model based on company size and number of sites, and design the interface between headquarters and local operations.
• Step 3: Consolidate knowledge and develop playbooks — Build a use case registry and a suite of standard templates to make horizontal rollouts reproducible across the organization.
• Step 4: Governance and performance measurement — Use a cross-functional review process and four-layer KPIs to make activities visible, and evaluate performance across both risk and business value dimensions.

The CoE is an organization whose value is difficult to measure through short-term performance indicators; however, it plays a structural role in resolving duplication, gaps, and siloing in AI investment across multi-site ASEAN operations. Allowing for an initial ramp-up period of 18–24 months and maintaining a posture of reviewing the design quarterly will determine long-term success or failure.

For related reading, see AI-Native Organizations and the Role of the Chief AI Officer on designing executive AI roles, What Is AgentOps — A Design Guide for AI Agent Operations Organizations on AI agent operations structures, and AI Governance Framework Construction Guide for Companies Expanding into ASEAN for an overview of AI regulations across ASEAN countries.