Agentic Ai Maturity Assessment Extending The 18 Domain Model

Level 2: AI Transformation Practitioner Module M2.2: Advanced Assessment Methodology Article 11 of 10 11 min read Version 1.0 Last reviewed: 2025-01-15 Open Access

COMPEL Certification Body of Knowledge — Module 2.2: Advanced Maturity Assessment and Diagnostics

Article 11 of 12

The COMPEL Enterprise AI Maturity Model provides organizations with a structured framework for assessing their AI capabilities across eighteen domains, from data governance and model management to organizational culture and ethical AI practices. This model has proven effective for evaluating traditional AI deployments — predictive models, classification systems, recommendation engines, and generative AI applications. However, the emergence of agentic AI introduces capabilities and risks that the existing eighteen domains do not adequately address. An organization may score at Level 4 (Advanced) across all eighteen domains and still be fundamentally unprepared for agentic AI deployment.

This article proposes the extension of the maturity model with a nineteenth domain — Agentic AI Capability — that specifically addresses the organizational, technical, and governance competencies required for autonomous AI systems. This extension preserves the existing model's structure and scoring methodology while adding the assessment dimensions needed for organizations deploying or planning to deploy agentic AI.

Why a New Domain Is Necessary

Gaps in Existing Coverage

The existing eighteen domains provide partial coverage of agentic AI requirements, but significant gaps remain:

Domain 4: Model Management addresses model lifecycle management but assumes static models that are retrained periodically. Agentic systems that learn from experience, maintain persistent memory, and adapt their behavior in real time (Module 1.2, Article 12: Agent Learning, Memory, and Adaptation — Governance Implications) require management capabilities beyond traditional model lifecycle approaches.

Domain 7: AI Governance covers governance frameworks but is oriented toward approval processes for model deployment. Agentic AI requires runtime governance — continuous monitoring and control of autonomous decision-making — that extends far beyond deployment-time approval.

Domain 12: Risk Management addresses AI risk but focuses on prediction errors, bias, and data quality. Agentic AI introduces action-space risks, cascading failure risks, and delegation risks (Module 3.4, Article 12: Agentic AI Risk Taxonomy and Enterprise Risk Framework Extension) that require distinct assessment criteria.

Domain 15: Human-AI Collaboration touches on human-AI interaction but does not address the specific design patterns for human oversight of autonomous systems — escalation protocols, autonomy calibration, and collaboration quality metrics (Module 2.4, Article 11: Human-Agent Collaboration Patterns and Oversight Design).

None of the existing domains adequately addresses tool governance, multi-agent coordination, action-space safety, or the unique cost dynamics of agentic systems. Rather than retrofitting these concerns into existing domains — which would dilute their focus and create assessment confusion — a dedicated nineteenth domain provides clean, comprehensive coverage.

Alignment with the Autonomy Spectrum

The proposed nineteenth domain is structured around the autonomy spectrum introduced in Module 1.4, Article 11: Agentic AI Architecture Patterns and the Autonomy Spectrum. Maturity levels within the domain correspond to the organization's readiness for progressively higher levels of agent autonomy:

Level 1 (Initial): Organization lacks awareness of agentic AI requirements; no dedicated governance or infrastructure.
Level 2 (Developing): Organization recognizes agentic AI as distinct from traditional AI; initial policies and pilot projects underway.
Level 3 (Defined): Organization has established governance frameworks, technical infrastructure, and evaluation practices for agentic AI; deploying agents at Autonomy Level 1-2.
Level 4 (Advanced): Organization has mature agentic AI capabilities with comprehensive governance, multi-agent coordination, and continuous improvement; deploying agents at Autonomy Level 2-3.
Level 5 (Optimizing): Organization leads in agentic AI practices with adaptive governance, cross-organizational agent interaction, and industry leadership; deploying agents at Autonomy Level 3-4 for appropriate use cases.

Domain 19: Agentic AI Capability — Assessment Framework

Sub-Domain 19.1: Agentic Architecture and Design

This sub-domain assesses the organization's capability to design and implement agentic AI architectures.

Level 1 indicators:

No dedicated architecture patterns for agentic AI.
Agents are built ad hoc using general-purpose LLM frameworks without structured design.
No distinction between single-agent, multi-agent, and hierarchical architectures.

Level 3 indicators:

Established architecture patterns for agentic AI with documented design guidelines.
Clear criteria for selecting single-agent, multi-agent, or hierarchical architectures based on task requirements.
Standardized planning loop implementations (ReAct, chain-of-thought) with consistent reasoning trace capture.
Defined agent role templates with specified capabilities, constraints, and interaction patterns.

Level 5 indicators:

Advanced architecture patterns including dynamic agent composition, adaptive orchestration, and self-healing agent networks.
Architecture decision framework that balances capability, governance, cost, and risk across the autonomy spectrum.
Continuous architecture evolution informed by operational performance data and emerging best practices.

Sub-Domain 19.2: Tool Governance and Permission Management

This sub-domain assesses how the organization manages agent access to external tools and systems.

Level 1 indicators:

Agents have ad hoc tool access with no formal permission management.
No distinction between read-only and state-modifying tools.
Tool invocations are not logged or audited.

Level 3 indicators:

Formal tool permission framework with tiered access levels (unrestricted, logged, constrained, approved, prohibited).
Principle of least privilege applied to agent tool access.
Comprehensive logging of all tool invocations with sufficient detail for audit and debugging.
Tool schema validation preventing parameter construction errors.

Level 5 indicators:

Dynamic permission management that adjusts tool access based on context, track record, and policy.
Automated anomaly detection in tool use patterns.
Tool governance integrated with enterprise identity and access management systems.
Continuous optimization of tool sets based on usage analysis and effectiveness metrics.

Sub-Domain 19.3: Safety and Containment

This sub-domain assesses the organization's safety infrastructure for autonomous AI.

Level 1 indicators:

Safety relies solely on prompt-level instructions.
No technical enforcement of safety boundaries.
No escalation protocols for agent operations.

Level 3 indicators:

Multi-layer containment architecture (prompt, application, infrastructure, data, monitoring).
Defined escalation protocols with clear triggers, context preservation, and response handling.
Kill switch capabilities for immediate agent shutdown.
Regular adversarial testing (red-teaming) of safety boundaries.

Level 5 indicators:

Adaptive safety boundaries that adjust based on real-time risk assessment.
Formal verification of critical safety properties.
Cross-organizational safety protocols for multi-enterprise agent interactions.
Industry-leading safety practices that contribute to standards development.

Sub-Domain 19.4: Agent Evaluation and Monitoring

This sub-domain assesses how the organization evaluates and monitors agentic AI performance.

Level 1 indicators:

Agent evaluation uses only outcome-based metrics (task completion yes/no).
No behavioral assessment of agent reasoning or decision-making.
No continuous monitoring of deployed agents.

Level 3 indicators:

Multi-dimensional evaluation framework covering goal achievement, behavioral quality, efficiency, and safety (Module 1.2, Article 11: Evaluating Agentic AI — Goal Achievement and Behavioral Assessment).
Continuous monitoring of deployed agents with defined alert thresholds.
Regular human evaluation of agent reasoning quality and safety behavior.
Benchmark suites specific to organizational agent use cases.

Level 5 indicators:

Predictive monitoring that identifies potential failures before they occur.
Automated behavioral drift detection with root cause analysis.
Cross-agent performance optimization using operational data.
Contribution to industry-standard agentic AI benchmarks.

Sub-Domain 19.5: Human-Agent Collaboration

This sub-domain assesses how the organization designs and manages human oversight of autonomous agents.

Level 1 indicators:

No structured human-agent interaction design.
Agents operate either fully autonomously or fully under human control, with no calibrated oversight.
No UX patterns for agent monitoring or intervention.

Level 3 indicators:

Autonomy level classification for each agent deployment based on risk profile and reliability.
Defined HITL checkpoint design patterns with clear interaction protocols.
UX interfaces that enable effective human monitoring, understanding, and intervention.
Collaboration quality metrics that assess the effectiveness of human-agent partnerships.

Level 5 indicators:

Dynamic autonomy calibration that adjusts oversight levels based on agent performance and situational risk.
Sophisticated collaboration patterns that optimize the division of labor between humans and agents.
Continuous improvement of collaboration effectiveness through structured feedback mechanisms.
Research and innovation in human-agent collaboration that advances organizational capabilities.

Sub-Domain 19.6: Agentic AI Cost Management

This sub-domain assesses the organization's ability to understand and manage the costs of agentic AI.

Level 1 indicators:

No visibility into agentic AI costs beyond aggregate API spend.
No cost allocation to individual agents, tasks, or business units.
No cost optimization strategies specific to agentic AI.

Level 3 indicators:

Cost-per-task tracking for agentic operations with allocation to business units.
Understanding of cost drivers: planning loop depth, tool call frequency, context window utilization, reasoning token consumption.
Cost budgets and alerts for individual agents and agent groups.
ROI analysis comparing agentic AI costs to human or traditional automation alternatives.

Level 5 indicators:

Predictive cost modeling that forecasts agentic AI expenses based on planned deployments and usage patterns.
Automated cost optimization through dynamic model selection, caching, and reasoning depth management.
Cost-benefit optimization that balances quality, speed, and cost across the agent portfolio.
Industry benchmarking of agentic AI economics.

Cross-Domain Dependencies

The nineteenth domain does not exist in isolation. It has strong dependencies on several existing domains:

Domain 1: Strategy and Vision — Agentic AI capability should align with overall AI strategy. Organizations should not pursue agentic AI maturity for its own sake but as a means to strategic objectives.

Domain 3: Data Governance — Agent grounding, memory management, and tool data access all depend on robust data governance. An organization at Level 1 in Data Governance should not attempt Level 3 in Agentic AI Capability.

Domain 7: AI Governance — Agentic AI governance extends and depends on the organization's overall AI governance framework. The agentic domain adds runtime governance requirements, but foundational governance must already be in place.

Domain 12: Risk Management — Agentic AI risk management builds on the organization's risk management capabilities. The new risk categories introduced by agentic AI require existing risk management maturity as a foundation.

Domain 15: Human-AI Collaboration — The human-agent collaboration sub-domain extends the broader human-AI collaboration capabilities. Organizations with mature human-AI collaboration practices are better positioned to implement effective agent oversight.

Assessment guidance should flag cases where the Agentic AI Capability score significantly exceeds dependent domain scores. An organization cannot sustainably operate at Level 4 in Agentic AI if its Data Governance is at Level 2 — the governance gap creates risks that agentic capabilities cannot compensate for.

Conducting the Assessment

Assessment Methodology

The Domain 19 assessment follows the same methodology as the existing eighteen domains:

Evidence gathering. Collect documentation, system configurations, process descriptions, and performance data relevant to each sub-domain.
Stakeholder interviews. Interview technology teams, governance bodies, risk management, and business stakeholders with agentic AI involvement.
Technical review. Examine agent architectures, safety configurations, monitoring systems, and evaluation frameworks.
Operational analysis. Review agent performance data, incident logs, escalation records, and cost reports.
Scoring. Apply the maturity level criteria to each sub-domain, using the highest level for which all indicators are satisfied.
Gap analysis. Identify the highest-priority gaps between current maturity and target maturity.
Roadmap development. Create an improvement plan that addresses gaps in priority order, accounting for cross-domain dependencies.

Common Assessment Findings

Based on early assessments using this framework, common findings include:

Organizations typically score 1-2 levels lower in Domain 19 than in their overall AI maturity, reflecting the novelty of agentic AI.
Safety and containment (Sub-Domain 19.3) is the most common area of significant under-investment.
Cost management (Sub-Domain 19.6) is frequently at Level 1 even in organizations with sophisticated agentic deployments.
Human-agent collaboration (Sub-Domain 19.5) maturity strongly predicts successful agentic AI outcomes.

Key Takeaways

The existing 18-domain maturity model provides partial but insufficient coverage of agentic AI requirements, necessitating a dedicated 19th domain.
Domain 19: Agentic AI Capability encompasses six sub-domains: architecture and design, tool governance, safety and containment, evaluation and monitoring, human-agent collaboration, and cost management.
Maturity levels align with the autonomy spectrum — higher maturity enables safe deployment of more autonomous agents.
Cross-domain dependencies mean that agentic AI maturity cannot sustainably exceed maturity in foundational domains like data governance and risk management.
Assessment follows the established COMPEL methodology with evidence gathering, stakeholder interviews, technical review, and gap analysis.
Safety and containment and cost management are the most common areas of under-investment in early agentic AI deployments.