Multi-Model Routing & Policy Orchestration

Executive Summary

This sub-scenario focuses on routing strategies for Planner/Orchestrator in multi-model, multi-tenant environments: automatically selecting primary and backup models based on task tags, cost, latency, and risk levels, supporting A/B testing and staged rollout, with quick rollback or degradation when anomalies occur, ensuring hit rate and experience.

Scope & Guardrails

• In Scope: Capability tags, policy configuration, decision API, fallback, safe mode, policy version management, rollback.
• Out of Scope: Provider onboarding (handled by Provider sub-scenario), cost billing details (handled by governance sub-scenario).
• Environment & Flags: multi-model-router, routing-safe-mode; depends on Capability Graph, Feature Flag, Telemetry.

Participants & Responsibilities

Scope	Repository	Layer	Responsibilities & Deliverables	Owners
planner-integration	powerx	integration	Decision API, trace, policy execution	Agent Platform Guild
policy-center	powerx	integration	Policy templates, versioning, audit	Agent Platform Guild
ops	powerx	ops	Monitor hit rate, trigger safe mode	Ops Reliability Center

End-to-End Flow

1. Planner inputs task tags → 2. Router selects primary/backup model based on policy weights and health scores → 3. Output decision/trace/cost estimation → 4. Monitor hit rate and fallback, roll back policy when necessary.

Key Interactions & Contracts

• POST /internal/model-routing/route, POST /internal/model-routing/rollback, EVENT agent.routing.policy.updated.
• Policy files: backend/config/agents/routing/*.yaml, config/policies/model-routing.json.

Acceptance Criteria

• Hit rate ≥90%, fallback success rate ≥95%; policy change to effect <5 minutes; safe mode activatable within 1 minute.

Telemetry & Ops

• Metrics: agent.routing.hit_rate, agent.routing.fallback_total, agent.routing.policy_rollback_duration.
• Alerts: Hit rate drop, fallback failure, latency threshold exceeded.

References

• docs/meta/scenarios/powerx/agent-and-automation/agent-model-platform/primary.md
• backend/config/agents/routing/*.yaml