v4.2 Gateway Foundation Planning

v4.2.0 is prepared as the Gateway Foundation release after the v4.1 Skill OS Foundation release. This document records the architecture plan that shaped the release; npm publication remains manual and v4.1.0 remains npm latest until publication is confirmed.

1. Strategic Goal

MultiModel Dev OS is evolving from a workspace and configuration operating layer into a governed multi-provider AI gateway and orchestration platform.

The long-term goal is one local-first AI gateway that can sit between coding agents and model providers:

• One unified AI gateway endpoint.
• Multiple AI provider integrations.
• Model and provider routing.
• Fallback chains.
• Cost, capability, quota, and latency-aware selection.
• Compatibility with Codex, Claude Code, Cursor, Cline, Aider, Antigravity, MCP, and other agents.
• Token and context optimization.
• Local-first observability and governance.

The v4.1 Skill OS work remains central. Skills, prompts, workflows, permissions, guardrails, memory, adapters, validation, and business operator templates should become the governance and control plane around the future runtime.

2. Relationship to OmniRoute

OmniRoute is architectural inspiration only. MultiModel Dev OS should not copy OmniRoute code, branding, provider claims, provider counts, screenshots, proprietary design, or marketing language.

The relevant strategic lesson is the product shape: a single endpoint that lets many AI coding tools route through multiple model providers with fallback, cost awareness, token/context optimization, and observability.

MultiModel Dev OS should differentiate through:

• Skill OS governance.
• Portable workspace memory.
• Workflow metadata and validation.
• Adapter portability across AI coding tools.
• Declarative permissions and guardrails.
• Business/operator orchestration templates.
• Zero-lock-in local repository standards.
• Maintainer-controlled release and package governance.

3. Current Control Plane

The current repository already contains a strong control plane:

• Skill OS: JSON schemas, YAML registries, validation, and read-only skill-os inspection.
• Prompt templates: RACE+ prompt metadata under .ai/prompts/ and .ai/registries/prompt-templates.yaml.
• Skills: Markdown skills and Skill OS metadata under .ai/skills/ and .ai/registries/skills.yaml.
• Permissions: Declarative tool permission classes under .ai/registries/tool-permissions.yaml.
• Guardrails: Advisory-only guardrail metadata under .ai/registries/guardrails.yaml.
• Workflows: Workflow registry entries with optional skill_os references.
• Adapters: Codex, Claude, Cursor, Gemini, Antigravity, VS Code, and other adapter assets.
• Memory/intelligence: Hash-compressed memory, feedback, proposals, and learning-rule structures.
• Validation: Strict verify pipeline, Skill OS validation, registry signing/provenance checks, and package hygiene checks.
• Model metadata: Model, provider, local model, and routing preset YAML registries.

This layer is mostly declarative. It defines intent, policy, compatibility, metadata, and validation boundaries. It does not currently execute model requests or enforce live gateway behavior.

4. Remaining Runtime Plane Gaps

The prepared foundation now includes a localhost mock runtime, deterministic mock streaming, local observability, and preview-only client integration plans. It is still not a production multi-provider gateway.

Remaining gateway capabilities:

• External runtime provider adapters.
• Provider credential loading through explicit safe configuration.
• Real provider model execution.
• Runtime fallback chains.
• Runtime retry and rate-limit handling.
• Live quota tracking.
• External provider health checks.
• Persistent audit log design.
• Runtime integration with permissions and guardrails.

The existing .ai/models/ files are runtime-readable metadata and dry-run routing inputs. They are not yet live provider execution configuration.

5. Proposed Architecture

Control Plane

The control plane should define what is allowed, preferred, and explainable:

• Skill OS.
• Workflows.
• Policies.
• Permissions.
• Guardrails.
• Memory.
• Configuration.
• Registry validation.
• Adapter compatibility metadata.

The control plane should remain local-first, inspectable, and package-safe.

Routing Plane

The routing plane should decide where a request should go:

• Provider registry.
• Model registry.
• Route resolver.
• Scoring engine.
• Fallback planner.
• Quota strategy.
• Cost strategy.
• Latency strategy.
• Capability matching.
• Context-window matching.
• Privacy/local-first policy.

The first implementation should be deterministic and testable before any adaptive behavior is added.

Gateway Plane

The gateway plane should execute requests safely:

• HTTP API.
• Authentication.
• Request normalization.
• Streaming.
• Provider clients.
• Retries.
• Timeouts.
• Response normalization.
• Error normalization.
• Health endpoints.
• Usage accounting.

The gateway should bind locally by default. Non-local access should require explicit authentication and configuration.

Client Plane

The client plane should include AI coding agents and integrations that can point at the gateway:

• Codex.
• Claude Code.
• Cursor.
• Cline.
• Aider.
• Antigravity.
• MCP.
• Custom agents.

Client compatibility should be validated through documented configuration examples and protocol tests.

6. Proposed Source Structure

Recommended source layout:

src/
  gateway/
    server.js
    routes/
    middleware/
    protocol/
  providers/
    base-provider.js
    registry.js
    openai-compatible.js
  router/
    route-resolver.js
    scoring.js
    fallback.js
    policy.js
  usage/
    tokens.js
    cost.js
    quota.js
  observability/
    events.js
    logger.js
    health.js

Zero runtime dependencies should remain the default preference. Node's native http, https, crypto, stream, and AbortController APIs may be enough for the first local gateway.

A minimal HTTP/runtime dependency should be evaluated only if it clearly reduces security risk, protocol complexity, or maintenance cost. That decision should be made explicitly in a future sprint, not assumed in this plan.

7. API Contract

Plan an OpenAI-compatible subset:

• POST /v1/chat/completions
• GET /v1/models
• GET /health
• Optional streaming through server-sent events.
• Standardized error shape.
• Provider/model metadata extensions.

Early compatibility goals:

• Accept basic OpenAI-style chat completion requests.
• Preserve request IDs for traceability.
• Normalize provider errors into stable gateway errors.
• Never log raw secrets or full sensitive prompts by default.
• Make unsupported fields explicit instead of silently ignoring risky options.

8. Routing Strategies

Planned routing strategies:

• Explicit provider/model: user or client chooses exact target.
• Capability-based: route by required modality, tool-use, reasoning, coding, or context window.
• Cost-first: prefer lower-cost viable models.
• Latency-first: prefer fast healthy providers.
• Context-window-aware: avoid models that cannot fit the prompt safely.
• Privacy/local-first: prefer local providers when policy requires it.
• Fallback-chain: ordered failover across viable providers.
• Balanced: combine cost, latency, capability, and health.
• User policy: respect local policy files and explicit maintainer constraints.

Routing should be explainable. A dry-run route decision should be available before live provider calls are introduced.

9. Provider Adapter Contract

Provider adapters should be small, deterministic modules with a shared contract:

• Validate configuration.
• Expose available models.
• Normalize gateway requests into provider requests.
• Invoke provider APIs.
• Normalize provider responses.
• Stream responses when supported.
• Classify errors.
• Expose usage data.
• Expose health.
• Declare supported capabilities.
• Redact secrets in diagnostics.

The first adapter should be a generic OpenAI-compatible adapter. A mock provider should be included for tests and fallback simulation.

10. Security Model

Gateway runtime introduces new security risks and must be designed conservatively:

• Provider keys must never be committed.
• Provider credentials should be loaded only from explicit environment variables or approved local config.
• Logs must redact secrets, authorization headers, tokens, and sensitive request metadata.
• The gateway should bind to localhost by default.
• Non-local binding should require authentication and explicit configuration.
• Provider URLs must be validated to reduce SSRF risk.
• Local/private network targets should be restricted unless explicitly allowed.
• Requests need size limits.
• Streaming needs idle and total timeouts.
• Provider calls need retry ceilings and backoff.
• Audit logs should avoid prompt bodies by default.
• Permission and guardrail metadata should inform future confirmation and enforcement flows.
• No autonomous spending or provider account creation should exist in v4.2 foundation work.

Security tests should be written before any external provider call path is considered release-ready.

11. Revised Sprint Sequence

Sprint 0: v4.2 Development Lane Safety

• Move main to 4.2.0-dev.0 during development.
• Keep this version-only and release-state-only.
• Confirm v4.1.0 remains npm latest.
• Preserve v4.0.1 as a superseded, unpublished prepared release.

Sprint A: Gateway Protocol and Architecture Contracts

• Define gateway API request/response contracts.
• Define provider adapter interface.
• Define routing decision and route explanation objects.
• Add protocol and architecture docs.
• Add schema/fixture tests where useful.

Sprint A establishes deterministic contracts only. It does not start an HTTP server, call providers, load credentials, execute routing decisions, or run fallback chains.

Sprint B: Runtime Provider and Model Registry

• Convert existing provider/model metadata into runtime-readable registry loaders.
• Validate provider URLs, credential variable names, and model capability metadata.
• Keep real credentials out of fixtures.
• Preserve existing models, providers, and route-model CLI behavior.

Sprint B loads and validates metadata only. It does not read credential values, call providers, probe local engines, execute routing, or execute fallback chains.

Sprint C: Deterministic Routing Engine

• Implement pure route resolution functions.
• Score by explicit provider/model, capability, cost, latency hints, context window, and local-first policy.
• Add route dry-run fixtures.
• Produce rejected-candidate reasons, fallback plans, and route explanations.
• Avoid live provider calls, credential reads, model execution, fallback execution, tokenization, and live pricing lookups.

Sprint D: Fallback, Retry, and Resilience

• Add failure classification and retry eligibility contracts.
• Add deterministic retry-budget, backoff, and timeout planning.
• Add fallback transition planning over Sprint C fallback chains.
• Add circuit-breaker, rate-limit, and quota simulation contracts.
• Add resilience events and human-readable/machine-readable explanations.
• Test failure chains without real provider calls.

Sprint D remains simulation-only. It does not perform retries, perform provider failover, wait on timeouts, persist circuit state, call providers, or load credentials.

Sprint E: Local OpenAI-Compatible Gateway

• Add a localhost-only HTTP gateway.
• Support a minimal /v1/chat/completions, /v1/models, and /health contract.
• Start with a mock provider before any real-provider execution.
• Keep credentials explicit and unbundled.

Sprint E introduces local HTTP request/response handling for the mock provider only. It does not activate external providers, execute retries, execute fallback chains, persist request logs, or load provider credentials.

Sprint F: Client and Agent Integrations

• Document configuration for Codex, Claude Code, Cursor, Cline, Aider, Antigravity, MCP, and custom agents.
• Add examples that point clients at the local gateway endpoint.
• Validate docs and examples without requiring live provider accounts.

Sprint F adds preview-only client profile metadata, endpoint configuration generation, compatibility validation, and local mock protocol tests. It does not install or execute third-party clients, write user-global configuration, activate external providers, or change runtime retry/fallback behavior.

Sprint G: Observability, Token, and Cost Tracking

• Add local event records.
• Add token and cost estimation hooks.
• Add provider health snapshots.
• Keep logs redacted by default.

Sprint G implements this as bounded in-memory observability for the localhost mock gateway. It does not persist logs, upload telemetry, probe external providers, perform live pricing lookup, or retain prompts/completions by default.

Sprint H: Documentation, Security, and Release Hardening

• Complete security review.
• Run full validation.
• Verify package contents.
• Harden release notes and public docs.
• Keep all gateway claims tied to implemented behavior.

12. Non-Goals

For the v4.2 foundation, exclude:

• Hundreds of providers.
• Hosted SaaS gateway.
• Billing.
• Public cloud deployment.
• Automatic provider account creation.
• Silent credential discovery.
• Autonomous spending.
• Copying OmniRoute feature claims.
• Claiming all agents work before compatibility examples are tested.
• Runtime permission enforcement beyond explicitly approved scope.

13. Migration and Compatibility

v4.2 should preserve:

• Existing CLI commands.
• Existing adapters.
• Existing Skill OS registries.
• Existing workflows.
• Existing manual npm publish flow.
• Existing package identity: multimodel-dev-os.
• Zero-lock-in model.
• Existing docs that describe v4.1 as the released Skill OS foundation.

Migration should be additive. Existing users should not need to run a gateway to keep using current workspace, adapter, Skill OS, registry, memory, and workflow features.

14. Validation Strategy

Planned validation:

• Provider adapter contract tests.
• Router unit tests.
• Fallback simulation tests.
• API protocol tests.
• Streaming tests.
• Secret-redaction tests.
• Local binding tests.
• Provider URL validation tests.
• Timeout and request-size tests.
• Package and CLI regression tests.
• Docs build and link checks.
• Existing Skill OS validation checks.

No test should require real provider credentials unless it is explicitly marked optional and skipped by default.

15. Final Recommendation

Build the smallest viable v4.2 gateway foundation:

• One generic OpenAI-compatible provider adapter.
• One mock provider.
• Deterministic routing.
• Fallback simulation.
• Local-only gateway.
• Explicit credentials only.
• No real provider credentials bundled.
• No hosted gateway.
• No silent enforcement.
• No autonomous spending.

The next release-prep step should preserve the current boundary: mock runtime executable, external providers metadata-only, deterministic routing dry-run only, resilience simulation only, preview-only client plans, and bounded in-memory observability. Keep Skill OS as the control plane, and require a fresh security review before any real provider execution is introduced.