Skip to content

About

Deterministic multi-agent debugger — zero LLM calls in the analysis path. Causal graphs, weighted root-cause scoring, failure prediction. 206 zero-mock tests, zero runtime deps.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

3 stars

Watchers

0 watching

Forks

CipherClaw: An OpenClaw Debugging & Observability Skill

CipherClaw

An OpenClaw Debugging & Observability Skill
An open-source debugging toolkit that traces root causes, profiles agent behavior, and predicts failures in multi-agent systems.
Originally built for Clawli AI. Released as open source for the OpenClaw Ecosystem.

Quick Start · What It Does · How It Works · OpenClaw Skill · API Reference · Tests · Architecture · Innovations · Contributing

CI Release License Tests TypeScript Strict OpenClaw Zero Dependencies


Why this exists

What it does. CipherClaw is a deterministic debugging engine for multi-agent systems. It ingests traces and spans (including OTLP spans converted by convertOtlpTrace), builds a causal graph with weighted root-cause scores, fingerprints each agent's behavior, analyzes memory-tier health, and predicts failures from six pattern recognizers. The analysis path makes no LLM calls and the package has zero runtime dependencies.

Who it serves. Operators and developers running OpenClaw-style agent swarms: agents delegating to agents, writing to shared memory, and working across CRM, content, and infrastructure domains, where a flat span list no longer explains what broke.

Why Alex built it. Alex Cinovoj, Founder & CTO of TechTide AI, created the repository on 2026-02-16. The history records the motivation directly: the original README called it a bug-hunter agent for OpenClaw with ten capabilities documented in docs/PATENT.md (commit c14594b, 2026-02-16); the same day commit b9f782d ("align with OpenClaw open-source philosophy") renamed that document to docs/INNOVATIONS.md, replaced patent language with "capability" terminology, and added the OpenClaw SKILL.md, examples, CI, and security policy so the toolkit could be released for the OpenClaw ecosystem. The README has since stated that it was originally built for Clawli AI. Commit 8544ef4 (2026-03-20, v1.1.0) added the OpenTelemetry converter so traces exported from standard tooling can be analyzed without hand-written adapters.

Real-world use case (2025-2026 evidence). Verified in this repository at the test-suite level: src/__tests__/otel-converter.test.ts (29 tests) converts OTLP spans carrying gen_ai.* token and cost attributes into CipherClaw traces, and src/__tests__/e2e-session-lifecycle.test.ts (65 tests) runs sessions from ingest through root-cause report; CI runs the full 235-test suite on Node 20, 22, and 24 and checks that the built ESM and CJS bundles load on Node 18. Scenario: an OpenClaw operator exports GenAI spans from their agents with OpenTelemetry, the emerging standard for agent observability,1 feeds them to CipherClaw, and gets ranked root causes instead of a 400-span timeline. The same toolkit is positioned alongside NVIDIA's NemoClaw security stack for OpenClaw, announced at GTC on 2026-03-16 (see docs/NEMOCLAW.md).2 Node.js 18 is end-of-life and no longer receives security patches; CipherClaw keeps a Node 18 load check in CI for operators still on that line, but new deployments should use Node 20 or later.3

The problem in practice

You're running a swarm of OpenClaw agents. Something breaks. You open your trace viewer and see 400 spans across 12 agents, 3 domains, and 5 memory tiers. Good luck.

Existing debug tools were built for single-agent chains. They show you a flat list of spans and call it a day. That's fine when you have one agent calling one tool. It falls apart the moment you have agents delegating to agents, writing to shared memory, and operating across CRM, content, and infrastructure domains simultaneously.

CipherClaw was built for this world. It's an autonomous debug agent that joins your OpenClaw system as a peer, communicates through the event bus, respects agent hierarchy, and does the actual debugging work, not just the observability part.

What It Does

CipherClaw ships 10 capabilities that go beyond what trace viewers offer. Each one is independently tested and verified.

Capability What It Actually Does
Causal Debug Graph Builds a DAG from your traces and scores root cause probability. Stops you from chasing symptoms.
Cognitive Fingerprinting Profiles each agent's behavior across 8 dimensions. Detects when an agent starts acting differently.
Hierarchy Propagation Routes debug events up, down, and laterally through your agent command structure. Parent agents know when children fail.
Memory Tier Debugging Analyzes health across working, short-term, episodic, semantic, and archival memory. Finds stale data and retrieval failures.
Predictive Failure Uses 6 pattern recognizers to predict failures before they happen. Catches error rate spikes, token budget exhaustion, and cascade risks.
Soul Integrity Checks whether agents are staying true to their defined personality and values. Detects drift over time.
Cross-Domain Correlation Finds shared failure patterns across agent, CRM, content, and infrastructure domains. One broken API can cascade everywhere.
Self-Debugging CipherClaw monitors its own operation. If the debugger has a bug, it catches that too.
Flow Test Synthesis Generates integration tests from observed execution traces. Your production runs become your test suite.
Anomaly Cascade Detection Identifies when multiple anomalies cluster in time, indicating a cascading failure rather than isolated incidents.

For the full technical deep-dive, see ARCHITECTURE.md. For the research behind these approaches, see INNOVATIONS.md.


Quick Start

Install

The package is not yet published to the npm registry (checked 2026-10-20). Install from the repository:

git clone https://github.com/Alexi5000/CipherClaw.git
cd CipherClaw
pnpm install --frozen-lockfile
pnpm build       # emits dist/esm and dist/cjs

Then import from the built dist/ output, or add the repository as a git dependency in your own project (npm install github:Alexi5000/CipherClaw), which runs the prepare build on install.

Use

import { createCipherClaw } from 'cipherclaw';

const cc = createCipherClaw();

// Start a debug session
const session = cc.startSession({ domain: 'agent' });

// Feed it traces from your agent system
cc.ingestTrace({
  id: 'trace-001',
  sessionId: session.id,
  rootSpanId: 'span-1',
  agentId: 'my-agent',
  domain: 'agent',
  startTime: Date.now() - 5000,
  endTime: Date.now(),
  durationMs: 5000,
  status: 'error',
  totalTokens: 0,
  totalCost: 0,
  spans: [
    {
      id: 'span-1',
      traceId: 'trace-001',
      parentSpanId: null,
      name: 'agent.plan',
      category: 'planning',
      startTime: Date.now() - 5000,
      endTime: Date.now() - 3000,
      durationMs: 2000,
      status: 'ok',
      agentId: 'my-agent',
      domain: 'agent',
      attributes: {},
      events: [],
    },
    {
      id: 'span-2',
      traceId: 'trace-001',
      parentSpanId: 'span-1',
      name: 'agent.tool_call',
      category: 'action',
      startTime: Date.now() - 3000,
      endTime: Date.now(),
      durationMs: 3000,
      status: 'error',
      agentId: 'my-agent',
      domain: 'agent',
      attributes: { error: 'Tool timeout' },
      events: [{ name: 'error', timestamp: Date.now(), attributes: { message: 'Tool execution timed out' } }],
    },
  ],
});

// Ask it questions
const error = cc.classifyError('Tool execution timed out');
console.log(error.module);       // 'action'
console.log(error.severity);     // 'high'
console.log(error.suggestedFix); // 'Increase tool timeout or add retry logic'

// Get the causal graph: who caused what
const graph = cc.getCausalGraph();
console.log(graph?.rootCauses.length); // root cause nodes

// Behavioral fingerprint: is this agent acting normal?
const fp = cc.computeCognitiveFingerprint('my-agent');
console.log(fp?.driftScore); // 0 = stable, higher = drifting

// Predict what's about to break
const predictions = cc.getPredictions();
predictions.forEach(p => {
  console.log(`${p.predictedFailureType}: ${(p.confidence * 100).toFixed(0)}%`);
});

// Full health report
const report = cc.generateReport();
console.log(`Health: ${report?.healthScore}/100`);

cc.completeSession();

OpenClaw Skill

CipherClaw is a standard OpenClaw skill. Drop the skills/cipherclaw/ directory into your ~/.openclaw/skills/ directory.

SKILL.md

---
name: cipherclaw
description: Debug agent that traces causes, profiles behavior, and predicts failures in agent swarms.
user-invocable: true
---

Programmatic Integration

import { CIPHERCLAW_MANIFEST, createCipherClaw } from 'cipherclaw';

// The manifest describes CipherClaw to any OpenClaw gateway
console.log(CIPHERCLAW_MANIFEST.name);    // 'cipherclaw'
console.log(CIPHERCLAW_MANIFEST.agents);  // 5 agents
console.log(CIPHERCLAW_MANIFEST.skills);  // 10 skills
console.log(CIPHERCLAW_MANIFEST.tools);   // 14 tools
console.log(CIPHERCLAW_MANIFEST.events);  // 12 events

// Subscribe to real-time debug events
const cc = createCipherClaw();
cc.on('error-classified', (e) => console.log(e.payload.module));
cc.on('anomaly-detected', (e) => console.log(e.payload.type));
cc.on('prediction-generated', (e) => console.log(e.payload.confidence));
cc.on('soul-drift-detected', (e) => console.log(e.payload.driftScore));

Agent Hierarchy

When running inside an OpenClaw system, CipherClaw operates as an orchestrator-level agent (Phantom) with specialized sub-agents:

Your Sovereign Agent
├── Your Other Orchestrators
└── Phantom (CipherClaw Debug Orchestrator)
    ├── Trace Analyst: ingests and correlates traces
    ├── Error Classifier: categorizes and scores errors
    ├── Cognitive Profiler: fingerprints agent behavior
    └── Flow Tester: synthesizes and runs integration tests

CipherClaw Architecture


How It Works

Causal Debug Graph

Instead of showing you a flat trace, CipherClaw builds a directed acyclic graph of error propagation. Each node is scored with root cause probability so you know where to look first.

Causal Debug Graph

Debug Session Lifecycle

A session moves through 6 phases: init → ingest → analyze → test → report → complete. Each phase emits events that other agents in your system can subscribe to.

Debug Session Flow


API Reference

Factory

const cc = createCipherClaw({
  maxTraces: 10000,              // max traces per session
  anomalyThresholdStdDev: 2.5,  // z-score threshold for anomaly detection
  cascadeWindowMs: 30000,        // time window for cascade grouping
  soulDriftThreshold: 15,        // drift alert threshold (0-100)
  enableSelfDebug: true,         // recursive self-monitoring
  enableHierarchyPropagation: true,
});

Sessions

Method Returns
startSession(opts?) DebugSession
completeSession(id?) DebugSession
getSession(id?) DebugSession
getAllSessions() DebugSession[]
pauseSession(id?) void
resumeSession(id?) void

Ingestion

Method Returns
ingestTrace(trace, id?) void
ingestSpan(span, id?) void

Analysis

Method What It Does
classifyError(msg, span?, id?) Classifies an error by module, severity, and suggested fix
getCausalGraph(id?) Returns the causal debug graph for the session
getRootCauses(id?) Returns root cause nodes sorted by probability
computeCognitiveFingerprint(agentId, id?) Profiles agent behavior across 8 dimensions
analyzeSoulIntegrity(agentId, soul, behavior, id?) Checks personality and value adherence
analyzeMemoryHealth(state, id?) Analyzes health across 5 memory tiers
detectAnomalies(spans, id?) Statistical anomaly detection on span data
detectCrossDomainCorrelations(id?) Finds shared failures across domains
getPredictions(id?) Returns predicted failures with confidence scores
propagateDebugEvent(event, id?) Routes a debug event through agent hierarchy
selfDebug() Runs self-diagnostics on the engine
synthesizeFlowTest(traceId, id?) Generates an integration test from a trace
runFlowTests(domain?, id?) Runs all flow tests for a domain

Breakpoints

Method Returns
addBreakpoint(type, condition?, meta?, id?) Breakpoint
removeBreakpoint(bpId, id?) void
toggleBreakpoint(bpId, id?) void
getSnapshots(id?) StateSnapshot[]
replayToSnapshot(snapId, id?) StateSnapshot

11 breakpoint types: on_error, on_tool_call, on_iteration, on_span_category, on_agent, on_memory_op, on_cost_threshold, on_token_threshold, on_latency, on_pipeline_stage, conditional.

Conditional breakpoints use a bounded expression language, for example span.status === "error" && span.durationMs >= 1000. Supported syntax:

  • Roots: span, session; own data properties through .name, ["name"], or [0].
  • Literals: single/double quoted strings, finite decimal numbers (including negative/exponent notation), true, false, null, undefined.
  • Operators: !, <, <=, >, >=, ==, !=, ===, !==, &&, ||, ??, and parentheses. Precedence from highest to lowest is !, relational comparisons, equality, &&, ||, ??. Logical operators short-circuit after the entire expression is parsed.
  • Strings accept escaped quotes, backslash, newline, carriage return and tab. Comparisons accept primitive values only; objects are never coerced through user methods.

Conditions cannot execute JavaScript. Calls, assignments, globals, computed property expressions, prototype access, getters, optional chaining, arithmetic, and template strings are unsupported. Conditions that are malformed, unsupported, or access a missing parent evaluate to false (no snapshot). Limits are 4096 characters, 256 tokens, 32 nested expression levels, and 32 property accesses per path.

Migration: earlier versions executed arbitrary JavaScript conditions. Rewrite function calls and other unsupported expressions using the grammar above. Trace and session objects must remain ordinary application-owned data, not hostile JavaScript Proxy objects; the expression interpreter is not an isolation boundary for executable objects supplied by code in the same process.

Events

Event When
session-started New session begins
session-completed Session completes
error-classified Error classified
anomaly-detected Anomaly found
cascade-detected Cascade identified
prediction-generated Failure predicted
breakpoint-hit Breakpoint triggered
soul-drift-detected Personality drift
cognitive-drift-detected Behavioral drift

Tests

235 tests. No mocks. No stubs. Every test runs against the real engine.

pnpm test
Suite Count What It Covers
Unit: Core Capabilities 66 All 10 capabilities with edge cases
End-to-End: Sessions 65 Full lifecycle across all domains
Smoke: API Surface 75 Every public method and event
OpenTelemetry Converter 29 OTLP span and trace conversion, gen_ai token and cost attributes

Platforms

Runtime Status
Node.js 18+ Tested
Deno Compatible (ESM)
Bun Compatible
Browser Compatible (tree-shakeable)
Any OpenClaw system Plug and play

Zero runtime dependencies. Pure TypeScript.


Project Structure

CipherClaw/
├── src/
│   ├── index.ts              # all exports
│   ├── types/index.ts        # 40+ interfaces
│   ├── core/
│   │   ├── engine.ts         # thin orchestrator (~300 lines)
│   │   ├── anomaly-detector.ts
│   │   ├── breakpoints.ts
│   │   ├── causal-graph.ts
│   │   ├── cognitive-profiler.ts
│   │   ├── cross-domain.ts
│   │   ├── error-classifier.ts
│   │   ├── flow-runner.ts
│   │   ├── flow-tests.ts
│   │   ├── hierarchy-propagation.ts
│   │   ├── memory-debugger.ts
│   │   ├── patterns.ts
│   │   ├── predictive-engine.ts
│   │   ├── report-generator.ts
│   │   ├── self-debug.ts
│   │   ├── snapshots.ts
│   │   ├── soul-monitor.ts
│   │   └── utils.ts
│   └── openclaw/
│       ├── manifest.ts       # OpenClaw skill manifest
│       └── adapter.ts        # event bus adapter
├── docs/
│   ├── ARCHITECTURE.md       # technical deep-dive
│   ├── INNOVATIONS.md        # novel approaches
│   ├── assets/               # logo, banner
│   └── diagrams/             # .mmd sources + .png renders
├── skills/
│   └── cipherclaw/
│       ├── SKILL.md          # OpenClaw skill definition
│       └── SOUL.md           # Phantom agent soul
├── CONTRIBUTING.md
└── src/__tests__/            # 235 tests

Security

Read the security policy before submitting sensitive trace data or reporting a suspected vulnerability. Private vulnerability reporting is enabled on the repository.

Conditional breakpoints now use a bounded expression interpreter rather than executing caller-supplied JavaScript. The regression suite exercises code-injection rejection through engine ingestion and preserves ordinary comparisons and snapshots. CodeQL must rescan the merged change before its finding can be counted closed. CI runs typecheck, tests, and build on every pull request (.github/workflows/ci.yml) with the GITHUB_TOKEN limited to contents: read and the third-party pnpm/action-setup action pinned to a commit SHA.

Contributing

Contributions are welcome. See CONTRIBUTING.md for development and pull-request guidance.

git clone https://github.com/Alexi5000/CipherClaw.git
cd CipherClaw
pnpm install
pnpm test        # 235 tests
pnpm typecheck   # strict mode
pnpm build

License

Apache 2.0. Use it, fork it, build on it. Attribution appreciated.

Some approaches in this project represent novel work. See INNOVATIONS.md for details.


CipherClaw

Built by Clawli AI
Released as open source for the OpenClaw Ecosystem
CipherClaw: zero-dependency AI agent debugging toolkit

Footnotes

  1. OpenTelemetry, "AI Agent Observability - Evolving Standards and Best Practices", https://opentelemetry.io/blog/2025/ai-agent-observability/, published 2025-03-06. ↩

  2. NVIDIA Newsroom, "NVIDIA Announces NemoClaw for the OpenClaw Community", https://nvidianews.nvidia.com/news/nvidia-announces-nemoclaw, published 2026-03-16. ↩

  3. Node.js, "Beware of End-of-Life Node.js Versions - Upgrade or Seek Post-EOL Support", https://nodejs.org/en/blog/announcements/node-18-eol-support, published 2025-06-06. ↩

About

Deterministic multi-agent debugger — zero LLM calls in the analysis path. Causal graphs, weighted root-cause scoring, failure prediction. 206 zero-mock tests, zero runtime deps.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages