An OpenClaw Debugging & Observability Skill
An open-source debugging toolkit that traces root causes, profiles agent behavior, and predicts failures in multi-agent systems.
Originally built for Clawli AI. Released as open source for the OpenClaw Ecosystem.
Quick Start · What It Does · How It Works · OpenClaw Skill · API Reference · Tests · Architecture · Innovations · Contributing
What it does. CipherClaw is a deterministic debugging engine for multi-agent systems. It ingests traces and spans (including OTLP spans converted by convertOtlpTrace), builds a causal graph with weighted root-cause scores, fingerprints each agent's behavior, analyzes memory-tier health, and predicts failures from six pattern recognizers. The analysis path makes no LLM calls and the package has zero runtime dependencies.
Who it serves. Operators and developers running OpenClaw-style agent swarms: agents delegating to agents, writing to shared memory, and working across CRM, content, and infrastructure domains, where a flat span list no longer explains what broke.
Why Alex built it. Alex Cinovoj, Founder & CTO of TechTide AI, created the repository on 2026-02-16. The history records the motivation directly: the original README called it a bug-hunter agent for OpenClaw with ten capabilities documented in docs/PATENT.md (commit c14594b, 2026-02-16); the same day commit b9f782d ("align with OpenClaw open-source philosophy") renamed that document to docs/INNOVATIONS.md, replaced patent language with "capability" terminology, and added the OpenClaw SKILL.md, examples, CI, and security policy so the toolkit could be released for the OpenClaw ecosystem. The README has since stated that it was originally built for Clawli AI. Commit 8544ef4 (2026-03-20, v1.1.0) added the OpenTelemetry converter so traces exported from standard tooling can be analyzed without hand-written adapters.
Real-world use case (2025-2026 evidence). Verified in this repository at the test-suite level: src/__tests__/otel-converter.test.ts (29 tests) converts OTLP spans carrying gen_ai.* token and cost attributes into CipherClaw traces, and src/__tests__/e2e-session-lifecycle.test.ts (65 tests) runs sessions from ingest through root-cause report; CI runs the full 235-test suite on Node 20, 22, and 24 and checks that the built ESM and CJS bundles load on Node 18. Scenario: an OpenClaw operator exports GenAI spans from their agents with OpenTelemetry, the emerging standard for agent observability,1 feeds them to CipherClaw, and gets ranked root causes instead of a 400-span timeline. The same toolkit is positioned alongside NVIDIA's NemoClaw security stack for OpenClaw, announced at GTC on 2026-03-16 (see docs/NEMOCLAW.md).2 Node.js 18 is end-of-life and no longer receives security patches; CipherClaw keeps a Node 18 load check in CI for operators still on that line, but new deployments should use Node 20 or later.3
You're running a swarm of OpenClaw agents. Something breaks. You open your trace viewer and see 400 spans across 12 agents, 3 domains, and 5 memory tiers. Good luck.
Existing debug tools were built for single-agent chains. They show you a flat list of spans and call it a day. That's fine when you have one agent calling one tool. It falls apart the moment you have agents delegating to agents, writing to shared memory, and operating across CRM, content, and infrastructure domains simultaneously.
CipherClaw was built for this world. It's an autonomous debug agent that joins your OpenClaw system as a peer, communicates through the event bus, respects agent hierarchy, and does the actual debugging work, not just the observability part.
CipherClaw ships 10 capabilities that go beyond what trace viewers offer. Each one is independently tested and verified.
| Capability | What It Actually Does |
|---|---|
| Causal Debug Graph | Builds a DAG from your traces and scores root cause probability. Stops you from chasing symptoms. |
| Cognitive Fingerprinting | Profiles each agent's behavior across 8 dimensions. Detects when an agent starts acting differently. |
| Hierarchy Propagation | Routes debug events up, down, and laterally through your agent command structure. Parent agents know when children fail. |
| Memory Tier Debugging | Analyzes health across working, short-term, episodic, semantic, and archival memory. Finds stale data and retrieval failures. |
| Predictive Failure | Uses 6 pattern recognizers to predict failures before they happen. Catches error rate spikes, token budget exhaustion, and cascade risks. |
| Soul Integrity | Checks whether agents are staying true to their defined personality and values. Detects drift over time. |
| Cross-Domain Correlation | Finds shared failure patterns across agent, CRM, content, and infrastructure domains. One broken API can cascade everywhere. |
| Self-Debugging | CipherClaw monitors its own operation. If the debugger has a bug, it catches that too. |
| Flow Test Synthesis | Generates integration tests from observed execution traces. Your production runs become your test suite. |
| Anomaly Cascade Detection | Identifies when multiple anomalies cluster in time, indicating a cascading failure rather than isolated incidents. |
For the full technical deep-dive, see ARCHITECTURE.md. For the research behind these approaches, see INNOVATIONS.md.
The package is not yet published to the npm registry (checked 2026-10-20). Install from the repository:
git clone https://github.com/Alexi5000/CipherClaw.git
cd CipherClaw
pnpm install --frozen-lockfile
pnpm build # emits dist/esm and dist/cjsThen import from the built dist/ output, or add the repository as a git dependency in your own project (npm install github:Alexi5000/CipherClaw), which runs the prepare build on install.
import { createCipherClaw } from 'cipherclaw';
const cc = createCipherClaw();
// Start a debug session
const session = cc.startSession({ domain: 'agent' });
// Feed it traces from your agent system
cc.ingestTrace({
id: 'trace-001',
sessionId: session.id,
rootSpanId: 'span-1',
agentId: 'my-agent',
domain: 'agent',
startTime: Date.now() - 5000,
endTime: Date.now(),
durationMs: 5000,
status: 'error',
totalTokens: 0,
totalCost: 0,
spans: [
{
id: 'span-1',
traceId: 'trace-001',
parentSpanId: null,
name: 'agent.plan',
category: 'planning',
startTime: Date.now() - 5000,
endTime: Date.now() - 3000,
durationMs: 2000,
status: 'ok',
agentId: 'my-agent',
domain: 'agent',
attributes: {},
events: [],
},
{
id: 'span-2',
traceId: 'trace-001',
parentSpanId: 'span-1',
name: 'agent.tool_call',
category: 'action',
startTime: Date.now() - 3000,
endTime: Date.now(),
durationMs: 3000,
status: 'error',
agentId: 'my-agent',
domain: 'agent',
attributes: { error: 'Tool timeout' },
events: [{ name: 'error', timestamp: Date.now(), attributes: { message: 'Tool execution timed out' } }],
},
],
});
// Ask it questions
const error = cc.classifyError('Tool execution timed out');
console.log(error.module); // 'action'
console.log(error.severity); // 'high'
console.log(error.suggestedFix); // 'Increase tool timeout or add retry logic'
// Get the causal graph: who caused what
const graph = cc.getCausalGraph();
console.log(graph?.rootCauses.length); // root cause nodes
// Behavioral fingerprint: is this agent acting normal?
const fp = cc.computeCognitiveFingerprint('my-agent');
console.log(fp?.driftScore); // 0 = stable, higher = drifting
// Predict what's about to break
const predictions = cc.getPredictions();
predictions.forEach(p => {
console.log(`${p.predictedFailureType}: ${(p.confidence * 100).toFixed(0)}%`);
});
// Full health report
const report = cc.generateReport();
console.log(`Health: ${report?.healthScore}/100`);
cc.completeSession();CipherClaw is a standard OpenClaw skill. Drop the skills/cipherclaw/ directory into your ~/.openclaw/skills/ directory.
---
name: cipherclaw
description: Debug agent that traces causes, profiles behavior, and predicts failures in agent swarms.
user-invocable: true
---import { CIPHERCLAW_MANIFEST, createCipherClaw } from 'cipherclaw';
// The manifest describes CipherClaw to any OpenClaw gateway
console.log(CIPHERCLAW_MANIFEST.name); // 'cipherclaw'
console.log(CIPHERCLAW_MANIFEST.agents); // 5 agents
console.log(CIPHERCLAW_MANIFEST.skills); // 10 skills
console.log(CIPHERCLAW_MANIFEST.tools); // 14 tools
console.log(CIPHERCLAW_MANIFEST.events); // 12 events
// Subscribe to real-time debug events
const cc = createCipherClaw();
cc.on('error-classified', (e) => console.log(e.payload.module));
cc.on('anomaly-detected', (e) => console.log(e.payload.type));
cc.on('prediction-generated', (e) => console.log(e.payload.confidence));
cc.on('soul-drift-detected', (e) => console.log(e.payload.driftScore));When running inside an OpenClaw system, CipherClaw operates as an orchestrator-level agent (Phantom) with specialized sub-agents:
Your Sovereign Agent
├── Your Other Orchestrators
└── Phantom (CipherClaw Debug Orchestrator)
├── Trace Analyst: ingests and correlates traces
├── Error Classifier: categorizes and scores errors
├── Cognitive Profiler: fingerprints agent behavior
└── Flow Tester: synthesizes and runs integration tests
Instead of showing you a flat trace, CipherClaw builds a directed acyclic graph of error propagation. Each node is scored with root cause probability so you know where to look first.
A session moves through 6 phases: init → ingest → analyze → test → report → complete. Each phase emits events that other agents in your system can subscribe to.
const cc = createCipherClaw({
maxTraces: 10000, // max traces per session
anomalyThresholdStdDev: 2.5, // z-score threshold for anomaly detection
cascadeWindowMs: 30000, // time window for cascade grouping
soulDriftThreshold: 15, // drift alert threshold (0-100)
enableSelfDebug: true, // recursive self-monitoring
enableHierarchyPropagation: true,
});| Method | Returns |
|---|---|
startSession(opts?) |
DebugSession |
completeSession(id?) |
DebugSession |
getSession(id?) |
DebugSession |
getAllSessions() |
DebugSession[] |
pauseSession(id?) |
void |
resumeSession(id?) |
void |
| Method | Returns |
|---|---|
ingestTrace(trace, id?) |
void |
ingestSpan(span, id?) |
void |
| Method | What It Does |
|---|---|
classifyError(msg, span?, id?) |
Classifies an error by module, severity, and suggested fix |
getCausalGraph(id?) |
Returns the causal debug graph for the session |
getRootCauses(id?) |
Returns root cause nodes sorted by probability |
computeCognitiveFingerprint(agentId, id?) |
Profiles agent behavior across 8 dimensions |
analyzeSoulIntegrity(agentId, soul, behavior, id?) |
Checks personality and value adherence |
analyzeMemoryHealth(state, id?) |
Analyzes health across 5 memory tiers |
detectAnomalies(spans, id?) |
Statistical anomaly detection on span data |
detectCrossDomainCorrelations(id?) |
Finds shared failures across domains |
getPredictions(id?) |
Returns predicted failures with confidence scores |
propagateDebugEvent(event, id?) |
Routes a debug event through agent hierarchy |
selfDebug() |
Runs self-diagnostics on the engine |
synthesizeFlowTest(traceId, id?) |
Generates an integration test from a trace |
runFlowTests(domain?, id?) |
Runs all flow tests for a domain |
| Method | Returns |
|---|---|
addBreakpoint(type, condition?, meta?, id?) |
Breakpoint |
removeBreakpoint(bpId, id?) |
void |
toggleBreakpoint(bpId, id?) |
void |
getSnapshots(id?) |
StateSnapshot[] |
replayToSnapshot(snapId, id?) |
StateSnapshot |
11 breakpoint types: on_error, on_tool_call, on_iteration, on_span_category, on_agent, on_memory_op, on_cost_threshold, on_token_threshold, on_latency, on_pipeline_stage, conditional.
Conditional breakpoints use a bounded expression language, for example
span.status === "error" && span.durationMs >= 1000. Supported syntax:
- Roots:
span,session; own data properties through.name,["name"], or[0]. - Literals: single/double quoted strings, finite decimal numbers (including negative/exponent notation),
true,false,null,undefined. - Operators:
!,<,<=,>,>=,==,!=,===,!==,&&,||,??, and parentheses. Precedence from highest to lowest is!, relational comparisons, equality,&&,||,??. Logical operators short-circuit after the entire expression is parsed. - Strings accept escaped quotes, backslash, newline, carriage return and tab. Comparisons accept primitive values only; objects are never coerced through user methods.
Conditions cannot execute JavaScript. Calls, assignments, globals, computed property expressions, prototype access, getters, optional chaining, arithmetic, and template strings are unsupported. Conditions that are malformed, unsupported, or access a missing parent evaluate to false (no snapshot). Limits are 4096 characters, 256 tokens, 32 nested expression levels, and 32 property accesses per path.
Migration: earlier versions executed arbitrary JavaScript conditions. Rewrite function calls and other unsupported expressions using the grammar above. Trace and session objects must remain ordinary application-owned data, not hostile JavaScript Proxy objects; the expression interpreter is not an isolation boundary for executable objects supplied by code in the same process.
| Event | When |
|---|---|
session-started |
New session begins |
session-completed |
Session completes |
error-classified |
Error classified |
anomaly-detected |
Anomaly found |
cascade-detected |
Cascade identified |
prediction-generated |
Failure predicted |
breakpoint-hit |
Breakpoint triggered |
soul-drift-detected |
Personality drift |
cognitive-drift-detected |
Behavioral drift |
235 tests. No mocks. No stubs. Every test runs against the real engine.
pnpm test| Suite | Count | What It Covers |
|---|---|---|
| Unit: Core Capabilities | 66 | All 10 capabilities with edge cases |
| End-to-End: Sessions | 65 | Full lifecycle across all domains |
| Smoke: API Surface | 75 | Every public method and event |
| OpenTelemetry Converter | 29 | OTLP span and trace conversion, gen_ai token and cost attributes |
| Runtime | Status |
|---|---|
| Node.js 18+ | Tested |
| Deno | Compatible (ESM) |
| Bun | Compatible |
| Browser | Compatible (tree-shakeable) |
| Any OpenClaw system | Plug and play |
Zero runtime dependencies. Pure TypeScript.
CipherClaw/
├── src/
│ ├── index.ts # all exports
│ ├── types/index.ts # 40+ interfaces
│ ├── core/
│ │ ├── engine.ts # thin orchestrator (~300 lines)
│ │ ├── anomaly-detector.ts
│ │ ├── breakpoints.ts
│ │ ├── causal-graph.ts
│ │ ├── cognitive-profiler.ts
│ │ ├── cross-domain.ts
│ │ ├── error-classifier.ts
│ │ ├── flow-runner.ts
│ │ ├── flow-tests.ts
│ │ ├── hierarchy-propagation.ts
│ │ ├── memory-debugger.ts
│ │ ├── patterns.ts
│ │ ├── predictive-engine.ts
│ │ ├── report-generator.ts
│ │ ├── self-debug.ts
│ │ ├── snapshots.ts
│ │ ├── soul-monitor.ts
│ │ └── utils.ts
│ └── openclaw/
│ ├── manifest.ts # OpenClaw skill manifest
│ └── adapter.ts # event bus adapter
├── docs/
│ ├── ARCHITECTURE.md # technical deep-dive
│ ├── INNOVATIONS.md # novel approaches
│ ├── assets/ # logo, banner
│ └── diagrams/ # .mmd sources + .png renders
├── skills/
│ └── cipherclaw/
│ ├── SKILL.md # OpenClaw skill definition
│ └── SOUL.md # Phantom agent soul
├── CONTRIBUTING.md
└── src/__tests__/ # 235 tests
Read the security policy before submitting sensitive trace data or reporting a suspected vulnerability. Private vulnerability reporting is enabled on the repository.
Conditional breakpoints now use a bounded expression interpreter rather than executing caller-supplied JavaScript. The regression suite exercises code-injection rejection through engine ingestion and preserves ordinary comparisons and snapshots. CodeQL must rescan the merged change before its finding can be counted closed. CI runs typecheck, tests, and build on every pull request (.github/workflows/ci.yml) with the GITHUB_TOKEN limited to contents: read and the third-party pnpm/action-setup action pinned to a commit SHA.
Contributions are welcome. See CONTRIBUTING.md for development and pull-request guidance.
git clone https://github.com/Alexi5000/CipherClaw.git
cd CipherClaw
pnpm install
pnpm test # 235 tests
pnpm typecheck # strict mode
pnpm buildApache 2.0. Use it, fork it, build on it. Attribution appreciated.
Some approaches in this project represent novel work. See INNOVATIONS.md for details.
Built by Clawli AI
Released as open source for the OpenClaw Ecosystem
CipherClaw: zero-dependency AI agent debugging toolkit
Footnotes
-
OpenTelemetry, "AI Agent Observability - Evolving Standards and Best Practices", https://opentelemetry.io/blog/2025/ai-agent-observability/, published 2025-03-06. ↩
-
NVIDIA Newsroom, "NVIDIA Announces NemoClaw for the OpenClaw Community", https://nvidianews.nvidia.com/news/nvidia-announces-nemoclaw, published 2026-03-16. ↩
-
Node.js, "Beware of End-of-Life Node.js Versions - Upgrade or Seek Post-EOL Support", https://nodejs.org/en/blog/announcements/node-18-eol-support, published 2025-06-06. ↩




