gdev-agent

gdev-agent is a Python/FastAPI reference backend for game-studio support teams. It turns incoming player messages into structured triage, draft replies, and reviewable actions. Signed webhooks, tenant isolation, human approval, and audit trails connect the workflow to downstream services.

Representative workflow

A player reports a billing problem. A signed webhook brings the message into the backend, which checks the input, classifies the issue, and extracts structured fields. A risky action waits for a human decision; execution and its outcome leave an audit trail. Try the synthetic local demo.

Project origin

The project began as paid work for a game studio on a concrete player-support workflow. The public repository now demonstrates a local reference implementation using synthetic tickets. Its published tests and evaluations describe that public implementation; they do not establish the original client's deployment or results.

Current Maturity

Status reviewed on 24 October 2026; evidence index.

The supported proof boundary is a tested local Docker Compose workload with synthetic demo/eval data and database-enforced tenant isolation. Its 55-case conformance set passes, while the harder published 100-case Eval Lab gate fails. There is no claimed pilot, external deployment, live customer traffic, production readiness, or production SLO.

Relationship to the Portfolio

• This repository owns application behavior, signed ingress, approval/audit flows, tenant controls, and candidate fixes.
• Eval Ground Truth Lab owns external workflow-quality gates. Its canonical 100-case result for this revision is FAIL and remains authoritative for those challenge outcomes.
• Agent Runtime Grid is an optional execution layer. It does not replace gdev lifecycle/security tests or Eval Lab quality decisions.
• AI Workflow Playbook is an independent governance companion, not a runtime dependency.
• The thin umbrella pins compatible revisions and runs integration evidence; it does not absorb component code or history.

Product Boundary and Non-Goals

The repository demonstrates one bounded local support workflow. It is not a hosted support platform, customer-data product, generic agent framework, production multi-tenant SaaS, or proof that the demo classifier generalizes.

Evidence Path

For a claim-by-claim proof map, start with docs/EVIDENCE_INDEX.md.

Claim to inspect	Start here	Bounded status
Evidence question map	docs/EVIDENCE_INDEX.md#evidence-questions	One-click route for problem, architecture, controls, quality, failure behavior, metrics, demo, limits, and production changes
Three-project stack map	docs/STACK_OVERVIEW.md	Explains how gdev-agent, Eval Ground Truth Lab, and Agent Runtime Grid fit together as workflow, quality, and runtime layers
One-page engineering story	docs/CASE_STUDY.md	Evidence-backed case study for problem, architecture, controls, eval, load, trade-offs, and production changes
Architecture and workflow boundaries	docs/ARCHITECTURE.md, docs/architecture-diagram.md	Implemented local stack with documented gaps and ADRs
Human approval decision path	docs/APPROVAL_FLOW.md	Code-and-test-backed risk, tenant, TTL, reject, and one-time execution flow; not an operator UI or routing-quality claim
Agent harness boundary	docs/HARNESS_CARD.md, docs/TRACE_SCHEMA.md, AGENTS.md	Model + prompt/tool loop + guards + approvals + trace + eval are reviewed as one bounded harness
Repeatable demo path	docs/DEMO.md	Local Compose demo with deterministic/free mode
Evaluation discipline	docs/EVALUATION.md, docs/EVAL_REPORT.md, docs/EVAL_SCOPE_RECONCILIATION.md	180-case internal smoke eval, 55-case Eval Lab conformance baseline, and a canonical 100-case challenge run whose stricter gate failed
Canonical failure preview	docs/evidence/GDEV_EVAL_FAILURE_PREVIEW_2026-07-13.md	Rendered from Eval Lab v0.2.0 content-addressed raw JSON; five failed thresholds remain visible and limitations are explicit
Observability	docs/observability.md	Metrics, traces, logs, and alerting design for local evidence
Load profile	docs/load-profile.md, docs/LOAD_TEST_REPORT.md	Local deterministic/synthetic report and scenario targets; not production capacity claims
Tenant isolation and security	docs/TENANT_ISOLATION.md, docs/data-map.md#6-tenant-isolation-model, docs/ARCHITECTURE.md#7-security-model	RLS, tenant-scoped JWT, webhook signature, secrets, approval, and cost ledger boundaries
Tests	Current State, maintainer-surface verification	P0 code baseline: 310 passing tests at 0e4c5f0; 2026-07-13 maintainer-surface verification: 312 passing tests after two documentation contract tests
Failure modes and SLO/runbook	docs/FAILURE_MODES.md, docs/SLO_RUNBOOK.md, docs/observability.md#alert-runbooks	Local taxonomy and runbook evidence; external incident evidence is out of scope
Deployment readiness boundaries	docs/DEPLOYMENT_READINESS.md, Known Limits	Secrets checklist, backup/restore notes, local production-like config, and known limitations without production readiness claims
Known limits and production changes	Known Limits, docs/DEPLOYMENT_READINESS.md	Explicitly bounded as local reference evidence, not production SaaS readiness

The image is a preview of pinned negative evidence, not a current quality-pass badge. Follow it to the exact Eval Lab tag, raw JSON hashes, rendering command, and interpretation boundary.

Why This Project Exists

The synthetic reference scenario covers billing disputes, account-access incidents, bug reports, moderation signals, and repetitive gameplay questions. gdev-agent exercises the orchestration boundary between inbound support traffic and downstream tools: routine paths move through deterministic controls, while low-confidence or risky actions require review.

Architecture

flowchart LR
    caller[Webhook caller\nTelegram / n8n / Make / HTTP client] --> webhook[POST /webhook]
    webhook --> sig[Signature + rate-limit middleware]
    sig --> guard[Input guard\nlength and injection checks]
    guard --> llm[LLM tool loop\nclassify + extract + draft]
    llm --> policy[Policy + output guard\nrisk, confidence, secret and URL checks]
    policy --> decision{Risky action?}
    decision -->|Yes| pending[Store pending approval\nRedis + reviewer callback]
    pending --> approve[POST /approve]
    approve --> execute[Execute approved action\nticket + reply + audit]
    decision -->|No| execute
    execute --> result[HTTP response + audit trail\nlogs, metrics, cost ledger]

Loading

The current stack includes FastAPI, Redis, PostgreSQL with Row-Level Security, pgvector-backed embeddings, a clean service layer (WebhookService, ApprovalService, AuthService, EvalService), RCA clustering jobs, and an n8n integration path for orchestration and approvals. Architecture detail lives in docs/ARCHITECTURE.md.

Feature Snapshot

Area	What ships today
Ingress	POST /webhook entrypoint with WebhookService (tenant resolution, dedup, OTel tracing); per-tenant HMAC verification, rate limiting
AI pipeline	Claude tool_use classification and extraction, guarded draft generation, configurable auto-approve threshold
Safety	Input injection guard, output secret scan, URL allowlist enforcement, approval workflow with ApprovalService (HMAC + cross-tenant enforcement)
Execution	Tool registry for ticketing and reply actions, dedup cache for idempotent replays, pending approval storage with TTL
Multi-tenancy	PostgreSQL FORCE RLS on all 16 tenant-scoped tables, non-owner gdev_app runtime role, tenant registry, per-tenant encrypted secrets
Operations	Cost ledger with daily budget enforcement, structured JSON logs, Prometheus metrics (OTel child spans on all endpoints), Grafana/Loki/Tempo stack
Analytics	Eval runner with budget check, eval API, tenant learning metrics from approval latency/overrides, RCA clustering job (DBSCAN + pgvector), cluster read endpoints with DB-backed membership
Admin	gdev-admin CLI for tenant/budget/RCA operations; separate maintenance role with BYPASSRLS
Platform	Docker Compose full stack; Python 3.12 CI; ruff, full pytest, eval, Compose RLS, and deterministic demo gates

Quick Start

Docker Compose

This path is aligned to docker-compose.yml and is the fastest way to get a healthy local stack.

git clone https://github.com/ashishki/gdev-agent.git
cd gdev-agent
cp .env.example .env
docker compose up --build

What starts:

Service	URL / Port	Purpose
agent	http://localhost:8000	FastAPI application
postgres	localhost:5432	Primary database
redis	localhost:6379	Rate limit, dedup, approval state
n8n	http://localhost:5678	Workflow orchestration
prometheus	http://localhost:9090	Metrics scrape
grafana	http://localhost:3000	Dashboards
tempo	http://localhost:3200	Trace backend
loki	http://localhost:3100	Log backend

The migrate service runs Alembic and seeds the database before the API starts. Compose creates a bootstrap/migration owner (gdev_owner) and a distinct non-superuser request role (gdev_app). Their local-only passwords come from GDEV_OWNER_PASSWORD and GDEV_APP_PASSWORD in the copied .env; Compose has no password fallback. LLM_MODE defaults to deterministic demo mode.

Verify the stack:

curl -i http://localhost:8000/health
bash scripts/verify_compose_rls.sh

Expected response:

HTTP/1.1 200 OK

{"status":"ok","app":"gdev-agent"}

If you want to exercise live LLM behavior, set LLM_MODE=live and ANTHROPIC_API_KEY in .env before startup.

Environment Variables

Copy .env.example and adjust only what you need for your environment. Deployment-readiness boundaries, required secrets, backup, restore, and known limitations are documented in docs/DEPLOYMENT_READINESS.md.

Variable	Required	Notes
ANTHROPIC_API_KEY	Yes for LLM_MODE=live	Live provider calls fail startup without it
ANTHROPIC_MODEL	No	Defaults to claude-sonnet-4-6
VOYAGE_API_KEY	No	Needed for embedding-backed features outside stub mode
EMBEDDING_MODEL	No	Defaults to voyage-3-lite
KB_BASE_URL	Recommended	FAQ links should also be present in URL_ALLOWLIST
REDIS_URL	Yes	Approval store, rate limiting, dedup, caching
DATABASE_URL	Yes for Postgres features	Compose provides it automatically
GDEV_OWNER_PASSWORD / GDEV_APP_PASSWORD	Yes for Compose	Separate migration-owner and request-role passwords; local examples are in .env.example
TEST_DATABASE_URL	No	Test-only override
DB_POOL_SIZE / DB_MAX_OVERFLOW	No	Async Postgres pool sizing
WEBHOOK_SECRET	Optional legacy path	Global webhook secret; per-tenant secret storage is the main design
WEBHOOK_SECRET_ENCRYPTION_KEY	Recommended	Fernet key for encrypted per-tenant webhook secrets
JWT_SECRET	Yes outside demo mode	JWT signing secret
JWT_ALGORITHM	No	Defaults to HS256
JWT_TOKEN_EXPIRY_HOURS	No	Access token lifetime
APPROVE_SECRET	Recommended	Shared secret for approval callbacks
RATE_LIMIT_RPM / RATE_LIMIT_BURST	No	Request and burst limits
AUTH_RATE_LIMIT_ATTEMPTS	No	Login throttling
MAX_INPUT_LENGTH	No	Input guard length cap
AUTO_APPROVE_THRESHOLD	No	Confidence threshold for auto execution
EXEMPLAR_GUARD_ENABLED	No	Enables curated-example consistency checks before auto execution
EXEMPLAR_GUARD_THRESHOLD / EXEMPLAR_GUARD_TOP_K	No	Similarity threshold and nearest-example count
EXEMPLAR_GUARD_EXAMPLES_PATH	No	Optional JSONL override; empty uses eval/exemplars/triage_v1.jsonl
APPROVAL_CATEGORIES	No	Comma-separated category list
APPROVAL_TTL_SECONDS	No	Pending approval expiry
OUTPUT_GUARD_ENABLED	No	Enables output checks
URL_ALLOWLIST	No	Comma-separated hostname allowlist
OUTPUT_URL_BEHAVIOR	No	strip or reject
RCA_LOOKBACK_HOURS	No	RCA clustering window
RCA_BUDGET_PER_RUN_USD	No	Budget cap per RCA run
LINEAR_API_KEY / LINEAR_TEAM_ID	Optional	Ticket creation integration
TELEGRAM_BOT_TOKEN / TELEGRAM_APPROVAL_CHAT_ID	Optional	Messaging and approval notifications
GOOGLE_SHEETS_CREDENTIALS_JSON / GOOGLE_SHEETS_ID	Optional	Audit export integration
SQLITE_LOG_PATH	Optional	Enables local SQLite event logging
OTLP_ENDPOINT / OTEL_SERVICE_NAME	Optional	OpenTelemetry export
APP_NAME / APP_ENV / LOG_LEVEL	No	App identity and logging controls
LLM_INPUT_RATE_PER_1K / LLM_OUTPUT_RATE_PER_1K	No	Cost ledger rates

API Overview

Core workflow

Endpoint	Purpose
POST /webhook	Main ingestion path for support messages; returns executed, pending, or guard-blocked
POST /approve	Human decision endpoint for pending actions
GET /health	Application liveness check used by Docker health checks
GET /metrics	Prometheus scrape endpoint

Auth and governance

Endpoint	Purpose
POST /auth/token	Login and issue JWT
POST /auth/logout	Blocklist a token
POST /auth/refresh	Refresh an access token
POST /eval/run	Start an eval run
GET /eval/runs	List eval history

Tenant read APIs

Endpoint	Purpose
GET /tickets and GET /tickets/{ticket_id}	Ticket history and detail
GET /audit	Audit log history
GET /metrics/cost	Cost ledger readout
GET /metrics/learning	Tenant approval latency, override, rejection, and reviewed-volume metrics
GET /agents and PUT /agents/{agent_id}	Agent config inspection and versioned updates
GET /clusters	RCA cluster list
GET /clusters/{cluster_id}	RCA cluster detail
GET /clusters/{cluster_id}/tickets	Tickets associated with a cluster

Most endpoints outside /health, /webhook, and /metrics require JWT auth plus tenant context.

Request Flow At A Glance

1. A caller sends a support event to POST /webhook.
2. Middleware verifies the request signature, applies rate limits, and assigns request correlation.
3. The agent blocks oversized or injection-shaped input before any model call.
4. The LLM classifies the message, extracts entities, and drafts a response.
5. Policy and output-guard checks decide whether the action can auto-execute or must wait for review.
6. The service either executes the tool path immediately or stores a pending approval for POST /approve.
7. Audit rows, metrics, and cost tracking capture the outcome.

Repository Guide

• docs/EVIDENCE_INDEX.md: evidence question map and claim-by-claim proof table.
• docs/STACK_OVERVIEW.md: three-project stack map and provider strategy.
• docs/EVAL_SCOPE_RECONCILIATION.md: reconciles the internal 180-case smoke, Eval Lab 55-case conformance baseline, canonical 100-case challenge failure, and Runtime Grid proofs.
• docs/ARCHITECTURE.md: system structure, service boundaries, request flow, deployment view.
• docs/HARNESS_CARD.md: agent harness boundary across model, tools, memory, retries, permissions, HITL, trace, and eval.
• docs/TRACE_SCHEMA.md: trace completeness contract for debugging, eval, audit, and approval retrospectives.
• docs/architecture-diagram.md: GitHub-rendered architecture diagram for the main workflow.
• docs/APPROVAL_FLOW.md: implemented human-approval decision path with focused regression commands and limits.
• docs/evidence/GDEV_EVAL_FAILURE_PREVIEW_2026-07-13.md: content-addressed preview of the canonical failed Eval Lab challenge.
• docs/evidence/GDEV_MAINTAINER_SURFACE_2026-07-13.md: local receipt for the maintainer, visual, full-suite, eval, and Compose gates.
• docs/CASE_STUDY.md: concise evidence-backed engineering case study.
• docs/spec.md: product scope, API intent, and behavioral contract.
• docs/N8N.md: n8n integration and approval workflow blueprint.
• docs/observability.md: metrics, tracing, and logging conventions.
• docs/agent-registry.md: agent configuration model and governance.
• docs/llm-usage.md: prompt/versioning and model-usage rules.
• docs/load-profile.md: load targets and performance assumptions.
• docs/TENANT_ISOLATION.md: canonical tenant-isolation proof with exact tests and migrations.
• docs/DEPLOYMENT_READINESS.md: local production-like config, secrets checklist, backup/restore notes, and known limitations.
• docs/data-map.md: schema, Redis keys, and tenant-boundary rules.
• n8n/README.md: workflow assets committed in this repository.
• AGENTS.md: operating rules for coding/ops agents working in this repo.

Maintainer Paths

• Suspected vulnerabilities must follow the private path in SECURITY.md; do not disclose them in a public issue.
• Supported local defects use the reproducible bug form with an exact commit, sanitized environment, and deterministic commands.
• There is no generic feature or hosted-product roadmap intake. The maintained boundary is the current local reference workload described in Current Maturity.

Known Limits

• The project is a tested local reference workload. It has no claimed external deployment, production SaaS readiness, live tenant traffic, or real customer operations.
• Demo, eval, and load evidence is synthetic unless a later report explicitly says otherwise. The current load report is deterministic/local evidence, not live capacity proof.
• Eval metrics have multiple scopes. The internal 180-case smoke report exposes broad demo-mode routing gaps, while the external Eval Lab 55-case baseline is an integration/conformance pass over the configured /webhook adapter. A canonical Eval Lab run of the separate 100-case challenge against exact revision 0e4c5f0fd50382bbf12ffd35cfca4632384fb0cc failed its stricter gate: reconciled pass rate 0.32, classification accuracy 0.244444, 68 unexpected failures, and 58 blocking failures. This is published negative evidence, not a threshold pass. See docs/EVAL_SCOPE_RECONCILIATION.md.
• Live load measurements remain out of scope for the current local evidence. Deployment readiness notes are local-only and explicitly do not prove production readiness.
• Live LLM behavior requires a real Anthropic API key and budget controls; the local stack is the supported review path today.

Current State

The local stack demonstrates the governed request pipeline. It includes the tenant-scoped storage foundation, JWT/RBAC boundary, approval hardening, eval APIs with budget enforcement, auth service flows, embedding persistence, RCA clustering with persisted cluster membership, service-layer separation for the main write/auth/eval workflows with read-route extraction still tracked as architecture drift, Dockerized observability, admin CLI, and n8n workflow artifacts for local demonstrations.

At revision 0e4c5f0fd50382bbf12ffd35cfca4632384fb0cc, the dated 2026-07-13 local baseline is 310 tests passed (unit + integration, including migration up/down, role flags, FORCE RLS topology, cross-tenant rejection, eval metric validators, load fixtures, observability signals, and cluster membership persistence). The same repair run also passed the 180-case demo eval gate and a clean Compose auth/approval demo. Exact commands and bounded outputs are recorded in docs/evidence/GDEV_P0_TRUTH_REPAIR_2026-07-13.md.

The later maintainer/evidence-surface tranche added two focused documentation contracts and reran the complete local suite at 312 passing tests, along with Ruff, the non-writing 180-case eval gate, and an isolated default-Compose RLS and approval demo. Its bounded receipt is docs/evidence/GDEV_MAINTAINER_SURFACE_2026-07-13.md.

The main value is the governed request pipeline: webhook in → guardrails → LLM-assisted triage → human approval where needed → auditable execution throughout, with tenant isolation enforced at the database layer and observable at every step.