Skip to content

feat(typesafe): add Jev decision integration - #8119

Closed
BillLeoutsakosvl346 wants to merge 4 commits into
stagingfrom
jev
Closed

BillLeoutsakosvl346 wants to merge 4 commits into
stagingfrom
jev

Conversation

@BillLeoutsakosvl346

@BillLeoutsakosvl346 BillLeoutsakosvl346 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a TypeSafe (Jev) block and four Agent tools: Choose an Option, Assign a Score, Estimate Yes/No Probability (Noul), and Evaluate Questions. They call TypeSafe’s System One endpoint with a user-supplied API key and use Sim’s existing request transport, retries, and model-input projection.

Preserve structured inputs, nullable question descriptions, fractional scores, complete probability maps, and named mixed-question answers. Validate choices, probability keys, score ranges, and structured legends against the request actually sent after model-input projection. Include seven catalog templates and seven suggested skills, documentation, and regenerated tool metadata, catalog, navigation, icon mappings, block-name metadata, and Copilot docs manifest. No SDK dependency, provider implementation, API route, OAuth, billing, or migrations.

Use the user-supplied standalone TypeSafe mark (typesafe-ai-dark.svg) for the block, catalog templates, and documentation. Preserve its exact path and viewBox; use currentColor for light/dark theme support.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Other

Testing

Used Bun 1.4.1 and installed existing locked dependencies with --frozen-lockfile.

All PR checks passed on 812299aeb4: app build, lint/type checks and repository audits, test shards, both PostgreSQL integration jobs, companion, security scan, docs preview, and automated review checks. Greptile reports 5/5; both reviewers have zero open threads.

Passed:

  • Workspace formatting, lint, and type checks.
  • All 47 repository audits, plus block-registry and canvas-sentence checks.
  • Tool request-boundary, registry-boundary, parameter-reachability, and strict API-validation checks.
  • Tool metadata, integration catalog, deployment configuration, documentation, and docs-manifest drift checks.
  • Bare-icon, icon-path, and icon-precision checks with the supplied TypeSafe icon.
  • 368 existing tests across executor, request transport, model-input adapters, block definitions, and parameter handling.

Manual verification:

  • All four block forms and Agent operations appear in the local UI. Criteria examples and required state match each operation; the advanced model combobox offers the three initial suggestions and accepts a custom ID.
  • Request/response probes covered text, JSON strings, resolved objects/arrays, shared reference navigation, fractional scores, zero/one Noul values, mixed answers, invalid JSON/criteria, missing or mismatched answers, and low confidence as success.
  • Nested model-input projection preserves authentication, model selection, and question-type controls, and rejects ambiguous question-ID projection instead of reassigning content.
  • Live HTTP 200 calls passed for all four operations using the configured key. Score returned 0.67 with a structured legend; mixed answers preserved question IDs and zero probabilities. Live negative calls confirmed HTTP 401 for invalid credentials and HTTP 400 for an invalid question type.

Live throttling/overload and cancellation were not induced; retry/error behavior uses the existing executor and its existing tests. No complete Agent model-driven conversation was executed.

No tests were added or modified, as requested. Existing CI remains enabled. The API key is excluded from the commit.

The final response-validation change also passed all 47 audits, workspace type checks, 228 existing executor/transport/projection tests, manual malformed-response and projection probes, and four successful live API calls. Probes included undeclared choices, missing/extra outcomes, out-of-range scores, mismatched legends, and comparison against projected option names and descriptions.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing — no test changes requested; existing tests passed
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

Screenshots/Videos

Local UI verification completed. The final React icon was visually checked in a browser at 16, 24, 32, and 48 px on light, dark, and block-tile backgrounds. Its original SVG path and viewBox are preserved exactly.

@vercel

vercel Bot commented Sep 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Sep 21, 2026 9:21pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 24 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with the previously identified response-validation gap addressed and no new actionable issue found.

Summary

Adds a TypeSafe (Jev) workflow integration with four operations, Agent-tool support, structured input projection, response validation, documentation, catalog metadata, and generated registry artifacts. The latest revision completes criteria-relative validation using the request body produced after model-input projection.

  • Adds Choice, Score, Noul, and mixed-question Evaluate operations.
  • Validates answer IDs, types, option/rubric keys, score bounds, and score legends against the projected request.
  • Registers the integration across workflow blocks, Agent tools, documentation, icons, deployment metadata, and generated catalogs.
  • Both previous review threads are resolved: criteria validation was implemented, while additional test files were intentionally excluded from the agreed scope.
Diagram
sequenceDiagram
  participant User
  participant Block as TypeSafe Block / Agent Tool
  participant Projector as Model-input projection
  participant Executor as Sim request executor
  participant Jev as TypeSafe System One
  participant Validator as Response validator

  User->>Block: State, question criteria, API key, model
  Block->>Projector: Select model-visible input
  Projector->>Executor: Projected parameters
  Executor->>Jev: POST state, model, and questions
  Jev-->>Executor: Typed answers, model, and usage
  Executor->>Validator: Response and serialized request body
  Validator->>Validator: Check IDs, types, criteria, bounds, and legends
  Validator-->>User: Operation-specific workflow output
Loading

Reviews (2) · Last reviewed commit: "fix(typesafe): validate answers against ..."

Comment thread apps/sim/tools/typesafe/utils.ts
Comment thread apps/sim/tools/typesafe/model-input.ts
@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@greptile

@BillLeoutsakosvl346

Copy link
Copy Markdown
Contributor Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@BillLeoutsakosvl346 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 26 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Re-trigger cubic

@waleedlatif1

Copy link
Copy Markdown
Collaborator

superseded by #8103

@waleedlatif1
waleedlatif1 deleted the jev branch September 21, 2026 22:05

This branch was previously deployed

1 inactive deployment
Preview 812299ae Deployed Sep 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants