MicroQL is a declarative JSON-based query language for composing and orchestrating services: APIs, AI models, scrapers, databases, your CRM. You describe what each step needs; MicroQL works out the order, runs independent steps in parallel, retries and rate-limits the flaky ones, keeps failures from sinking the whole job, and remembers every result so an interrupted run picks up where it left off.
import query from 'microql'
import {scraper, claude, hubspot} from './services/index.js'
// Find manufacturers in the chamber of commerce directory and put them in the CRM
const leads = await query({
given: {directory: 'https://chamber.example.org/member-directory'},
services: {scraper, claude, hubspot},
phases: {
members: ['[scraper]/listings', {url: '$given.directory', cache: true}],
// every member is qualified in parallel; Claude calls are spaced out by the rate limit
qualified: ['$members', '[util]/each', {
service: ['POST [claude]/qualify', {
company: '@',
criteria: 'manufacturing, shipping, or industrial repair',
cache: true
}],
// a member Claude can't qualify is logged and left out; the rest carry on
catch: ['@err', '[util]/log', {to: 'failed/qualify.log'}],
skip: ['error']
}],
contacts: ['$qualified', '[util]/filter', {service: ['[util]/exists', {value: '@.fit'}]}],
created: ['$contacts', 'POST [hubspot]/companies']
},
settings: {rateLimit: {claude: 1000}, retry: 2},
select: '$created'
})If you're a programmer and you want to achieve the low risk/high productivity of something like Zapier or n8n, but without all the baggage that those services come with, use MicroQL. You may 1) make yourself more productive, 2) enable a non-coder on the team to be able to compose queries, enabling a division of labor where you and other coders can focus on building out new services, or working on something else entirely.
npm install microqlMicroQL needs Node.js 22 or later. import query from 'microql' gives you core with the node runtime wired in. Core alone, microql/core, has no environment dependencies and runs anywhere JavaScript does, Cloudflare Workers included, with a runtime you supply; see Runtime Integration.
A Query's named nodes are phases. Each phase is a service call, or a chain of them. $name refers to a phase's result, and that reference is how MicroQL knows the order: a phase runs as soon as everything it references is ready, and phases that don't depend on each other run at the same time.
phases: {
post: ['[cms]/post', {slug: '$given.slug'}],
// these three only need the post, so they're written in parallel
linkedin: ['POST [claude]/write', {format: 'linkedin', source: '$post'}],
thread: ['POST [claude]/write', {format: 'x-thread', source: '$post'}],
newsletter: ['POST [claude]/write', {format: 'newsletter', source: '$post'}],
// and this waits for all three
scheduled: ['POST [buffer]/schedule', {posts: ['$linkedin', '$thread'], email: '$newsletter'}]
}Circular references are caught before anything runs.
A call is named by its service key, [VERB] [service]/resource: POST [claude]/write calls the /write resource of the claude service with POST, and [util]/each calls util's /each, which takes no verb. Whether a resource takes a verb, and which, is the service's business. Services are passed in through services; a key naming a service, resource or verb that doesn't exist is an error before anything runs.
| Form | Means |
|---|---|
$name.path |
A phase's result, or $given — the inputs you supply |
@ / @.path |
The value flowing in: the previous call in a chain, the current item of an each |
@name.path |
The current item of the enclosing iteration named name by as: |
@err.path |
The failure, inside a catch (@@err for the one outside it) |
Paths are .field and [index] segments: $listings[0].images[1].src. A path that names nothing gives undefined.
Every string beginning with $ or @ is a reference. A literal like "$5.00" is supplied through given. Mistakes — an unknown name, @ where nothing flows in, the 0.x $.name form — are reported with the call's address before anything runs.
A service answers a key with an operation: an async function that takes an object of named fields and returns a promise, plus an optional declaration of those fields. Most services are a table of them, and wrapping an API, a model or a database takes a few lines:
import {serviceFromTable} from 'microql'
const headers = {Authorization: `Bearer ${process.env.HUBSPOT_TOKEN}`, 'Content-Type': 'application/json'}
export const hubspot = serviceFromTable({
'/companies': {
async GET({domain}) {
const response = await fetch(`https://api.hubapi.com/crm/v3/objects/companies/${domain}?idProperty=domain`, {headers})
return response.json()
},
POST: {
input: {on: ['target']},
async call({on: companies}) {
const response = await fetch('https://api.hubapi.com/crm/v3/objects/companies/batch/upsert', {
method: 'POST',
headers,
body: JSON.stringify({inputs: companies.map(c => ({idProperty: 'domain', id: c.domain, properties: c}))})
})
return response.json()
}
}
}
})A resource with several verbs has one operation per verb; a resource given as one operation takes keys with no verb. A declaration's tags say which field method syntax fills, which fields hold nested calls, and the types of the input and output. See the Service Development Guide.
A chain runs its calls in order, each receiving the previous one's result as @. The first call of a chain receives nothing.
summary: [
['[reviews]/google', {placeId: '$given.placeId'}],
['POST [claude]/summarize', {reviews: '@', focus: 'complaints'}],
['@', 'POST [slack]/post', {channel: '#customer-experience'}]
][util]/each runs a service for every item of a list, in parallel. A lone call receives the item as @; name the item with as: to reach it from a chain or a nested iteration:
ocr: ['$listings', '[util]/flattenEach', {as: 'listing',
service: ['@listing.images', '[util]/each', {as: 'image', service: [
['POST [claude]/ocr', {imageUrl: '@image.src', lookFor: ['name', 'phone', 'website']}],
['@', '[util]/merge', {with: {industry: '@listing.industry'}}]
]}]
}],[util]/flattenEach concatenates the lists its calls return. Iteration names are used once per Query, so @listing always means the same thing.
The first argument may be written on the left: ['$members', '[util]/each', {...}] is ['[util]/each', {on: '$members', ...}]. Services opt in by tagging the field that comes first: input: {on: ['target']}.
Real pipelines fail: a page won't load, a model returns junk, an API rate-limits you. Every call takes these settings:
| Setting | Effect |
|---|---|
retry: 3 |
Try again, up to 3 more times — including when the result fails its postcheck |
timeout: 30000 |
Fail after 30s |
postcheck: {...} |
Treat a wrong-shaped success as a failure: a Service Call Validation |
A failure is thrown, and fails the call holding it, and the one holding that, up to the Run, until a catch handles it. A catch sits on a call that holds other calls (each, filter, try, ...) and handles a failure of any of them, from anywhere below: it receives the failure as @err, sees the failed item by the call's as: name, and what it returns takes the failed call's place. The holding call carries on with the rest. A catch that throws fails the holding call in turn.
listings: ['$categories', '[util]/each', {as: 'category',
service: ['[scraper]/extract', {
url: '@category.href',
postcheck: {images: ['array', ['object'], {min: 1}]}
}],
// log each page that failed, and leave it out; keep going with the rest
catch: ['@err', '[util]/log', {to: 'failed/listings.log'}],
skip: ['error']
}],- The failure is data:
{'@type': 'Error', name, message, cause, address, key, queryHash, formHash, args}, a JSON-LD node of type Error with the address and arguments of the call that raised it. It's plain JSON, so it is stored and resumed like any result, and theerrorvalidation type matches it. skiponeachandflattenEachleaves out the results that fit a schema:['error'],['null'], or['union', ['error'], ['null']].- To fail anyway, rethrow:
catch: [['@err', '[util]/log', {to: 'failed.log'}], ['@', '[util]/throw']]. Or fail with an error of your own, keeping the failure as its cause:catch: ['[util]/throw', {on: 'Listings stopped', cause: '@err'}]. - To catch a single call, hold it in
[util]/try:['[util]/try', {service: ['[scraper]/extract', {...}], catch: [...]}]. - Retries come first: a
catchsees only a failure that outlasted its retries. An aborted Run catches nothing. settings.catchhandles each phase that fails: its result is the phase's, and the Run carries on.
An error names the call that raised it by its address: [$listings.@1a1 - [scraper]/extract] Timed out after 30000ms. A catch is always lettered e ($listings.@1e1), so adding one moves no other address.
The util service is always available:
| Resource | Does |
|---|---|
/each |
Run a call for every item; skip leaves out the results that fit a schema |
/flattenEach |
Run a call that returns a list for every item, and concatenate the lists; skip too |
/filter |
Keep the items a service is truthy for; mode: 'remove' removes them |
/try |
Run one call, so a catch can handle its failure |
/throw |
Fail with the error given (as data), or with an error saying what was given; cause attaches the error behind it |
/when |
then when test is truthy, otherwise otherwise |
/partitionErrors |
Split a list into {successes, failures} |
/concat, /pick, /merge, /template |
Shape data |
/eq, /gt, /lt, /exists, /length |
Tests |
/format |
A value as text |
/inspect, /log |
Write a value out, formatted with the Query's inspect settings or as plain text, to the console or to a destination (to:); both return it |
settings: {
timeout: 30000, // default for every call
retry: 2, // default for every call
rateLimit: {claude: 1000}, // at most one call per second to claude
concurrency: 6, // at most 6 calls in flight at once
cache: {ttl: '30D'}, // how long cached results last (default 7D)
debug: true // print every call, its arguments and its result
}Per call, cache: true (or cache: {ttl: '1D'}) reuses the result across runs while the inputs match, so you pay for an expensive scrape or model call once.
select picks what the Query returns: select: '$created', or select: ['$created', '$found'].
A Run is one or more executions of a Query against one Run State, the store holding the result of every call it made. MicroQL writes each result there as it goes and reuses any result already there, so running again against the same Run State resumes where the last attempt stopped. A Run is only a concept: MicroQL keeps no record of Runs, and the Run State is their only trace.
What the node runtime (import query from 'microql') does with results:
| Store | Where | Lasts | To clear it |
|---|---|---|---|
| Run State | in memory, fresh for each query() call, unless you pass {runState: fileStore(dir)} |
the call, or as long as dir exists |
start a new store, or await fileStore(dir).clear() |
Cache, for calls with cache set |
the .cache directory under the current working directory |
until its ttl passes; expired entries are swept when the store is first used | rm -rf .cache, or await nodeRuntime.cache.clear() |
What util/log and util/inspect write to: a destination |
appended to that file | until you delete it | delete the file |
fileStore(dir) keeps one directory per call form and one JSON file per input, so a Run State on disk can be inspected by hand. How core keys and reuses results is in Runtime Integration § Storage.
A Query is an ordinary module, so a script is all a command needs. Pass a Run State on disk and an interrupted run resumes without repeating finished work — the scrape you already paid for isn't paid for again:
#!/usr/bin/env -S node --env-file=.env
// queries/lead-gen.js
import query, {fileStore} from 'microql'
import {scraper, claude, hubspot} from '../services/index.js'
const today = new Date().toISOString().split('T')[0]
const directory = process.argv[2] ?? 'https://chamber.example.org/member-directory'
const created = await query({
given: {directory},
services: {scraper, claude, hubspot},
phases: {/* as above */},
select: '$created'
}, {runState: fileStore(`runs/lead-gen-${today}`)})
console.log(`Created ${created.length} companies`)
process.exit(0) // HTTP clients may keep connections alivechmod +x queries/lead-gen.js
./queries/lead-gen.js https://www.example-chamber.org/directoryRun it again the same day and every finished call is read back from runs/; start a new run by using a new directory. Failures a catch logs to: a file are there to inspect.
Pull reviews from several sources at once, flag the unhappy ones, and open a ticket for each:
await query({
given: {business: {googlePlaceId: 'ChIJ...', yelpId: 'acme-bakery'}},
services: {google, yelp, claude, zendesk, slack},
phases: {
google: ['[google]/reviews', {placeId: '$given.business.googlePlaceId', since: '7d'}],
yelp: ['[yelp]/reviews', {businessId: '$given.business.yelpId', since: '7d'}],
scored: [
[['$google', '$yelp'], '[util]/concat'],
['@', '[util]/each', {service: ['POST [claude]/sentiment', {review: '@', cache: true}]}]
],
unhappy: ['$scored', '[util]/filter', {service: ['[util]/lt', {l: '@.score', r: 3}]}],
tickets: ['$unhappy', '[util]/each', {service: ['POST [zendesk]/ticket', {
subject: '@.summary', body: '@.text', priority: 'high'
}]}],
digest: ['POST [slack]/post', {channel: '#reviews', scored: '$scored', tickets: '$tickets'}]
}
})Turn one blog post into a LinkedIn post, an X thread and a newsletter, written in parallel, each checked before anything is scheduled:
await query({
given: {slug: 'why-we-switched-to-four-day-weeks'},
services: {cms, claude, buffer, mailchimp},
phases: {
post: ['[cms]/post', {slug: '$given.slug'}],
linkedin: ['POST [claude]/write', {
format: 'linkedin', voice: 'warm, plain-spoken', source: '$post',
postcheck: {text: ['string', {max: 3000}]}, retry: 2
}],
thread: ['POST [claude]/write', {
format: 'x-thread', source: '$post',
postcheck: {posts: ['array', ['string', {max: 280}], {max: 8}]}, retry: 2
}],
newsletter: ['POST [claude]/write', {format: 'newsletter', source: '$post'}],
social: ['POST [buffer]/schedule', {posts: ['$linkedin', '$thread'], at: 'next weekday 9am'}],
email: ['POST [mailchimp]/draftCampaign', {content: '$newsletter', list: 'subscribers'}]
}
})A draft that fails its postcheck — a thread entry over 280 characters — is rewritten, up to twice. If it still fails, the Run stops before anything is scheduled, with the call's address in the error; run it again with the same Run State (see Running from the Command Line) and only that draft is rewritten, while the post and the drafts that passed are read back.
- Service Development Guide - How to write and structure services
- Validation System - Type validation and contracts
- Runtime Integration - Building on MicroQL beyond a node script: core, runtimes and platforms, storage, planning, partial runs, progress updates and abort
npm testperformance.md and function-names.md mix prose with generated sections, marked <!-- generated:name --> … <!-- /generated:name -->. The scripts rewrite only the text between their markers:
npm run docs:bench # performance.md results: this checkout vs 0.6.6, about a minute
npm run docs:functions # function-names.md tables: after changing coreGraphQL is primarily developed for field extraction, whereas MicroQL was designed for service orchestration. You can do service orchestration with GraphQL but it requires that you put a lot more leg work into resolvers and that you manually think about dependency resolution. MicroQL infers dependency resolution and parallelism from the paths that are referenced. MicroQL also doesn't invent a completely new syntax - it's mostly a subset of JSON. We found ways to do that without being pedantic and verbose.
The MicroQL codebase is also small, dependencies zod, @noble/hashes and @tlsft/service-key.
See here for a more detailed comparison.
I think the problem that most people are trying to solve with TypeScript would be better solved with input validations. That is: you can type for internal consistency all you like, but as soon as you incorporate something outside your domain, your static type analysis fails. MicroQL is literally for composing things outside of your domain.
Secondly, Javascript compilation layers became a Tower of Babel over ten years ago. Your main tool is literally called babel. Do you really need all that? Maybe all you need is JSON. Try it out.
It's faster than n8n, slower than native Javascript. Performance optimization should be considered in its infancy - particularly I have not tried this with any long running processes to ensure that there are no memory leaks.
MIT
MicroQL - Microservice orchestration made simple.