onlyworlds.okf: the OKF exporter (0.6.0, waits on B10)

CHANGELOG.md

@@ -2,6 +2,12 @@
 
 The package version runs on its own semver; the schema it is generated from is `SCHEMA_DIST_TAG`.
 
+## 0.6.0 (2026-10-17)
+
+- **Export a world folder to Open Knowledge Format (OKF) v0.2** (`onlyworlds.okf`, Keel #191): `export_okf(folder, out_dir)` or `python -m onlyworlds.okf FOLDER OUT-DIR` writes one markdown concept per element (fields in the frontmatter, text under headings, links as bundle paths and as markdown links), the World as `world.md`, and an index per type. The same folder gives the same bytes. Nothing is left out in silence: a file the reader skipped, a key that is not a field of its type, or a link that is not an id or points at the wrong type is a note in the returned `OkfReport` (stderr from the command line). Ported from schema-dist's `walk/okf_export.py`, it reads through `read_folder` and the generated schema tables. It differs from the branch on purpose: no `generated` actor (OKF §7 asks for a `human:` actor on hand-written words, and a folder does not say who wrote them), in-prose `ow://` references become links into the bundle, and a text line that already begins with a backslash gets one more, so every escaped line reads back.
+- **PyYAML only for OKF**: the export needs the new `okf` extra (`pip install "onlyworlds[okf]"`). `import onlyworlds` still imports nothing outside the standard library.
+- **The schema module grows** (same pin, `v0.32.0-dist.23`): `BASE_FIELDS`, `WORLD_FIELDS`, `SECTIONS` (each type's fields under their schema groups, in order), `LINK_TARGETS`, `GENERIC_LINKS` and `EXTENSION_PREFIXES`, generated from the vendored walk like the rest and checked against `FIELD_KINDS` and `GENERIC_PAIRS` at generation.
+
 ## 0.5.1 (2026-10-17)
 
 - **The reader skips a dot-prefixed file in a type directory**, a tool's litter rather than an element, and lists it in `Folder.skipped`, even when it holds a valid body. Before, it was read like any file, and one that sorted first could stand in for the real element (Skeld's clarification of the folder spec, 2026-10-17).

README.md

@@ -1,6 +1,6 @@
 # onlyworlds (Python)
 
-OnlyWorlds is an open standard for worldbuilding data: 22 element types (characters, locations, events, laws and the rest) that any tool can read and write. This is a Python client for its API, and a reader and writer for OnlyWorlds world folders (a world as a folder of JSON files). Python 3.12+, no runtime dependencies.
+OnlyWorlds is an open standard for worldbuilding data: 22 element types (characters, locations, events, laws and the rest) that any tool can read and write. This is a Python client for its API, and a reader and writer for OnlyWorlds world folders (a world as a folder of JSON files). Python 3.12+, no runtime dependencies (the optional OKF export needs PyYAML).
 
 ## Install
 
@@ -71,6 +71,24 @@ result = push("moppetopia-before", "moppetopia", api_key="ow_w_…", pin="ow_s_
 
 `push` sends only the fields that changed, retries what the server asks it to retry, checks that what came back is what it sent, and logs every write, so running it again skips what already landed. `dry_run=True` shows the plan without sending it.
 
+## Export to OKF
+
+A world folder can be exported as a valid [Open Knowledge Format](https://github.com/GoogleCloudPlatform/open-knowledge-format) (OKF) 0.2 bundle, as the spec stands at one pinned commit (`onlyworlds.okf.OKF_COMMIT`): one markdown file per element, with its fields in the frontmatter, its text under headings and its links (in-prose `ow://` references included) as markdown links, plus an index per type. This needs PyYAML, which the rest of the package does not:
+
+```bash
+pip install "onlyworlds[okf]"
+python -m onlyworlds.okf moppetopia moppetopia-okf
+```
+
+```python
+from onlyworlds.okf import export_okf
+
+report = export_okf("moppetopia", "moppetopia-okf")
+print(report.elements, "elements,", len(report.notes), "notes")
+```
+
+The output directory must not exist yet. Anything the export leaves out (a file that isn't an element, a key that isn't a field, a link to the wrong type) is listed in `report.notes`; the command prints these on stderr. The same folder always gives the same bytes. What a bundle never carries: the server's bookkeeping (`change_seq`, `created_at`, who wrote what) and the world's key, so a bundle is a copy to read elsewhere, not a folder to sync.
+
 ## Errors
 
 A failed call raises `ApiError` with the API's `code`, `param` (the field that failed) and `doc_url`, and flags for the cases you handle differently: `is_validation_error`, `is_auth_error`, `is_not_author`, `is_owner_only`, `is_id_conflict`, `is_resync_required`, `is_busy`, `is_precondition_failed`.
@@ -107,6 +125,8 @@ Three groups of tests run only when asked:
 - `OW_OPENAPI=https://www.onlyworlds.com/api/v2/openapi.json` (or a saved copy): checks the client's routes, parameters and fields against the API's own OpenAPI document.
 - `OW_LIVE=1`: read-only calls against the demo world.
 
+`OKF_REFERENCE_SRC=<path>` (the `src/` of an open-knowledge-format checkout) also runs OKF's own parser and viewer over an exported bundle.
+
 
 ## License
 

codegen/generate_schema.py

@@ -126,6 +126,70 @@ def generic_pairs(walk: Any, types: list[str]) -> dict[str, list[tuple[str, str]
     return out
 
 
+def decoded_schema(walk: Any) -> dict[str, Any]:
+    """The walk's decode_schema (the shape schema.json ships), with the same notes policy as field_kinds."""
+    notes: list[str] = []
+    decoded: dict[str, Any] = walk.decode_schema(DIST / "schema", note=notes.append)
+    unexpected = [n for n in notes if not n.startswith(EXPECTED_NOTES)]
+    if unexpected:
+        raise SystemExit(f"the walk raised notes this generator does not know: {unexpected}")
+    return decoded
+
+
+def layout_tables(
+    decoded: dict[str, Any],
+    types: list[str],
+    kinds: dict[str, dict[str, str]],
+    pairs: dict[str, list[tuple[str, str]]],
+) -> dict[str, Any]:
+    """What a reader of whole elements needs beyond FIELD_KINDS: the base and World field names in
+    order, each type's fields under their YAML groups, each link's target, each generic link's two
+    keys. Cross-checked against FIELD_KINDS and GENERIC_PAIRS, so the tables cannot drift apart."""
+    sections: dict[str, list[tuple[str, list[str]]]] = {}
+    targets: dict[str, dict[str, str]] = {}
+    generic: dict[str, dict[str, tuple[str, str]]] = {}
+    for t in types:
+        groups: list[tuple[str, list[str]]] = []
+        for f in decoded["types"][t]:
+            if not groups or groups[-1][0] != f["section"]:
+                groups.append((f["section"], []))
+            groups[-1][1].append(f["name"])
+            if f["kind"] in ("single", "multi"):
+                if not f.get("target"):
+                    raise SystemExit(f"{t}.{f['name']}: a link with no target from the walk")
+                targets.setdefault(t, {})[f["name"]] = f["target"]
+            elif f["kind"] == "generic":
+                generic.setdefault(t, {})[f["name"]] = (f["type_key"], f["id_key"])
+            if f["kind"] != "generic" and f["name"] not in kinds[t]:
+                raise SystemExit(f"{t}.{f['name']}: in the walk's sections but not in FIELD_KINDS")
+        if len({s for s, _ in groups}) != len(groups):
+            raise SystemExit(f"{t}: a section split in two by the walk's order")
+        sections[t] = groups
+        if sorted(generic.get(t, {}).values()) != sorted(pairs.get(t, [])):
+            raise SystemExit(
+                f"{t}: the walk's generic keys {generic.get(t)} disagree with GENERIC_PAIRS {pairs.get(t)}"
+            )
+    return {
+        "base": [f["name"] for f in decoded["base"]],
+        "world": [f["name"] for f in decoded["world"]],
+        "sections": sections,
+        "targets": targets,
+        "generic": generic,
+    }
+
+
+def extension_prefixes() -> tuple[str, ...]:
+    """The extension prefixes as the dist's own folder lint declares them (ruling extension-passthrough)."""
+    spec = importlib.util.spec_from_file_location("lint_folder", DIST / "walk" / "lint_folder.py")
+    assert spec is not None and spec.loader is not None
+    lint = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(lint)
+    prefixes = tuple(lint.EXTENSION_PREFIXES)
+    if not prefixes or not all(isinstance(p, str) and p.endswith("_") for p in prefixes):
+        raise SystemExit(f"lint_folder.EXTENSION_PREFIXES is not a tuple of prefixes: {prefixes!r}")
+    return prefixes
+
+
 def element_types(walk: Any) -> list[str]:
     from_walk: list[str] = list(walk.ELEMENT_TYPES)
     from_files = sorted(p.stem for p in (DIST / "schema").glob("*.yaml") if p.stem not in NON_ELEMENT_SCHEMA_FILES)
@@ -139,13 +203,31 @@ def render(
     types: list[str],
     kinds: dict[str, dict[str, str]],
     pairs: dict[str, list[tuple[str, str]]],
+    layout: dict[str, Any],
+    prefixes: tuple[str, ...],
 ) -> str:
+    def strs(names: list[str]) -> str:
+        return "(" + ", ".join(f'"{n}"' for n in names) + ("," if len(names) == 1 else "") + ")"
+
+    def wrapped(head: str, names: list[str], tail: str) -> list[str]:
+        """``head(names)tail`` on one line, or the names packed onto lines within ruff's 120."""
+        if len(head + strs(names) + tail) <= 120:
+            return [head + strs(names) + tail]
+        indent = " " * (len(head) - len(head.lstrip()))
+        out, cur = [head + "("], ""
+        for piece in (f'"{n}",' for n in names):
+            if cur and len(indent) + 4 + len(cur) + 1 + len(piece) > 120:
+                out.append(indent + "    " + cur)
+                cur = ""
+            cur = f"{cur} {piece}" if cur else piece
+        return [*out, indent + "    " + cur, indent + ")" + tail]
+
     lines = [
         "# GENERATED by codegen/generate_schema.py from the vendored schema distribution. Do not hand-edit.",
         "# Regenerate: python codegen/generate_schema.py  (CI/test: --check)",
         f"# Source: {pin['repo']} @ {pin['tag']} ({pin['commit']})",
         "",
-        '"""Schema provenance and the 22 element types, generated from the pinned schema distribution."""',
+        '"""Schema provenance, the 22 element types and their fields, from the pinned schema distribution."""',
         "",
         f'SCHEMA_DIST_TAG = "{pin["tag"]}"',
         f'SCHEMA_VERSION = "{pin["canonical_version"]}"',
@@ -173,6 +255,48 @@ def render(
         *[f'    "{t}": ({", ".join(f'("{a}", "{b}")' for a, b in pairs[t])},),' for t in types if t in pairs],
         "}",
         "",
+        "# Every element's shared fields (base_properties.yaml), in schema order, as data spells them.",
+        *wrapped("BASE_FIELDS: tuple[str, ...] = ", layout["base"], ""),
+        "",
+        "# The World's fields (world.yaml), in schema order.",
+        *wrapped("WORLD_FIELDS: tuple[str, ...] = ", layout["world"], ""),
+        "",
+        "# type -> ((section, (field, ...)), ...): each type's own fields in schema order, under the YAML",
+        "# group that holds them. A generic link is listed by its own name (pin's `element`); GENERIC_LINKS",
+        "# names the two keys that hold it in data.",
+        "SECTIONS: dict[str, tuple[tuple[str, tuple[str, ...]], ...]] = {",
+    ]
+    for t in types:
+        lines.append(f'    "{t}": (')
+        for s, names in layout["sections"][t]:
+            lines.extend(wrapped(f'        ("{s}", ', names, "),"))
+        lines.append("    ),")
+    lines += [
+        "}",
+        "",
+        "# type -> {link field: the type it links to}, for every single and multi link.",
+        "LINK_TARGETS: dict[str, dict[str, str]] = {",
+    ]
+    for t in types:
+        if t in layout["targets"]:
+            lines.append(f'    "{t}": {{')
+            lines.extend(f'        "{name}": "{target}",' for name, target in layout["targets"][t].items())
+            lines.append("    },")
+    lines += [
+        "}",
+        "",
+        "# type -> {generic link field: (type key, id key)}: the walk's names for the two keys that hold it.",
+        "GENERIC_LINKS: dict[str, dict[str, tuple[str, str]]] = {",
+        *[
+            f'    "{t}": {{{", ".join(f'"{n}": ("{a}", "{b}")' for n, (a, b) in layout["generic"][t].items())}}},'
+            for t in types
+            if t in layout["generic"]
+        ],
+        "}",
+        "",
+        "# Prefixes of the extension keys a tool may add to any element (dist walk/lint_folder.py).",
+        f"EXTENSION_PREFIXES: tuple[str, ...] = {strs(list(prefixes))}",
+        "",
     ]
     return "\n".join(lines)
 
@@ -184,7 +308,9 @@ def main() -> int:
     pin = verify_dist()
     walk = load_walk()
     types = element_types(walk)
-    text = render(pin, types, field_kinds(walk, types), generic_pairs(walk, types))
+    kinds, pairs = field_kinds(walk, types), generic_pairs(walk, types)
+    layout = layout_tables(decoded_schema(walk), types, kinds, pairs)
+    text = render(pin, types, kinds, pairs, layout, extension_prefixes())
     if args.check:
         current = OUT.read_bytes().decode("utf-8") if OUT.is_file() else ""
         if current != text:

pyproject.toml

@@ -11,12 +11,15 @@ classifiers = [
     "Typing :: Typed",
 ]
 
+[project.optional-dependencies]
+okf = ["PyYAML>=6"]       # onlyworlds.okf, the OKF bundle export; the core imports no YAML
+
 [dependency-groups]
 dev = [
     "pytest>=8",
     "ruff>=0.6",
     "mypy>=1.11",
-    "pyyaml>=6",          # build-time only: codegen imports the vendored schema walk
+    "pyyaml>=6",          # codegen imports the vendored schema walk; tests/test_okf.py needs it too
     "types-PyYAML",
 ]