Skip to content

Fix/installation assurance - #529

Merged
CraftOS-dev merged 6 commits into
V1.4.5from
fix/installation-assurance
Nov 16, 2026
Merged

CraftOS-dev merged 6 commits into
V1.4.5from
fix/installation-assurance

Conversation

@CraftOS-dev

Copy link
Copy Markdown
Owner

Fix installation and setup breaking issue.

The install path failed for most machines that were not a Windows box with
the right Python already on it. This makes the install self-contained and
fast everywhere.

Interpreter
- install.py runs on any Python 3.8+ and never prompts. CraftBot always
  installs into its own private Python 3.10 (python-build-standalone, from
  app/provision/python_downloads.json), never into the user's: Ubuntu
  22.04's system 3.10 has no pip or ensurepip, Homebrew's is externally
  managed, and installing 200 pinned packages into a user's interpreter
  rewrites their other projects. CRAFTBOT_PYTHON and an active conda env
  still pin an interpreter explicitly.
- Removed the version prompts and the python.org/apt auto-installer (its
  "recommended" path downloaded six 404s and landed on 3.10.11).
- Child processes run in an isolated environment (PYTHONHOME, PYTHONPATH,
  user site-packages and non-network PIP_* settings stripped).
- Tar extraction validated by our own code: tarfile's filter="data" is
  missing before 3.10.12/3.11.4 (macOS 3.9.6, python.org 3.10.11) and
  rejects valid hard links on Ubuntu 22.04's 3.10.12.

Dependencies
- torch, sentence-transformers and docling removed from the core install.
  The memory embedder (bge-small-en-v1.5) runs on ONNX Runtime with
  vectors identical to the sentence-transformers ones (max diff 1.8e-7),
  and scanned-PDF OCR uses RapidOCR. Linux drops from 3.26 GB of CUDA
  wheels to about 0.5 GB; Windows and macOS to about 0.4 GB.
- Locks for win_amd64, macosx_arm64, macosx_x86_64, linux_x86_64 and
  linux_aarch64, generated with uv from one machine. Every pin lists the
  hash of every file published for it, so older macOS and older glibc
  pick a compatible wheel and still verify. Versions match V1.4.5's locks.
- The lock is installed with uv (pinned by hash): ~1 minute instead of
  ~11 with pip. Byte-compilation runs in the background.
- The embedding model is downloaded at install, pinned by SHA-256 in
  app/provision/embedding_model.json, so the first start never depends on
  huggingface.co.
- openpyxl and pandas declared directly (they only arrived via docling).
- read_pdf no longer pip-installs packages at runtime.
- ripgrep bundled for Intel Mac, Linux ARM and Windows ARM.

Networks
- HF_ENDPOINT and CRAFTBOT_MIRRORS rewrite runtime downloads (Python,
  Node, PocketBase, model); the model is still checked by SHA-256.
- Downloads trust the OS certificate store as well as certifi; Node
  children get NODE_USE_ENV_PROXY and, on Node 24+, --use-system-ca; npm,
  Node and uv get the OS proxy when no proxy variable is set.

Checks
- Windows path-length preflight before any download; disk requirement
  measured on the drive that receives the install (3 GB).
- Visual C++ runtime registry check repaired (the key path held a literal
  form feed and never matched).
- Conda mode installs Miniforge (no Anaconda ToS prompt), detects conda
  outside PATH, and the macOS installer arch is detected correctly.

Verified end to end on this branch: Windows from Python 3.11.0 in 610 s
and stock Ubuntu 22.04 python3 without pip in 220 s, both previously
blocked; all five locks dry-run installed for their target platforms.
Built on V1.4.5's login startup (installer/startup.py, run.py --service),
which stays as is; this adds what it did not cover.

- craftbot.py start/install watch the process the whole time; a crash
  prints the log tail and exits non-zero instead of claiming success and
  closing the window. run.py exits with the agent's exit code.
- run.py writes STATE_ROOT/craftbot.pid and ui.json (the live UI
  address), shared by craftbot.py, the launcher and status; windowless
  starts log to STATE_ROOT/craftbot.log.
- If 7925/7926 are taken by another program, the next free pair is used;
  the backend port is injected into the served UI at runtime, so a
  prebuilt UI works on any port. Only a CraftBot of the same STATE_ROOT is
  ever stopped.
- CRAFTBOT_BIND_HOST for Docker and remote use; the Host/Origin guard is
  unchanged. Loopback calls never go through HTTP(S)_PROXY.
- Login services get the PATH, Python, Node, conda and proxy/mirror
  settings captured at install (service-env.json on Windows, loaded under
  --service); a missing systemd and the linger requirement are explained.
- The agent starts before the frontend build so the two overlap, and the
  launcher no longer imports the whole agent stack (via app.config) just
  to find its process ledger: a warm start went from 121 s to 56 s.
- The file index prewarms the agent workspace by default instead of every
  drive (no macOS privacy prompts, no recursive watch on /).
- Background --cli is refused with a pointer to run.py --cli; the console
  close no longer uses wmic or kills the parent shell.
- Uninstall removes CraftBot's private runtimes and npm trees and never
  touches a foreign interpreter or pip cache.
- The updater refuses cleanly outside a git checkout (Docker, ZIP,
  managed installs).
- Runtime files are seeded from shipped defaults (app/config/*.default.json
  and the agent file templates) when missing.
settings.json (API keys after onboarding), the other app/config JSON files
and the agent's MEMORY.md, ENTITIES.md and PROACTIVE.md were tracked and
rewritten while CraftBot runs, so `git pull` refused to update any
checkout that had been used. The tracked copies also shipped one
developer's memory, entities and provider auth modes to every new user.

They are now created on first start from shipped defaults
(app/config/<name>.default.json and app/data/agent_file_system_template),
only when missing, and ignored by git. The defaults drop the leftover
api_keys_configured and auth_mode entries (auth_mode defaults to api_key
in code).

The in-app updater carries the user's copies across the one pull that
untracks them (verified with a simulated repository for both updater.sh
and updater.bat: the pull succeeds and the user's API key and memory are
kept). AGENT.md, SOUL.md, FORMAT.md and GLOBAL_AGENT_APP.md stay tracked.
- Linux launchers build in manylinux_2_28 containers (glibc 2.28: Ubuntu
  20.04+, Debian 11+, RHEL 8+), fontconfig loaded at runtime. A gate fails
  the build on any newer glibc symbol or extra linked library, and the
  smoke test runs on Ubuntu 20.04-24.04, Debian 11-13 and AlmaLinux 8/9.
- Linux aarch64 installer added; Linux assets named by architecture.
- Windows Authenticode (Azure Artifact Signing) and macOS Developer ID
  signing, notarization and stapling when the secrets exist, skipped with
  a warning otherwise. Release notes are generated from what was signed,
  with the macOS 15 "Open Anyway" steps when unsigned.
- Linux installs into <data>/source: Uninstall used to delete <data>/app,
  which is also where settings and chat data live.
- Payload downloads retry, are verified against a published SHA-256, and a
  stale local zip is used only when its version matches. V1.4.5's Python
  download and its fallback release are unchanged, now with retries.
- System proxy support (Windows, macOS, GNOME) and CRAFTBOT_MIRRORS for
  every launcher download.
- Child Python processes run isolated from PYTHONHOME/PYTHONPATH/PIP_*.
- The install is recorded only after setup succeeds; failures show Repair
  with the setup log tail; a newer launcher offers Update.
- Running state from STATE_ROOT/craftbot.pid; Open uses ui.json.
- macOS 12 minimum checked before any download; disk check on the drive
  that receives the data.
- New install-e2e workflow: real dependency installs on Windows, macOS
  arm64 and Intel, Ubuntu 22.04, Ubuntu ARM and a Python 3.8 launcher,
  then a start and a 200 from the UI.
- Removed the stale launcher/packaging/release.yml copy and the unused
  OmniParser image build.
- README (all nine languages): installer first, then the source install
  with the right command per OS (python3 on macOS/Linux), a shallow clone
  or ZIP, and no Node.js or specific Python version required.
  Troubleshooting covers logs, ports, proxies, mirrors, headless servers,
  Gatekeeper, SmartScreen and auto-start on Linux. Removed the stale Node
  18, npm and pip advice and the dead INSTALLATION_FIX.md link.
- Docker runs the browser UI: multi-stage image with the prebuilt UI,
  Node 24, the locked dependencies and the embedding model baked in, a
  /data volume for all state, ports published on 127.0.0.1, a health
  check, and no required .env. The OmniParser service is removed.
- environment.yml uses conda-forge only (the defaults channel now stops
  unattended installs at Anaconda's ToS) and is ASCII so any locale can
  read it.
- .env.example lists only the variables CraftBot actually reads.
@zfoong
zfoong force-pushed the fix/installation-assurance branch from ab28824 to 1f6e5db Compare November 16, 2026 22:56
The parity fingerprint's embedding entry dropped its "effective" field
when the embedder moved to ONNX Runtime, so the compare step raised
KeyError on every platform. It is back, meaning the same as before: the
model this install's packages can actually run.
@CraftOS-dev
CraftOS-dev marked this pull request as ready for review November 16, 2026 23:43
@CraftOS-dev
CraftOS-dev merged commit 1b577e6 into V1.4.5 Nov 16, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants