Skip to content
smartloop-labsPublic

About

gpushed: your own GPU shed. Spin up an NVIDIA GPU box on Google Cloud with one command

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

76 Commits

Folders and files

Repository files navigation

gpushed

macOS only: Macs with Apple silicon on macOS 26+, using Apple's native container runtime. Linux, Windows and Intel Macs aren't supported.

Your dedicated NVIDIA GPU, on your Mac. Sign in with your email, add credit, and get an NVIDIA GPU box (Ubuntu 22.04 with the driver, CUDA, Python, Node.js and Rust ready) in a few minutes. There's no cloud account, quota request or Terraform to set up. You pay per second from a prepaid balance and delete the box when you're done.

Install. This also runs gpushed init, which sets up Apple's container runtime and signs you in by email:

curl -fsSL gpushed.dev/install.sh | bash

Then:

gpushed topup        # add credit with Stripe (minimum $10)
gpushed up           # create your box (default: 1x L4 24 GB)
gpushed ssh          # connect; try nvidia-smi
gpushed down         # delete it

Requirements

  • A Mac with Apple silicon on macOS 26 or later
  • Apple's native container runtime, which gpushed init sets up for you: it downloads the latest signed installer from https://github.com/apple/container/releases, checks it's signed by Apple (Developer ID "Apple Inc. - Containerization") and notarized, installs it (asks for your password once), and starts it

The gpushed client runs inside an Apple container, so nothing else needs to be installed. init also downloads the client image from our release bucket.

Plans and pricing

gpushed plans
Plan GPU Machine Price / hour (US)
t4 1x Tesla T4 16 GB n1-standard-8 $0.91
l4 (default) 1x L4 24 GB g2-standard-4 $0.88
a100 1x A100 40 GB a2-highgpu-1g $4.44
h100 (Spot, Taiwan) 1x H100 80 GB a3-highgpu-1g $9.19

h100 runs as a GCP Spot VM: GCP sells the single-H100 machine only that way. It's cheaper, but Google may reclaim it at any time. The box is then stopped with its disk kept (whatever was running ends), you get an email, and gpushed start resumes it once an H100 is free. Save checkpoints often.

Prices are GCP's on-demand cost (Spot for h100) of the machine, its 100 GB SSD and IP, plus 20%. A box is billed per second from when it's created until it's deleted. A stopped box costs only its disk (about $0.03/hour).

Regions. Boxes run in us-central1 (Iowa), us-east1 (South Carolina), us-west1 (Oregon), europe-west4 (Netherlands), asia-southeast1 (Singapore) and asia-east1 (Taiwan); each GPU is offered where we have quota for it (H100: Taiwan). A new box goes to a region with GPUs to spare, nearest you first (from Cloudflare's estimate of where your request comes from), and only then to one that looks full. It's billed at the price of the region it lands in. Europe is about 3-5% above the US and Singapore 9-23%; gpushed plans shows the prices for your region, gpushed plans --region asia-southeast1 for another, and gpushed up l4 --region europe-west4 picks the region yourself.

Balance. Credit is prepaid: the minimum top-up is $10, and you can top up any amount above that at any time with gpushed topup 25. To create or start a box you need at least one hour of its price in your balance.

Running out. When your balance falls below $2 you get an email. When it's nearly used up, your box is stopped (its disk is kept) and you get another email. Top up and run gpushed start to carry on. A box that stays stopped with no credit for 24 hours is deleted.

Commands

Command What it does
gpushed init Install Apple's container runtime, fetch the client, sign in
gpushed login Sign in with your smartloop account in the browser (--email [address]: with an emailed code instead)
gpushed logout Sign out on this Mac
gpushed balance Show your balance
gpushed topup [usd] Add credit (opens Stripe Checkout; minimum $10). Without an amount, pick $10, $25, $50 or $100
gpushed history Recent charges and top-ups
gpushed plans GPUs available and their prices
gpushed up [plan] Create your box and watch it install the driver + CUDA
gpushed status Show your box
gpushed ssh [cmd] SSH into the box, or run a command on it. Starts a stopped box first, or creates one (default plan) if you have none
gpushed stop / start Pause and resume (disk kept)
gpushed down Delete the box and its disk
gpushed update Upgrade to the latest release
gpushed uninstall Remove gpushed from this Mac

Your API token and the box's SSH key are in ~/.gpushed/state. To SSH from outside gpushed: ssh -i ~/.gpushed/state/box_ssh_key boxuser@<ip> (the IP is in gpushed status; it changes after a stop/start).

What you get

  • An Ubuntu 22.04 VM with the NVIDIA server driver and CUDA Toolkit 12.6
  • The everyday toolchain, preinstalled: Python 3.11 (python, pip, uv; the system python3 stays 3.10), Node.js 24 LTS with npm, Rust stable (rustup, cargo), and build-essential, git, cmake, tmux, htop and jq. You have root to add anything else
  • A 100 GB SSD and a public IP; only SSH (port 22) is open
  • Boxes boot from a prebuilt image with all of that baked in, so a new box is ready a minute or two after Google Cloud starts it. (If the image is ever unavailable, the box installs everything on first boot instead, which takes 10–15 minutes; gpushed up streams the progress.) If nvidia-smi can't talk to the driver, sudo reboot once.

Boxes run in smartloop's GCP project. They have no Google credentials and can't reach each other.


How it works

 Mac ─ gpushed ─► Apple container (gpushed-cli image, from R2; no cloud keys)
                        │ API token (per user)
                        ▼
              gpushed API  (Cloudflare Worker + D1)
              at api.gpushed.dev     · smartloop accounts · balances
              Stripe · metering cron (every 5 min)
                        │ queues the run in D1, starts an execution
                        │ (run.invoker only, no arguments)
                        ▼
              Cloud Run job: gpushed-provisioner
              provisioner image (Terraform + gcloud) from Artifact Registry;
              claims the queued run from the API, runs as the provisioner
              service account, R2 keys + API secret from Secret Manager
                        │ terraform (state in R2)
                        ▼
              GCE GPU VM in smartloop's GCP project
Path What it is
gpushed, install.sh Mac command and installer
client/ Client image (Containerfile) and the CLI that runs in it
api/ Cloudflare Worker: smartloop sign-in, billing, Stripe webhook, metering, dispatch
provisioner/ Provisioner image, entrypoint and the box's Terraform
infra/bootstrap/ One-time GCP setup: network, service accounts, the provisioner Cloud Run job, Workload Identity
.github/workflows/ release.yml, ci.yml (tests + API deploy), provision.yml (the old provisioner, until the switch to Cloud Run)

Why the keys aren't in the client image. The client runs on users' Macs, so anything inside it can be read by them. A cloud key there would let anyone create VMs in our project without paying. Instead, the client holds only a per-user API token; the API checks the balance before it asks the provisioner (which has the cloud access) to provision anything, and metering runs on the server.

Why the API can only start the provisioner. The API's GCP key (GCP_DISPATCH_KEY) may run the gpushed-provisioner job and nothing else, and not with arguments or environment overrides. What to do is queued in D1 (provision_jobs); the execution claims it from POST /internal/jobs/claim with the provisioner secret, taking the oldest run whose box has none going, so runs of one box happen in order. A leaked key starts executions that find nothing to do.

Accounts. gpushed users are smartloop accounts. gpushed login signs in in the browser: the CLI starts a sign-in (POST /auth/cli/start) and opens app.smartloop.ai/gpushed/cli-login?code=XXXX-XXXX; you sign in there if needed and approve the code shown in the terminal (POST /auth/cli/approve, with your smartloop session), and the CLI, polling POST /auth/cli/poll with its secret device code, gets a long-lived gpushed token. Sign-ins expire after 10 minutes. gpushed login --email instead asks smartloop's API (/v1/auth/otp/request and /verify) for an emailed code and trades the smartloop access token for a gpushed token (POST /auth/smartloop, checked with smartloop's /v1/users/me). The gpushed user id is the smartloop user id.

Web app. The smartloop web app can call the same API from the browser with the user's smartloop access token (Authorization: Bearer <token>), from the origins in APP_ORIGINS. Useful routes for a usage page: GET /me (balance and box), GET /usage?days=30 (spend and top-ups per day), GET /ledger?kind=topup (recent top-ups), GET /plans, and POST /topup (returns a Stripe checkout URL; pass return_url on the web app to come back to it with ?topup=done or ?topup=cancelled).

Billing. Money is stored in micro-dollars in D1. Every change is a ledger row with a unique ref, so Stripe webhooks and metering runs are idempotent. A box is billed from the moment Terraform reports it created, at the plan's price while running and the disk price while stopped.

Operating it

One-time setup

  1. Cloudflare R2: create two buckets:

    npx wrangler r2 bucket create gpushed-releases   # public, served at dl.gpushed.dev
    npx wrangler r2 bucket create gpushed-state      # private, Terraform state
    npx wrangler r2 bucket domain add gpushed-releases --domain dl.gpushed.dev --zone-id <zone>

    Create two R2 API tokens: one with write access to gpushed-releases (release workflow), one with read/write on gpushed-state (the provisioner and the bootstrap). Optional: a lifecycle rule to expire old releases' images after 30 days.

  2. API (Cloudflare Worker + D1):

    cd api && npm ci
    npx wrangler d1 create gpushed          # put the id in wrangler.toml
    npx wrangler d1 migrations apply gpushed --remote
    npx wrangler secret put STRIPE_SECRET_KEY
    npx wrangler secret put STRIPE_WEBHOOK_SECRET   # endpoint: https://api.gpushed.dev/stripe/webhook,
                                                    # events: checkout.session.completed, checkout.session.async_payment_succeeded
    npx wrangler secret put RESEND_API_KEY          # low-balance emails
    npx wrangler secret put PROVISIONER_SECRET      # random; same value as the GitHub secret below
    # GCP_DISPATCH_KEY comes after step 4
    npx wrangler deploy

    The Worker is served at api.gpushed.dev and, for CLIs installed before the move, gpushed-api.smartloop.ai: Worker custom domains (Cloudflare creates their DNS records and certificates on deploy), so the gpushed.dev and smartloop.ai zones must be in the same Cloudflare account. Downloads are likewise on both dl.gpushed.dev and dl.gpushed.smartloop.ai. Pricing knobs (PREMIUM_PCT, MIN_TOPUP_USD, LOW_BALANCE_USD, STOPPED_GRACE_HOURS, and REGIONS to limit regions) are in api/wrangler.toml. Regions are listed in api/src/regions.ts (and their subnets in infra/bootstrap); GCP list prices per plan and region are generated into api/src/regions.generated.ts by node api/scripts/gcp-prices.mjs <project>, and the zones offering each plan into api/src/zones.generated.ts by node api/scripts/gcp-zones.mjs <project> (rerun both when adding a region or plan).

  3. GitHub: create an environment named production, limited to main, with:

    Kind Name Value
    variable GCP_PROJECT, GCP_SERVICE_ACCOUNT, GCP_IMAGE_SERVICE_ACCOUNT, GCP_RELEASE_SERVICE_ACCOUNT, GCP_WORKLOAD_IDENTITY_PROVIDER, GCP_PROVISIONER_IMAGE, CLOUD_RUN_JOB set by step 4
    variable GPUSHED_API https://api.gpushed.dev
    variable GPUSHED_DL https://dl.gpushed.dev
    variable R2_ENDPOINT https://<account-id>.r2.cloudflarestorage.com
    variable R2_STATE_BUCKET / R2_RELEASE_BUCKET gpushed-state / gpushed-releases
    variable CLOUDFLARE_ACCOUNT_ID your account ID
    secret PROVISIONER_SECRET same as the Worker's (step 4 copies it into Secret Manager)
    secret R2_STATE_ACCESS_KEY_ID, R2_STATE_SECRET_ACCESS_KEY state bucket token
    secret R2_RELEASE_ACCESS_KEY_ID, R2_RELEASE_SECRET_ACCESS_KEY release bucket token
    secret CLOUDFLARE_API_TOKEN Workers + D1 edit, for API deploys
    secret GCP_BOOTSTRAP_KEY for step 4 only; delete afterwards
    secret GH_ADMIN_TOKEN optional, for step 4: fine-grained PAT that can write this repo's environment variables

    Limit the environment to the main branch and v* tags (the release workflow runs in it on tags).

    Boxes are provisioned on Cloud Run, so their logs are in Cloud Logging, not in this public repo's workflow logs.

  4. GCP (in our project; needs GPU quota in each region): run Actions > Bootstrap GCP with plan, check it, then run it with apply. It runs infra/bootstrap with Terraform state in gpushed-state/bootstrap/terraform.tfstate, and stores its outputs as production variables (with GH_ADMIN_TOKEN; otherwise copy them from the run summary). It also creates the gpushed-provisioner Cloud Run job (in us-central1), its Artifact Registry repo, and Secret Manager copies of PROVISIONER_SECRET and the R2 state keys; re-run it after changing any of those.

  5. Provisioner on Cloud Run:

    # The job runs a placeholder image until a release pushes the real one:
    git tag v0.x.y && git push origin v0.x.y      # see Releasing
    # The API's key, which may only start the job:
    gcloud iam service-accounts keys create key.json \
      --iam-account gpushed-dispatcher@<project>.iam.gserviceaccount.com
    (cd api && npx wrangler secret put GCP_DISPATCH_KEY < ../key.json) && rm key.json

    Then in api/wrangler.toml set CLOUD_RUN_JOB to the bootstrap's CLOUD_RUN_JOB output and PROVISIONER = "cloudrun", and deploy. To check it by hand, run gcloud run jobs execute gpushed-provisioner --region us-central1: with nothing queued it logs "nothing to do" and exits. Until the switch, PROVISIONER = "github" keeps using provision.yml (with the Worker secret GITHUB_TOKEN); set it back to roll back. If your organization blocks service account keys, the API can't start the job this way; keep PROVISIONER = "github".

    GCP_BOOTSTRAP_KEY is a JSON key for a service account with roles/owner on the project:

    gcloud iam service-accounts create gpushed-bootstrap --project <project>
    gcloud projects add-iam-policy-binding <project> \
      --member serviceAccount:gpushed-bootstrap@<project>.iam.gserviceaccount.com --role roles/owner
    gcloud iam service-accounts keys create key.json \
      --iam-account gpushed-bootstrap@<project>.iam.gserviceaccount.com
    gh secret set GCP_BOOTSTRAP_KEY --env production < key.json && rm key.json

    Afterwards delete the key (or the whole account); boxes are provisioned through Workload Identity and never use it. Re-run the workflow (with a new key) only to change the bootstrap, e.g. after renaming the repo. If your organization blocks service account keys (iam.disableServiceAccountKeyCreation), run infra/bootstrap by hand instead; the commands are at the top of its main.tf.

Releasing

git tag v0.2.0 && git push origin v0.2.0

The release workflow builds the CLI package, the client image (linux/arm64, OCI archive for container image load) and the provisioner image (linux/amd64), uploads them to R2 under v0.2.0/, then moves latest. It also pushes the provisioner image to Artifact Registry and points the gpushed-provisioner Cloud Run job at it, so new box runs use it at once.

Local development

cd api && npm ci
npx wrangler d1 migrations apply gpushed --local --env dev
npx wrangler dev --env dev --test-scheduled --ip 0.0.0.0   # emails and dispatches are logged, not sent
# Sign-in goes to SMARTLOOP_API in wrangler.toml [env.dev] (a local smartloop
# API on :8788, or set it to https://api.smartloop.ai to use real accounts).
npm test

# In another terminal, from a checkout (builds the client image locally). Use
# the Mac's address as seen from containers (the vmnet gateway, usually 192.168.64.1):
GPUSHED_API=http://192.168.64.1:8787 GPUSHED_SMARTLOOP_API=http://192.168.64.1:8788 \
  ./gpushed login --email you@example.com

curl localhost:8787/__scheduled runs the metering cron once.

License

Apache-2.0

About

gpushed: your own GPU shed. Spin up an NVIDIA GPU box on Google Cloud with one command

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages