Repository navigation
Support workflow aliases via folder names - #1579
midigofrank wants to merge 8 commits into
Conversation
* lexicon: remove trigger.enabled from spec * update handling of trigger.enabled * update version hash * changeset * remove log * update version util and fix tests * add notes to docs * remove .only * fix test * update test * fix integration test * first swing at refactor to split up openfn.yaml * order keys * fix ropey merge issues * add logic to checkout on git without tracking * Update tests for checkout file and flat openfn.yaml Project metadata now lives in .openfn/checkout.yaml, openfn.yaml holds workspace config (plus collections) as flat, sorted keys. Integration tests pin OPENFN_BRANCH=false so they don't pick up the kit repo's branch. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * format * move collections to resources.yaml * Add prompt to reworked openfn yaml (#1577) * add util for checking if there's a valid checkout * add helpers for writing checkout metadata * refactoring * use new helpers in deploy * prompt user for checkout * add user prompt * simplify * fix tests * better fix * ignore checkout meta (#1578) * changeset * changesets and docs --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
* project: tell local copies of the same project apart by alias A workspace can hold several local copies of one project, each from its own state file (main@app.openfn.org.yaml, backup@app.openfn.org.yaml). They share a UUID and id, so anything keyed on those mixed them up. Copies overwrote each other's entries in the path map, the checkout had no way to say which copy was checked out, and get() threw even when given an exact alias. matchProject now returns the exact alias match when all the other matches share its UUID. Real ambiguity still throws, and the error lists each clash as alias@host and suggests using one. Workspace keeps project paths by Project rather than by id, so each copy keeps its own file. getProjectPath() now takes a Project. extractCheckout writes the active project's alias to the checkout file, and getTrackedProject() looks that alias and host up first. The alias is left out of project.openfn so the metadata doesn't store it twice. merge used getProjectPath(id) to find where to write. With --base it only got the right file because the id-keyed map happened to point there. It now writes back to the --base file directly. * cli: don't fail fetch on duplicate local projects when the target is explicit `openfn project fetch <uuid> --endpoint X -o file.yaml` threw MultipleMatchingProjectsError if two local project files shared the UUID. With a full UUID, an endpoint and an output path, the command has everything it needs, so the local copies don't matter. Skip the local workspace lookup in that case. Also stop searching the workspace for an output target once -o is given, because the path is the target. Duplicates still throw when the UUID, endpoint or output path is missing. * cli: use the checked-out copy for pull and deploy without a target When no project was given, pull and deploy fell back to the active project's UUID. That throws if another local copy shares the UUID, even though the checkout file says which one is checked out. If the checkout file records an alias, pull now defaults to the tracked project's alias@host and deploy uses the tracked project. Without an alias, both fall back to the UUID as before. * add changeset * address review feedback getTrackedProject now tries alias, uuid then id in plain steps, and matches the alias on the full endpoint origin, so copies on different ports of the same host (localhost:4000, localhost:5000) are told apart. deploy picks its sync target in one place: an explicit target, the local copy of a deployed file, or the tracked project. The ambiguity error now suggests alias@domain or a path. Adds a test for fetching by alias when local copies share a UUID. * little tidy --------- Co-authored-by: Joe Clark <jclark@openfn.org>
5f8548f to
a7f2237
Compare
|
Hey @josephjclark, I've picked something up while doing a review: An alias is the workflow's folder name, and
The fix claude is suggesting is to keep |
|
Well if there are multiple matches for a workflow we need to throw an error. Just like we do with Workspace.getProject. I'm also up for a fuzzy |
A workflow's folder name in the workspace now acts as a local alias when it differs from the workflow id. Aliases can be used to look up workflows, are preserved across checkout and pull, and are never deployed. Aliases that clash with another workflow's id or alias are dropped with a warning.
Aliases are folder names, so they only exist on the checked-out project. List them next to the workflow ids of the active project so users can see which short names are available.
getWorkflow matched on id, alias, name and uuid, so id-based callers (diff, merge, checkout) could resolve an alias to the wrong workflow. It now matches ids only. findWorkflow handles user input and throws when more than one workflow matches.
a7f2237 to
ab36024
Compare
|
Thanks @josephjclark , I've done the refactor in ab36024:
|
|
Workflow aliases work great too with |
|
Confirming that if you rename a workflow in the app and pull, the alias will be lost and the folder deleted (including any temporary folders the user may have in it). This problem exists anyway: we should recognose the rename and rename the folder, rather than removing and rebuilding. When we get around to addressing it, we need to ensure that aliases survive the rename (We can always prompt) See #1436 |
|
grr why can't I reopen this |
* Support workflow aliases via folder names A workflow's folder name in the workspace now acts as a local alias when it differs from the workflow id. Aliases can be used to look up workflows, are preserved across checkout and pull, and are never deployed. Aliases that clash with another workflow's id or alias are dropped with a warning. * Show workflow aliases in project list Aliases are folder names, so they only exist on the checked-out project. List them next to the workflow ids of the active project so users can see which short names are available. * Split workflow lookup into getWorkflow and findWorkflow getWorkflow matched on id, alias, name and uuid, so id-based callers (diff, merge, checkout) could resolve an alias to the wrong workflow. It now matches ids only. findWorkflow handles user input and throws when more than one workflow matches. --------- Co-authored-by: Frank Midigo <midigofrank@gmail.com>
|
This is merged (via a different PR) |
* [rebased] Support workflow aliases via folder names- #1579 (#1585) * Support workflow aliases via folder names A workflow's folder name in the workspace now acts as a local alias when it differs from the workflow id. Aliases can be used to look up workflows, are preserved across checkout and pull, and are never deployed. Aliases that clash with another workflow's id or alias are dropped with a warning. * Show workflow aliases in project list Aliases are folder names, so they only exist on the checked-out project. List them next to the workflow ids of the active project so users can see which short names are available. * Split workflow lookup into getWorkflow and findWorkflow getWorkflow matched on id, alias, name and uuid, so id-based callers (diff, merge, checkout) could resolve an alias to the wrong workflow. It now matches ids only. findWorkflow handles user input and throws when more than one workflow matches. --------- Co-authored-by: Frank Midigo <midigofrank@gmail.com> * Deploy without fetch (#1581) * auto-fetch state file * changeset * Better execute output (#1584) * better default output * changeset * fix tests * versions --------- Co-authored-by: Frank Midigo <midigofrank@gmail.com>
Short Description
Workflows can now be given a short local alias by renaming their folder in the workspace, so
workflows/my-really-long-workflow-name→workflows/wflets you runopenfn wf.Fixes #1292
Implementation Details
The alias is just the folder name, as suggested in the issue, so there's no extra metadata to track.
@openfn/projectfrom-fs: if a workflow's folder name differs from its id, it's set asWorkflow.alias. The alias lives on theWorkflowinstance only, so it never ends up in the workflow yaml, the state file, or a deploy.to-fs: aliased workflows are written toworkflows/<alias>/instead ofworkflows/<id>/.Project.getWorkflowalso matches on alias, soopenfn <alias>,project version <alias>etc. work.Workspace.getCheckedOutProjectnow passes its logger to the fs parser so that warning is visible. Side effect: existing parser messages (e.g. "Error loading expression from …") that were previously silent now show up.@openfn/clicheckout(and thereforepullandclean) copies aliases from the checked-out project to the incoming one, matched by workflow id, so renamed folders aren't deleted. An alias that clashes with an incoming workflow's id is dropped with a warning.Known limitation: aliases are matched by workflow id. Renaming a workflow in the app changes its id, so the next
pullwrites it back to a folder named after the new id and the alias has to be set again. Matching by UUID would fix this, butpulloverwrites the state file before checkout runs, so the old id → UUID mapping is gone by then. Storing a UUID → alias map inopenfn.yamlwould also work, but it needs anopenfn project aliascommand to record it reliably. Both felt like more than this PR needed and can follow later if renames turn out to be common.QA Notes
Tested end to end against a local Lightning:
mv workflows/<id> workflows/wfopenfn wfruns the workflow;openfn project version wfresolves itopenfn project pullkeepsworkflows/wfand doesn't recreateworkflows/<id>workflows/wf, thenopenfn project deploy. The diff only contains the edit and no alias reaches Lightningnameand deploy), then pull. The workflow moves to a folder named after the new id (the known limitation above)AI Usage
Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):
You can read more details in our
Responsible AI Policy