PoC of a new in memory model for deployment artifact generations by aranega · Pull Request #874 · MetaCell/cloud-harness

aranega · 2026-09-28T10:04:22Z

Closes CH-X

Implemented solution

...

How to test this PR

...

Sanity checks:

The pull request is explicitly linked to the relevant issue(s)
The issue is well described: clearly states the problem and the general proposed solution(s)
In this PR it is explicitly stated how to test the current change
The issue seta the scope and the type of issue (bug, story, task, etc.)
The relevant components are indicated in the issue (if any)
All the automated test checks are passing
All the linked issues are included in one Sprint
All the linked issues are in the Review state
All the linked issues are assigned

Breaking changes (select one):

The present changes do not change the preexisting api in any way
This PR and the issue are tagged as a breaking-change and the migration procedure is well described above

Possible deployment updates issues (select one):

There is no reason why deployments based on CloudHarness may break after the current update
This PR and the issue are tagged as alert:deployment

Test coverage (select one):

Tests for the relevant cases are included in this pr
The changes included in this pr are out of the current test coverage scope

Documentation (select one):

The documentation has been updated to match the current changes
The changes included in this PR are out of the current documentation scope

Nice to have (if relevant):

Screenshots of the changes
Explanatory video/animated gif

…d generate

…affold

… and dockerfiles

…later)

…o the result

+# def test_collect_helm_values_wrong_dependencies_validate(tmp_path):
+#     out_folder = tmp_path / 'test_collect_helm_values_wrong_dependencies_validate'
+#     with pytest.raises(ValuesValidationException):
+#         create_helm_chart([CLOUDHARNESS_ROOT, f"{RESOURCES}/wrong-dependencies"], output_path=out_folder, domain="my.local",
+#                           namespace='test', env='prod', local=False, tag='1', include=["wrong-hard"])
+#     try:
+#         create_helm_chart([CLOUDHARNESS_ROOT, f"{RESOURCES}/wrong-dependencies"], output_path=out_folder, domain="my.local",
+#                           namespace='test', env='prod', local=False, tag='1', include=["wrong-soft"])
+#
+#     except ValuesValidationException as e:
+#         pytest.fail("Should not error because of wrong soft dependency")
+#
+#     with pytest.raises(ValuesValidationException):
+#         create_helm_chart([CLOUDHARNESS_ROOT, f"{RESOURCES}/wrong-dependencies"], output_path=out_folder, domain="my.local",
+#                           namespace='test', env='prod', local=False, tag='1', include=["wrong-build"])
+#     try:
+#         create_helm_chart([CLOUDHARNESS_ROOT, f"{RESOURCES}/wrong-dependencies"], output_path=out_folder, domain="my.local",
+#                           namespace='test', env='prod', local=False, tag='1', include=["wrong-services"])
+#     except ValuesValidationException:
+#         pytest.fail("Should not error because of missing use_services dependency")


+# def test_collect_helm_values_build_dependencies(tmp_path):
+#     out_folder = tmp_path / 'test_collect_helm_values_build_dependencies'
+#     values = create_helm_chart([CLOUDHARNESS_ROOT, RESOURCES], output_path=out_folder, domain="my.local",
+#                                namespace='test', env='prod', local=False, tag='1', include=["myapp"])
+#
+#     assert 'cloudharness-flask' in values[KEY_TASK_IMAGES], "Cloudharness-flask is included in the build dependencies"
+#     assert 'cloudharness-base' in values[KEY_TASK_IMAGES], "Cloudharness-base is included in cloudharness-flask Dockerfile and it should be guessed"
+#     assert 'cloudharness-base-debian' not in values[KEY_TASK_IMAGES], "Cloudharness-base-debian is not included in any dependency"
+#     assert 'cloudharness-frontend-build' not in values[KEY_TASK_IMAGES], "cloudharness-frontend-build is not included in any dependency"
+#
+#
+# def test_collect_helm_values_build_dependencies_nodeps(tmp_path):
+#     out_folder = tmp_path / 'test_collect_helm_values_build_dependencies_nodeps'
+#     values = create_helm_chart([CLOUDHARNESS_ROOT, RESOURCES], output_path=out_folder, domain="my.local",
+#                                namespace='test', env='prod', local=False, tag='1', include=["events"])
+#
+#     assert 'cloudharness-flask' not in values[KEY_TASK_IMAGES], "Cloudharness-flask is not included in the build dependencies"
+#     assert 'cloudharness-base' not in values[KEY_TASK_IMAGES], "Cloudharness-base is not included in the build dependencies"
+#     assert 'cloudharness-base-debian' not in values[KEY_TASK_IMAGES], "Cloudharness-base-debian is not included in any dependency"
+#     assert 'cloudharness-frontend-build' not in values[KEY_TASK_IMAGES], "cloudharness-frontend-build is not included in any dependency"
+#
+#
+# def test_collect_helm_values_build_dependencies_exclude(tmp_path):
+#     out_folder = tmp_path / 'test_collect_helm_values_build_dependencies_exclude'
+#     values = create_helm_chart([CLOUDHARNESS_ROOT, RESOURCES], output_path=out_folder, domain="my.local",
+#                                namespace='test', env='prod', local=False, tag='1', include=["workflows"], exclude=["workflows-extract-download"])
+#
+#     assert 'cloudharness-flask' in values[KEY_TASK_IMAGES], "Cloudharness-flask is included in the build dependencies"
+#     assert 'cloudharness-base' in values[KEY_TASK_IMAGES], "Cloudharness-base is included in cloudharness-flask Dockerfile and it should be guessed"
+#     assert 'workflows-extract-download' not in values[KEY_TASK_IMAGES], "workflows-extract-download has been explicitly excluded"


+# def test_cnpg_postgres_parameters_render_only_when_set(tmp_path):
+#     out_folder = tmp_path / 'test_cnpg_postgres_parameters_render_only_when_set'
+#     create_helm_chart([CLOUDHARNESS_ROOT, RESOURCES], output_path=out_folder, domain="my.local",
+#                       env='withpostgres', local=False, include=["myapp"], exclude=["legacy"])
+#
+#     helm_path = out_folder / HELM_CHART_PATH
+#     shutil.rmtree(helm_path / 'charts')
+#     values_path = helm_path / 'values.yaml'
+#     with open(values_path, 'r') as values_file:
+#         values = yaml.load(values_file)
+#     postgres = values['apps']['myapp']['harness']['database']['postgres']
+#     postgres['operator'] = True
+#     postgres['parameters'] = {
+#         # Simulate generated YAML values where on/off can be parsed as booleans before Helm renders the chart.
+#         'autovacuum': True,
+#         'max_connections': '200',
+#         'shared_buffers': '1GB',
+#         'synchronous_commit': True,
+#         'track_io_timing': False,


+#     with open(values_path, 'w') as values_file:
+#         yaml.dump(values, values_file)


+#     with open(values_path, 'w') as values_file:
+#         yaml.dump(values, values_file)


+#     with open(values_path, 'w') as values_file:
+#         yaml.dump(values, values_file)


+# def test_statefulset_option(tmp_path):
+#     out_folder = tmp_path / 'test_statefulset_option'
+#     # nfsserver is included to provide the storage class values needed by the usenfs case
+#     create_helm_chart([CLOUDHARNESS_ROOT, RESOURCES], output_path=out_folder, domain="my.local",
+#                       env='withpostgres', local=False, include=["myapp", "nfsserver"], exclude=["legacy"])
+#
+#     helm_path = out_folder / HELM_CHART_PATH
+#     shutil.rmtree(helm_path / 'charts')
+#     values_path = helm_path / 'values.yaml'
+#     with open(values_path, 'r') as values_file:
+#         values = yaml.load(values_file)
+#
+#     myapp = values['apps']['myapp']
+#     harness = myapp['harness']
+#     dep_name = harness['deployment']['name']
+#     db_name = harness['database']['name']
+#     service_name = harness['service']['name']
+#
+#     harness['deployment']['auto'] = True
+#     harness['deployment']['volume'] = {
+#         'name': 'myapp-data', 'mountpath': '/data', 'size': '1Gi', 'auto': True,


+#     with open(values_path, 'w') as values_file:
+#         yaml.dump(values, values_file)
+#
+#     # Default: Deployments with the Recreate/affinity workaround and a standalone PVC


+#     with open(values_path, 'w') as values_file:
+#         yaml.dump(values, values_file)


+        print(f"unknown: {unknown}")
+        sys.exit(2)
+
+    project = CHProject(root_paths[0], cloudharness_path=root_paths[0])


aranega · 2026-10-02T08:06:21Z

Closing the PR, I'll reopen a new one later, github code bot is stressing me out spamming so many messages at the moment while I'm still actively thinking/dev about the whole model/architecture

aranega added 13 commits September 23, 2026 08:31

Add first iteration of a base PoC for CH model

a45bfbe

Add missing entries in the deploy config and wire them in the skaffol…

45ed8f7

…d generate

Refactor the access to the template files

18aa16c

Add first iteration of the new codefresh generation

042dd4c

Add new fields in the deployment config and wire them in codefresh/sk…

17ee3e5

…affold

Expose "exists" as method on CHDockerfile

a94ea8c

Add new CHAppDefault concept to catch up with merging logic for tasks…

ee5c372

… and dockerfiles

Add first tests for the new model

f2f513a

Add CHBaseImage concept

16ca77b

Introduce CHContext for the build artifact skaffold generation

fd4e044

Fix missing tasks dependencies

a22354c

Add new tests covering skaffold generation

db18b74

Fix tasks resolution

33bfecd

github-code-quality Bot found potential problems Sep 28, 2026

View reviewed changes

aranega added 10 commits September 29, 2026 03:47

Add Dockerfile env dependent

c7e34d8

Add all_images and build_options for dockerfiles

82778dc

Add multi-env support from config

8ea87da

Add entrypoint parsing and primitive docker parser (might be removed …

520a1df

…later)

Add git-dependencies support

2ee83c8

Add missing build dependencies walk for skaffold generation

3758b78

Add skaffold documentation at the top of the module

5857925

Add values-template merge per app

7a16df1

Split tests for the NG version

bfd7b45

Add higher-level all_values that merges options from the cmd line int…

ffd6237

…o the result

github-code-quality Bot found potential problems Sep 30, 2026

View reviewed changes

aranega added 5 commits October 1, 2026 02:52

Add multi-project support

8aac829

Fix merge order for the raw_values and all_values at the project level

811998e

Add compat tests

bae71d6

Add output_path support for plugins/skaffold/codefresh

7b6f05f

Fix all_source_images to consider only involved apps

224db53

github-code-quality Bot found potential problems Oct 1, 2026

View reviewed changes

Comment thread tools/deployment-cli-tools/harness-deployment-ng

print(f"unknown: {unknown}")

sys.exit(2)

project = CHProject(root_paths[0], cloudharness_path=root_paths[0])

Add context merge

22cc0cc

aranega closed this Oct 2, 2026

Navigation Menu

PoC of a new in memory model for deployment artifact generations - #874

PoC of a new in memory model for deployment artifact generations#874
aranega wants to merge 29 commits into
developfrom
poc/ng-library-clean

aranega commented Sep 28, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aranega commented Oct 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

		# with open(values_path, 'w') as values_file:
		# yaml.dump(values, values_file)

Conversation

aranega commented Sep 28, 2026

Implemented solution

How to test this PR

Sanity checks:

Breaking changes (select one):

Possible deployment updates issues (select one):

Test coverage (select one):

Documentation (select one):

Nice to have (if relevant):

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aranega commented Oct 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant