Repository navigation
feat(aisix)!: drop default resources, document workers sizing the token-estimate pool, refuse utilization autoscaling without requests - #429
Draft
jarvis9443 wants to merge 5 commits into
Conversation
…hout the matching request
The gateway sizes its token-estimate pool by config.proxy.workers, one thread
per worker each holding its own tokenizer (about 17/27/39 MiB resident for
cl100k/o200k/o200k_harmony). With workers unset it follows every CPU the pod
sees, so the chart's fixed 1Gi memory limit could OOM a pod on a large node.
No single CPU or memory figure fits every deployment, so resources now
defaults to {} and the operator sets requests and config.proxy.workers.
An HPA with a CPU or memory utilization target needs the matching request;
the render now fails instead of deploying an HPA that never scales.
… matching request KEDA measures a cpu or memory trigger with metricType Utilization (its default when metricType is omitted) against the container's request, exactly as the chart's own HPA does, so without one the HPA KEDA creates never scales. Also print the result of the render-succeeds checks, which their >/dev/null redirect was silencing.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This drops the gateway chart's default
resourcesand makes a utilization HPA or KEDA cpu/memory trigger refuse to render without the request it measures against. Breaking for anyone who relied on the old defaults.From api7/aisix#1437 the gateway sizes its token-estimate pool by
proxy.workers, one thread per worker, and each thread holds one tokenizer instance per encoding it has counted with (about 17 MiB cl100k, 27 MiB o200k, 39 MiB o200k_harmony resident). Withconfig.proxy.workersunset the gateway uses every CPU the process sees, which without a CPU limit is the whole node, so the old fixedlimits.memory: 1Gicould OOM a pod on a large node. No single CPU or memory figure fits every deployment, so the chart stops guessing:resourcesdefaults to{}(previouslyrequests: {cpu: 500m, memory: 256Mi},limits: {memory: 1Gi}). The pod renders with noresourcesblock unless you set one. The values comment explains why and what to size.config.proxy.workersstays declared with no default (null); its comment now says it sizes both the proxy workers and the token-estimate pool, and to set it to the CPUs allocated to the pod.autoscaling.enabled, atargetCPUUtilizationPercentagewithoutresources.requests.cpu, or atargetMemoryUtilizationPercentagewithoutresources.requests.memory, now fails the render (... requires resources.requests.cpu: utilization is measured against the request, so without one the HorizontalPodAutoscaler never scales ...). Before, such an HPA deployed and silently never scaled. SincetargetCPUUtilizationPercentagedefaults to 70,autoscaling.enabled=truealone now needsresources.requests.cpu.keda.enabled, acpuormemorytrigger whosemetricTypeisUtilization(KEDA's default whenmetricTypeis omitted; the deprecatedmetadata.typeis honoured too) fails the render unless the matchingresources.requestsentry is set.metricType: AverageValueand non-resource triggers such asprometheusare unaffected.workers, andaisix-render-checks.shcovers the empty default, the HPA and KEDA refusals, and the cases that still render. The render-succeeds checks in that script now print their result; a>/dev/nullon thecheckline was swallowing it.Upgrading: a release that took the chart's resource defaults gets none after
helm upgrade(the pod becomes BestEffort) and, if it autoscales on a CPU or memory utilization target (HPA or KEDA), fails to render until the matching requests are set. Setresources(at leastrequests) andconfig.proxy.workersyourself before upgrading.--reuse-valueskeeps whatever the old release had.Held for the release that ships api7/aisix#1437: the
workerscomment describes behaviour older gateways do not have.charts/aisix-cpis untouched.🤖 Generated with Claude Code