Repository navigation
Drop the internal daily cap on YouTube analytics jobs - #402
Merged
Merged
Conversation
AnalyticsRateLimits capped YouTube analytics at 3,000 jobs a day on one project-wide key, assuming each job spent up to 2 Data API units and shared the 10,000-unit quota with uploads. Neither holds: a metrics job reads videos.list in batches of 50 ids, and videos.insert now has its own 100-call bucket. The console shows 16% of the Data API and 1.6% of the Analytics API used. The cap counted every job that passed the middleware, including discovery and duplicates, so the launch backfill of large channels exhausted it. Every YouTube metrics and discovery job was then released until the 24-hour window reset, and the ones past retryUntil failed together with MaxAttemptsExceededException each day at the same minute. A quota or 429 from Google is still retried as rate_limited; more quota is requested from Google when that happens.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Nightwatch #18 (
MaxAttemptsExceededException), 528 times in 24 hours, forCollectPublicationMetrics. The job fails at pickup in under a millisecond, whichWorker::markJobAsFailedIfAlreadyExceedsMaxAttemptsonly does onceretryUntil(the end of the day after the observation date) has passed. The jobs were not slow: they were never let through in time.AnalyticsRateLimitsgave YouTube a project-wideLimit::perDay(3000)(youtube:project), shared by metrics and discovery jobs. Once it runs out,RateLimitedreleases every YouTube job until the 24-hour window resets. The window is fixed and anchored at its first hit, so every blocked job comes back at the same instant, and the ones pastretryUntilfail together:The cap was far below what it was protecting. It assumed each job spent up to 2 Data API units and shared the 10,000-unit quota with uploads. In reality:
videos.listin batches of 50 ids, one unit for 50 videos (videos.list), plus one YouTube Analytics API report.videos.inserthas its own bucket of 100 calls a day (quota costs, getting started).Fix
Remove the YouTube shared daily cap. The per-channel limit (30 jobs a minute) stays. A
quotaExceeded,rateLimitExceededor 429 from Google is already classified asrate_limitedbyAnalyticsCollectionException::fromResponseand retried; if that happens, the project asks Google for more quota.Tests
PublicationMetricsJobsTest: YouTube is limited only per channel (fails with the old cap).RemoteMediaImporterTest"a transfer that runs…", failed once in the parallel run; it compares temp file names, and it passed 4 of 4 times on its own, both with and without this change.