Repository navigation
ci: notify search engines once a day, from a sitemap snapshot - #103
Merged
Merged
Conversation
On every push, three merges touching one page sent that URL to IndexNow three times and resubmitted the sitemap three times. IndexNow asks sites not to resend a URL many times a day; Google says repeat requests do not get anything crawled faster. - Daily schedule (and by hand) instead of push. The run compares the live sitemap with the last run's snapshot in the Actions cache and sends only new URLs and URLs whose lastmod moved. - No git diff and no wait for the deploy: what the sitemap says is live. - The snapshot only advances after IndexNow accepted the URLs, so a failed or skipped send is retried the next day.
Deploying jlog-web with
|
| Latest commit: |
ad0aca5
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://384c9560.jlog-web.pages.dev |
| Branch Preview URL: | https://ci-notify-search-engines-dai.jlog-web.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Notify ran on every push that touched a page. Three merges to one page in a day sent that URL to IndexNow three times and resubmitted the sitemap to Google three times. IndexNow asks sites to avoid "submitting the same URL many times a day", and Google says "requesting a recrawl multiple times for the same URL won't get it crawled any faster".
Now
lastmodmoved. Google gets one sitemap resubmit, only when something changed.search-engines, main only), same secrets, Google and IndexNow functions unchanged. Cache actions pinned by commit like the others.Honest
lastmod(from #96) matters more now: the comparison is only as good as the dates.Checked: new tests for reading the sitemap, and for finding new or changed URLs (5 passing; web suite 61). A run against the live sitemap without keys reads all 32 URLs, skips both sends with notices, and writes no snapshot. Biome and typecheck clean.
After merging: the first daily run has no snapshot and sends all 32 URLs once. bhargav.dev uses the same pattern (brgv-code/bhargav.dev#73).