Repository navigation
Add Emergency Rescue investigation page - #547
Open
yasingunaydiin wants to merge 1 commit into
Open
yasingunaydiin wants to merge 1 commit into
yasingunaydiin wants to merge 1 commit into
Conversation
Add a troubleshooting page that explains what an Emergency Rescue is, why it shows up in the Control Panel logs and livelog, and how to use the incident files and PHP-FPM log to find out what caused it. Link to it from the 24/7 Monitoring and Alerting page.
yasingunaydiin
force-pushed
the
emergency-rescue
branch
from
November 7, 2026 22:18
6e46f06 to
78691df
Compare
poespas
reviewed
Nov 9, 2026
| html_meta: | ||
| description: Learn what an Emergency Rescue on Hypernode is, why it shows up in | ||
| your logs, and how to use the incident files to find out what caused it. | ||
| title: Emergency Rescue on Hypernode and How to Investigate It | Hypernode |
Collaborator
There was a problem hiding this comment.
Suggested change
| title: Emergency Rescue on Hypernode and How to Investigate It | Hypernode | |
| title: Emergency Rescue on Hypernode and How to Investigate It |
poespas
reviewed
Nov 9, 2026
|
|
||
| ## The Livelog Line | ||
|
|
||
| In `livelog` or the Control Panel logs you may see a line like `restore from backup <appname>`. This is how an Emergency Rescue shows up. It is an action by the platform. Nothing in the Control Panel or in your shop started it, and it does not mean an attacker or a team member did something. |
Collaborator
There was a problem hiding this comment.
why not just a emergency_rescue line in livelog?
poespas
reviewed
Nov 9, 2026
|
|
||
| ## Find Out What Caused It | ||
|
|
||
| For every rescue, Hypernode saves a snapshot of the state of your Hypernode in a folder named after the time of the rescue: |
Collaborator
There was a problem hiding this comment.
This is not true, it's only caused by OOM events.
poespas
reviewed
Nov 9, 2026
| - `process-list`: the running processes. | ||
| - `tcp-connections`: the open network connections. | ||
|
|
||
| ### Most Common Cause: PHP-FPM Is Full |
Collaborator
There was a problem hiding this comment.
Suggested change
| ### Most Common Cause: PHP-FPM Is Full | |
| ### Most Common Cause: PHP-FPM workers are full |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
restore from backup <appname>livelog line./data/web/incidentsand the PHP-FPM log to find the cause, and how to deal with unwanted traffic.