Automatic Restart of Plugins on Crash

Tags:

Starting from version 6.2, the ability to automatically restart plugins upon crashing has been added. Previously, after a plugin crash, the SyrveFront application continued to operate without it, and to restore lost functionality, the entire SyrveFront application had to be restarted along with all other plugins. Now, a crashed plugin can be restarted on the fly without stopping the SyrveFront application.

Since the restart on crash is a feature of the plugin hosting mechanism, this innovation is available for all supported API versions. For already released V4 and V5 versions, this feature works with default parameters, while starting from V6, plugins can specify their parameters through a special attribute RecoveryOptions. Currently, only the action to be taken on crash (OnCrashAction) can be specified:

Restart — restart the plugin on crash,
DoNotRestart — do not restart the plugin on crash,
Default — default behavior.

Considering that the changes will affect all plugins, including those running under V4 and V5, as well as V6 plugins released before the introduction of the RecoveryOptions attribute, the default behavior must be cautious and conservative. The current default behavior is to restart the plugin if it has crashed after running for at least 10 seconds. The minimum runtime limitation is necessary to avoid restarting plugins that perform a series of checks on startup and intentionally crash when conditions for operation are not met. To properly shut down a plugin, the method PluginContext.Shutdown() should be called, but in practice, many plugins are incorrectly implemented, and this must be taken into account. Other side effects of the innovation may also emerge, so the default behavior may change in the future.

A plugin is considered crashed if its host process exits with a non-zero return code. There may be some time between the occurrence of an unhandled exception that caused the crash and the actual termination of the process. For example, the default debugger WerFault (formerly Dr. Watson) may pause threads and hold the host process while collecting diagnostic information, preventing it from terminating.

Although automatic plugin restart on crash may be convenient for the user, it does not relieve the plugin developer of the responsibility to fix bugs. The goal should be to ensure that the plugin does not crash and that auto-restart is not required. For hopelessly broken plugins, a limit is set — no more than four crashes per hour. If a plugin crashes for the fifth time within an hour, it will be deemed hopeless and will not be restarted again in that session of the SyrveFront application.