-
Notifications
You must be signed in to change notification settings - Fork 126
Circuit breaker changes using pybreaker #705
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
Signed-off-by: Nikhil Suri <[email protected]>
c83cba7 to
e7e8b4b
Compare
Signed-off-by: Nikhil Suri <[email protected]>
What type of PR is this?
Description
This PR introduces a circuit breaker pattern to the telemetry system to prevent cascading failures and improve system resilience. The implementation uses the pybreaker library to monitor telemetry request failures and automatically open the circuit when failure rates exceed configurable thresholds, blocking further requests to protect downstream services. When the circuit is open, telemetry requests are temporarily blocked until the system recovers, at which point the circuit transitions to half-open for testing and eventually closes when normal operation resumes. The circuit breaker configuration is centralized with immutable settings, includes comprehensive logging for monitoring, and maintains full backward compatibility with existing telemetry functionality.
How is this tested?
Related Tickets & Documents
https://docs.google.com/document/d/1ftRvby9bwDZzE3s1tOb4hJ4Pd9USiXskb9cDw-uQNPM/edit?usp=sharing