Fire drills help align your incident response team and prepare them for real incidents. We recommend running a fire drill to gain familiarity with the FireHydrant platform. We also recommend running fire drills often (once per month or once per quarter) to refresh your team's skills .
Here are some best practices for a successful fire drill:
- Create a low-stress environment for the drill.
- Focus on the incident management process and workflows.
- Encourage participation!
Six tips for running a fire drill with FireHydrant
1. Put a plan in place
For this first fire drill, it's probably best to have a plan in place and to put it on the calendar. This fire drill is really an opportunity for your team to get more familiar with the FireHydrant platform. For future fire drills, you may consider a more unannounced approach.
Make sure your FireHydrant runbook(s) are good to go. Review them to ensure all of the steps you want are in place but don't worry about perfection. Chances are you'll learn things from your first fire drill and make some changes to your runbooks.
2. Create a scenario
We recommend taking a recent incident your team worked through and recreating that scenario for your fire drill. Set the stage briefly so you have a situation to base your fire drill on.
3. Declare an incident
Now that the stage is set, declare an incident in FireHydrant. There are a number of ways you can do this depending on how you've configured things. The simplest way would be to enter
/firehydrant new into Slack. Use the "game day" severity to signify this incident is a fire drill.
Tip: We support the following aliases for all slash commands:
As you work through the incident, star important messages in Slack so that they're automatically captured in the timeline and the retrospective.
Here are some common Slack commands you'll find useful during an incident:
Think of notes as a piece of information more important than a normal chat message. You are able to filter specifically for notes on the FireHydrant incident timeline.
4. Resolve the incident
Once the incident is deemed "resolved", resolve it in FireHydrant. The easiest way to do that is by entering
/firehydrant resolve into the Slack channel. This will close an incident and set its status to resolved. A resolved incident will continue to collect chat messages.
5. Run a retrospective
Now that the incident is resolved, it's time to run a retrospective! For the most part, you can run your incidents entirely out of Slack, however, retrospectives need to be done inside the FireHydrant UI. Learn more about running retrospectives in FireHydrant here.
Tip: you can re-order and modify the retrospective questions.
6. Review your analytics
Once you've completed the retrospective, head over to the analytics section of the FireHydrant UI to see what types of metrics we track. Learn more about analytics in FireHydrant here.