.svg){kind=logo}
.svg)
There are many terms to keep track of - but what do they mean?
It's often unclear what you even mean by "IT readiness" when you have a conversation about it.
In many places, different terms and definitions get mixed up, and it's not helped by the fact that different standards also use the terms differently.
That's why we've created this model, which provides an overview of the elements of emergency response.
Let's talk you through the model, starting from the bottom:
The prerequisites for good emergency response
The fields at the bottom of the model are the company's basic prerequisites for the emergency response to work in practice and not just be an organizational exercise. This foundation should consist of:
6. Technical Recovery Plans(Technical Recovery Plans/Disaster Recovery Plans)
These plans provide a concrete step-by-step recipe on how to restore a system and what prerequisites, such as licenses, backups, rights, etc. are needed to complete the restoration.
The plans should only contain the essential information needed for recovery and should not be confused with system documentation. If you need external consultancy assistance for the recovery, this plan must contain the information that the consultant needs in order to perform the work.
Naturally, the plans must be placed in a place where they can still be accessed even if the infrastructure is unavailable. These plans are also used during regular outages where systems need to be restored, so they are not reserved for a crisis.
7. Scenario-based action plans
To counter a predictable scenario that will affect multiple systems simultaneously, you can describe an action pattern that is good for the specific situation. This is what you see in a first aid book, where there is a sequence of actions that are different for e.g. burns and drowning accidents.
It is common to have a scenario-based action plan for ransomware attacks. It describes the response pattern you've agreed on, with specific prepared actions that can counteract the situation from developing negatively.
For example, it can describe how a network is quickly divided into segments (also called islanding) or parts of the network are shut down.
A scenario-based action plan can be automated as scripts that execute the actions quickly, as the response must be completed within minutes. There should be management approval for these plans as they often dramatically impact business operations and will trigger one or more business contingency plans.
8. Prioritization and dependencies
If systems need to be re-established, it will be done according to a "queue system" operated by the IT department. Here, the IT infrastructure is restored first and then the systems that are a prerequisite for the rest of the systems to function. Based on their prioritization of the importance of the systems, the business can ask for them to be restored in a specific order that can be prepared.
This is often done based on a Business Impact Assessment(BIA). This ensures that the most important systems and services are addressed first and that critical business processes are re-established as quickly as possible.
As there are often multiple business areas, the people responsible for the business areas must agree on their priorities so that there are no internal conflicts in an emergency situation. This order can of course be changed in the situation, but it's much faster than starting with nothing.
Dependencies between systems and services can be harder to uncover if you don't have a well-maintained CMDB(Configuration Management Database). If it's not well described, it's beneficial to create some simple sketches that show basic dependencies, as this helps with decision-making in a crisis and affects the order of recovery.
9. Robust design and architecture
Some of the problems caused by a crash or cyber-attack can be reduced in advance with a robust design of architecture and systems. You should strive to ensure that there is a correlation between the criticality of a system and the robustness of the design.
This is often an economic consideration, as it is costly to have redundant systems.
Examples of robust design include redundant systems, duplicated services, segmented networks, backup designed for disaster recovery, and using cloud services that can run independently of on-premise systems.
10. Alternative Services is the IT function's plan B
The business contingency plans will describe the requirements for emergency operations, and this is based on some alternative services that must function in everyday life so that they are ready in a crisis.
It can be an alternative communication platform or daily data dumps to ensure business continuity. These services can be "dormant" until they are needed and must again be independent of your own network.
Similar to how you used to print out lists every day so you could use them in a crisis. You still can, but time has moved on from this approach as it's too resource-intensive and often doesn't solve all the needs you have.
The daily operations
The blue fields are the areas that belong to the company's daily operations. The whole purpose of this area is to ensure that an incident does not develop into a crisis.
5. Theoperations part consists of:
- Business processes are the regular business operations. If it becomes challenged, business continuity plans can be put in place.
- Incident Processes are a common and often local problem. For example, it could be that an employee's computer is not working.
- Major Incident Processes can, for example, be when many computers in the company are not working or several critical systems are affected.
On the left side of the model you can see a scale from one to five. This is a scale for the severity of the incident where 1 is the most critical. The most serious in an IT operations organization is called a Major Incident. One or more Major Incidents will often draw on all the resources available to an IT department and mean that the normal service level drops significantly.
When a serious incident occurs, the IT department immediately starts trying to solve the problem by any means necessary. If this is not possible, the button is pressed and the emergency management team steps in and takes over crisis management. It saves time to notify emergency management for all major incidents.
It will typically be an IT manager in the company who has the IT crisis manager role and thus decides whether there is an emergency situation.
Crisis management and emergency management
These fields are about the part of the emergency response that is triggered the second an incident has transitioned into a crisis. Below are the areas:
2. IT Crisis Management(IT Crisis Management)
This is the IT department's crisis management and is a layer on top of the Major Incident work. This is a war room with clear roles and structured management. More resources are often added and decisions are implemented with a prepared mandate.
This is where the big picture is kept and there will often be roles here such as HR, Communication, Coordination, Business Management and Facilities.
This is where long-term efforts are coordinated, prioritized and communicated to all stakeholders. The IT crisis management team can enlist the support of outside specialists to handle the situation and also takes responsibility for any deviations from policies and guidelines.
3. Maintaining ITService Continuity (IT Service Continuity)
This is an almost unknown concept to many and can only be defined and implemented in close collaboration with business management. Business continuity is not the same as redundancy, but will often be alternative systems or data extracts made available in other ways. It could be a spreadsheet ready to replace quality management systems if they go down.
The data needed to move your business forward in a crisis is often small, but it's hugely important. You may be able to retrieve goods from a warehouse with a pen and pad, but it will take a long time to get that data imported once the systems are back up.
That's why an alternative digital solution is often better.
4. Business Continuity(Business Continuity)
These plans describe how the business will continue if you are suddenly without your IT systems or other critical resources. These plans are not made by the IT department but by the employees who know the work processes.
They know their own area's daily routines, needs and regulations. Those responsible for processes and business areas are often the ones in charge of creating business contingency plans.
Business continuity is often used as an umbrella description for preparedness. But it's just one area of overall preparedness, and whatever you call it, be specific.
1. Overall crisis management(Corporate Crisis Management)
This is the crisis management of the entire organization. If IT is down, it can bring the whole company down, and then the whole company is in crisis. The IT crisis management team, led by the IT director, resolves the IT crisis, while the overall crisis management is handled here. Here, it is usually the CEO who is the crisis manager.
This plan is not limited to IT incidents, but can also include war, extreme weather, pandemics and the like. However, an IT incident will very often activate the entire company's emergency response, which is why it is important that there is a close and well-functioning collaboration between the two crisis teams.
Testing the plans
All plans should be tested regularly with scenarios that are realistic and challenging. Plans should be tested individually to make the testing as concrete as possible.
It's a good idea to create a test plan that spans three years, as it's often not possible to test all plans within 12 months.
Testing plans should be tailored to a specific purpose and have an appropriate level of ambition. It is important that concrete learning comes out of the testing, which is why you can benefit from a test plan that increases in complexity as you get better.
It could be in these steps:
- Peer-review the plan. If your colleagues don't understand the plan, it probably needs to be corrected. The plan is read through and adapted by someone with sufficient professional knowledge.
- Scenario-based simulation. The plan is tested in a simulated incident that is customized to the business. Actions are limited to being descriptive and are not taken.
- Technical testing. This is where you test the parts of the plan that will not negatively affect the company's operations. This could be a test of SMS crisis communication, failover of individual redundant systems or contacting suppliers.
- Operational testing. A scenario is tested as close to reality as possible. Systems are re-established, partners are contacted and employees can even be evaluated as part of a fire drill. This is associated with significant costs and risks and is therefore carefully planned well in advance. Often, the elements of a live test will have been tested at level 3 before being included in the exercise.
Good advice - and the human aspects
When a crisis arises, it's often necessary to control the battle with a military mindset.
You have to run fast. Decisions have to be made hard and fast and employees will have to go the extra mile. Therefore, you also need to keep an eye on the human aspect of the crisis.
Often through an attack, people are pushed very hard for a long time, perhaps around the clock, and at some point they can't take it anymore. The HR function will play an important role in safeguarding employee well-being and preventing stress and poor working conditions. It's also important to remember that most employees do not have a contractual obligation to work exceptionally in a crisis and should therefore be handled with care.
Demant and Maersk are examples of companies where a cyberattack hit and lasted for months, and where some systems never recovered. Employees don't forget how hard it was to be a part of the incident and how it was handled becomes part of the company's future image.
Do NOT lay the rails while the train is moving
It should go without saying that preparedness should be established before a cyber attack occurs. Building your preparedness takes time, and you don't have much of that when you're under attack.
The time spent preparing and making plans is taken directly out of the time needed to recover from a cyber attack. There are often difficult decisions to be made under intense time pressure, and it's better to have these discussions before the incident occurs. This way, the pace during a cyber attack can be accelerated and there is less uncertainty as you can stick to a known framework.
If you're looking for inspiration for good emergency preparedness, look towards the Armed Forces, Emergency Services or general First Aid. Here are methods that have been tested in life-threatening situations and adapted over generations. Use their operational experience and notice how simple and concise the methods often are.
Recommendations for inspiring reading material:
- The Checklist Manifesto, Atul Gawande
- Battle Mind. Performing under pressure, Merete Wedell-Weddelsborg
- Common first aid, Sundhed.dk
- Defense's 5 point commandmentDefense Academy.
