Managing Director & Partner
Cyberattacks and breaches are not going anywhere.
The frequency and cost of cyberattacks is accelerating. Globally, the cost of cybercrime is estimated to have risen from $445B in 2015 to over $2.2 trillion today.
Meanwhile, advances in AI and the internet of things (IoT), combined with organizations’ pandemic-accelerated adoption of flexible work arrangements and widespread digitization, have exponentially increased both our reliance on cybersecurity (CS) and the potential for attacks. The transition to 5G introduces another whole range of dangerous software-related vulnerabilities. According to the Brookings Institute, “never have the essential networks and services that define our lives, our economy, and our national security had so many participants, each reliant on the other—and none of which have the final responsibility for cybersecurity.” Looking forward, quantum computing—which may become widespread in as little as 5-10 years—will render today’s encryption standards obsolete, with additional major implications for cybersecurity.
Cybersecurity has emerged as a critical risk management priority for public and private sector organizations alike. Spending on information security and risk management technology has increased dramatically and is estimated to reach $168B by the end of 2022
Faced with such questions, CISOs must be able to evaluate and report on their cybersecurity program's maturity based on top-level risks and outcomes and demonstrate to Board members how their organization is performing against its industry and peers. Discussing risks with senior executives has also proven to be a challenge, absent a common language that can be understood by both technical and non-technical stakeholders. The problem for CISOs is that these stakeholders generally lack the technical knowledge needed to understand the details of cybersecurity initiatives, even at the Board level. Highly technical security metrics must be summarized into accurate, easily understood, business-relevant insights for management, Board, and shareholders’ meetings. This is where cybersecurity performance management can help.
Cybersecurity performance management is a process for evaluating the maturity of your cybersecurity program, systematically linking multiple levels of risk, metrics, investment and returns. When part of a coherent, ongoing process, these data-driven, dynamic measurements are valuable indicators of an organization's cybersecurity posture. Establishing a cybersecurity performance management program helps to baseline and prioritize what is important to the business, ensuring alignment with organizational goals and risk appetite, improving visibility, and achieving better outcomes from your security investment. Measuring cybersecurity performance across a range of relevant metrics allows organizations to target improvements, reducing vulnerability through corrective action.
Cybersecurity performance management enables CIOs and CISOs to answer the earlier questions, as well as:
Starting in 2019, Saudi Telecom Company (stc) partnered with Boston Consulting Group (BCG) to introduce a robust cybersecurity performance management framework as part of a larger cybersecurity transformation program. Its purpose was to track program execution progress and impact while providing a complete picture of stc’s cybersecurity maturity. The framework has been implemented, reviewed, and improved over the past three years and has allowed stc to successfully realize its cybersecurity strategy. In this paper, stc and BCG would like to share the experience and lessons learned throughout this journey.
An effective performance management framework is derived from the cybersecurity strategy, that is driven in turn by the vision and mission of the CS program. Cybersecurity vision and mission play critical role in communicating the purpose of cybersecurity to stakeholders, developing a CS strategy, and measuring its performance and success. Aligning the cybersecurity direction with business strategy is extremely important to reflect the internal and external context, mitigate risks and enable the business. A robust performance management framework is defined from the top down (Exhibit 2). Key Performance Indicators (KPIs) and metrics are derived from initiatives to achieve strategic objectives related to focus areas and CS capabilities (e.g., risk, governance, compliance, defense) that shape the cybersecurity practice and ensure business alignment.
While most organizations are aware of cyber-risks, cybersecurity program maturity is uneven. It is vitally important for organizations to have an accurate understanding of where they are and how best to improve. Robust cybersecurity performance starts with a well-defined framework that enables organizations to elevate and benchmark performance across a wide range of CS capabilities, including strategy, business alignment, operating models, governance, risk, compliance, defense, etc. In 2019, with support from BCG, stc established a detailed Cybersecurity Framework. The framework identifies 30 initiatives to shape the cybersecurity practices and set the direction, enforce strategy, build CS capabilities, and secure the business. It reflects international standards and best practices from NIST, ISO 27001, and Gartner. Along with the framework, a tailored Maturity Model was established and defined (Exhibit 3), showing maturity levels for each cybersecurity capability. The model is used in health check assessments to determine the baseline, as well as progressive annual targets for cybersecurity maturity.
Cybersecurity performance measurement and management should enable CISOs to automatically generate dynamic, visual dashboards for use by the Board, executive team, and security operations. In addition to providing a picture of current risks and performance, thoughtfully constructed dashboards link to longer term strategic and maturity goals, highlighting gaps and investment requirement, and accountabilities.
stc adopted a cybersecurity strategy and developed its performance management framework with assistance from BCG, as part of a larger effort to transform and advance the maturity of its cybersecurity capabilities. The performance management framework measures success via three sets of metrics, each focused on a different aspect of performance. From most granular to most strategic:
Each Performance Metric is composed of connected sets of KPIs. KPIs are the critical indicators of progress toward an intended result. They provide a focus for strategic and operational improvement, create an analytical basis for decision making, and help keep attention on what matters most. As Peter Drucker famously said, “what gets measured gets done.”
Within the performance management framework, each cybersecurity strategy initiative is translated into maturity KPIs, which are then aggregated into a Maturity Index for the cybersecurity program across initiatives. Maturity level per initiative is calculated by aggregating the individual maturity levels of its KPIs. For example, ‘Security Architecture’ is an initiative that may include maturity KPIs around the network architecture, application architecture and endpoint architecture. Another example is ‘Endpoint Protection’, which might translate into KPIs related to an endpoint protection suite, DLP solution, and Cyber Defense Center (CDC) integration.
All KPIs are based on measurements. They may be either qualitative or quantitative, with qualitative metrics reported as words in levels, statements, and letters and quantitative ones reported as numbers, including proportions and ratios (Exhibit 3). For instance, ‘Development and Maintenance of Frameworks’ is a qualitative KPI, so measured in terms of maturity levels: L1 Frameworks not developed, L2 Frameworks developed, L3 Frameworks include high-level processes, L4 Frameworks include relevant stakeholders and L5 Frameworks have been reviewed in past 12 months. Qualitative metric owners need to supply proof of completion to ensure objective measurement, such as framework reviewed in past 12 months, alignment validated, or changes implemented and documented. Time from incident occurrence to detection, is a quantitative KPI, measured by the average timespan of the occurrence until the detection of security incidents.
As with so many transformation programs, implementation was key to the success of stc’s cybersecurity strategy. In this section we share what worked for stc – some decisions that paid off and lessons learned along the way.
Executive support is the most important key success factor. Effective cybersecurity performance management hinges on executives’ willingness to prioritize, reinforce, and engage in an ongoing way with cybersecurity improvement efforts. It’s not just technology that must change, but people’s behavior throughout the organization – and that requires leadership commitment. Executives can demonstrate their support in several ways, for example:
Strategy is the starting point to achieving real business value from a cybersecurity program. Cybersecurity strategy includes vision, mission, strategic objectives, strategic initiatives, KPIs, and metrics. Its alignment with the corporate strategy and investment plan is crucial; cybersecurity is a key business enabler as organizations become increasingly heavy consumers of technology and data. Look for KPIs that link cybersecurity strategy to corporate strategic objectives.
Make sure that your cybersecurity strategy is not a just a static document but includes a defined process for execution and outcome tracking. Especially since it is linked to performance management, this is an ongoing effort. stc found a key to success was keeping progress on track and removing obstacles in timely manner – so building that process into the strategy development will avoid delays later.
Developing key performance indicators can be tricky. KPIs need to be well defined and weighted according to critical or core business objectives. Exhibit 4 illustrates stc’s approach to defining KPIs. The headline is not enough. stc created a card for each KPI summarizing its name, ID, initiative affiliation, owner, measurement frequency, data source, description and rationale, measurement formula, performance level, target, and index weighting. As well making CS and business stakeholders think through the details of each KPI, the cards increase the reliability of tracking and provide important consistency across diverse KPIs.
key to an effective ongoing program.stc established a formal Performance Management Team responsible for identifying KPIs, collecting reports, and validating data. KPI reports are collected based on their defined cycle (monthly, quarterly, yearly) or upon a specific request from management. The KPI owners provide current values to the Performance Management Team including relevant evidence to ensure transparency on how the values were derived and corroborate the provided values. This clear accountability – KPI owners, Performance Management Team – is key to an effective ongoing program.
The Performance Team then validates the values. If the provided evidence does not correspond with submitted values, they request the KPI owner to resubmit. There are two general validation levels for each KPI:
After validating the data, the Performance Management Team aggregates all collected values into dashboard to report to management.
Once the validation stage is complete and submitted values for the cycle are accepted, analysis begins, comparing actual KPI values with their targets. This is essentially the step that takes us from performance measurement to performance management. Trend analysis reports should capture any major deviations from the KPI’s cybersecurity maturity target, or from cybersecurity strategy execution initiatives. Results are then provided to top management who can evaluate these trends against cybersecurity targets and strategic objectives.
Cybersecurity KPI measurements, and the maturity levels to which they aggregate, are important inputs for CISO decision making. But they must also be shared beyond the cybersecurity department and incorporated into leadership’s balanced scorecard reviews. This should happen at least quarterly or, more commonly, monthly based on an organization’s needs. Especially when an organization is trying to track the effects of an important change, or in other situations where rapid feedback is needed, reports should be generated and shared monthly. They enable business leaders to monitor performance issues and provide timely support.
stc uses two main cybersecurity performance reports:
A cybersecurity Performance Management Committee (PMC) is responsible for monitoring performance and guiding the execution of Cyber Operations and Strategy. Using the detailed cybersecurity performance metrics report as its main source of reference, this committee meets on monthly basis to:
Maturity indices are used to measure the organization’s overall security posture. Following the principle of "you get what you measure", it is important to choose the right metrics and adjust them if necessary. Especially in the first year of operation, or when conditions are changing rapidly, the approach must be flexible enough to accommodate learning from experience. It is not necessary to wait until the end of an improvement cycle. KPIs may be adjusted during the year if needed, for example if:
To adjust a KPI, stc requires a Change Request (CR) to be completed by its owner, including justification for the change. After the KPI owner’s manager approves the request, it goes to the cybersecurity Performance Management Team for review and feedback. Finally, after the KPI owner and Performance Management Team are aligned, the CR needs to be approved by the PMC.
In addition to ad hoc changes of the kind noted above, an organization should build periodic metrics review and improvement into its cybersecurity performance management process. Such a review might cover each KPI’s rationale, formula, targets, and reporting cycle.
Companies around the world are redoubling their focus on cybersecurity as a business-critical capability. Increasingly, they are recognizing the need for holistic approaches, integrated with their business strategy and goals. Since introducing their cybersecurity strategy and performance management program in 2019, stc has seen a host of benefits including:
With dramatic increases in cyberattacks’ pace, frequency and cost, companies are seeking to learn from each other, identifying and adapting best practices to move faster and stay ahead of evolving threats. In response to this need, and in recognition that cybersecurity is an urgent global priority, stc and BCG are happy to share this paper.