Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators

The project

In November 2018, the CNR-Ircres signed a contract with NARUC, the National Association of Regulatory Utility Commissioners of the United States (www.naruc.org), for the development of guidelines (https://www.naruc.org/international/news/evaluating-the-prudency-of-cybersecurity-investments-guidelines-for-energy-regulators/) aimed at addressing the cybersecurity of electrical systems in countries of the Black Sea area: Armenia, Georgia, Moldova, and Ukraine. The contract is funded by the U.S. Agency for International Development (USAID www.usaid.gov).

Since December 2016, NARUC has been working with regulators from Armenia, Georgia, Moldova, and Ukraine, and later on from the Balkans area, as part of the USAID-supported Europe and Eurasia Cybersecurity initiative. This partnership aimed to help regulators and utilities develop a cybersecurity policy framework that establishes baseline standards and sets in place minimum defense capabilities and good practices at utilities.

In this context, the development of cybersecurity tariff guidelines is intended to provide the regulators with a means of cost-effectively improving energy sector security and resilience against the emerging threat of cyberattacks. As power systems modernize, digitize, and integrate, they are increasingly exposed to additional vulnerabilities that can be exploited by cyberattacks. Attacks on the power grid can have devastating effects on a nation’s security, economy, and public welfare, and are a potent threat to all nations worldwide.

Energy regulators have a unique role to play in the field of cybersecurity. While the implementation of cybersecurity measures is typically the responsibility of power system operators, regulators have an obligation to ensure that investments made in the name of cybersecurity and funded through tariffs are reasonable, prudent, and effective.

Regulators both in the Europe and Eurasia region and across the globe have struggled with understanding and quantifying the degree to which the power grid is better protected based on utility investments made in the name of cybersecurity. In the Europe and Eurasia region, especially the Black Sea countries of Armenia, Georgia, Moldova, and Ukraine, this is an issue of considerable importance given consumers’ sensitivities to rate hikes.

CNR-Ircres has decades of experience in the study of the economics of the power system, with a more recent focus on the specific theme of cybersecurity. In particular, the ESSENCE project (Emerging Security Standards to the EU power Network controls and other Critical Equipment, 2011-2014, funded by the EU CIPS Programme, http://essence.ceris.cnr.it/) carried out an exercise estimating the cost and the benefits of implementing security measures to protect critical infrastructures from cyber-attacks that has never been repeated later on. Then, it represents a unique background for the guidelines, both for the methodological approach and as a source of empirical evidence.

The development of the guidelines was carried out with the constant involvement of regulators, operators and experts, as well as melting skills and knowledge from different disciplines. The project includes also a final assistance activity directly involving the regulators to personalise the results and to pave the way for a practical implementation.

 

The guidelines

The guidelines “Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators” (by Elena Ragazzi (editor), Alberto Stefanini, Daniele Benintendi, Ugo Finardi, Dennis K. Holstein) are intended to assist regulators in defining tariffs by establishing a regulatory approach to enhance the cybersecurity stance of their power systems, and are based on literature and current practices. They attempt to answer the following questions:

• Which regulatory frameworks are best suited to evaluate the prudency of cybersecurity expenditures?
• How can regulators identify and benchmark cybersecurity costs?
• How can regulators identify good countermeasures for cybersecurity?
• How can regulators assess the reasonableness of the costs associated with these countermeasures?
• Is it possible to evaluate the effectiveness of cybersecurity investments?
• Who should identify, benchmark, measure and evaluate the countermeasures in different regulatory frameworks?

In conclusion, these guidelines provide tools and approaches, often discussing several alternatives for each action. Often the philosophy behind their application is discussed as well, but never unique turnkey solutions are suggested because the regulatory strategies are deeply linked to a country’s values and objectives.

These guidelines are a first-of-their-kind resource to empower energy regulators to support and encourage grid resilience by ensuring prudent and effective investments in cybersecurity by their regulated entities. The guidelines, melting skills and knowledge from different disciplines, strive to provide space for concepts, processes and methods rather than prescriptive lists or ready-to-use formulas.

While these guidelines were developed for the Europe and Eurasia region, much of their content can be applied universally.

Elena Ragazzi (ed.), Alberto Stefanini, Daniele Benintendi, Ugo Finardi, and Dennis K. Holstein (2020). Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators, NARUC, Washington DC.

(Russian Version)

Elena Ragazzi (2020). Costs and benefits of cybersecurity regulation. The terms of a complex assessment, Appendix 1 to “Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators”, NARUC, Washington DC.

Elena Ragazzi (ed), Ugo Finardi, Alberto Stefanini (2020). Summary of the main results of the ESSENCE project, Appendix 2 to “Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators”, NARUC, Washington DC.

Ugo Finardi, Elena Ragazzi, Alberto Stefanini (2020). EPRI cybersecurity metrics, Appendix 3 to “Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators”, NARUC, Washington DC.

Daniele Benintendi and Alberto Stefanini, Elena Ragazzi (2020). Implementing a cybersecurity regulation: the OFGEM approach, Appendix 4 to “Evaluating the prudency of cybersecurity investments: Guidelines for Energy Regulators”, NARUC, Washington DC.

 

Contents

1 CONTEXT AND PURPOSE OF THE GUIDELINES

2 INTRODUCTION: PRELIMINARY CONCEPTS FOR CORRECTLY USING THESE GUIDELINES

2.1 Enhancing cyber preparedness in different regulatory frameworks

2.1.1 Performance-Based regulation (PBR)

2.1.2 Cost-of-service regulation (cost-plus)

2.1.3 Regulatory framework – conclusions

2.2 New threats require new defense strategies

3 EVALUATING CYBER-RELATED EXPENDITURES INCURRED BY UTILITIES: COST IDENTIFICATION AND BENCHMARKING

3.1 From cost identification to cost calculation

3.2 Identifying priorities

3.3 Benefit analysis

3.4 Costs and benefits of cybersecurity countermeasures

3.5 Security areas

3.5.1 The governance of cybersecurity

3.5.2 Hardening

3.6 Estimating the cost of countermeasures

3.6.1 The costs calculated in the ESSENCE project

3.6.2 How to transfer ESSENCE results to other contests?

4 EFFECTIVENESS METRICS

4.1 Identifying good effectiveness indicators

4.2 What is effectiveness? The concepts of output, outcome, and impact

4.3 The governance of metrics 

4.4 Cybersecurity metrics

4.4.1 Maturity metrics

4.4.2 The EPRI metrics: the most comprehensive and mature approach to assess general cybersecurity performance

4.4.3 A critical issue of the EPRI metrics: data aggregation

4.5 Comparative assessment and conclusions

5 AN APPROACH TO INVESTMENTS IN CYBERSECURITY

5.1 Background

5.2 Types of measures and actions

5.3 Building cybersecurity scenarios starting from cybersecurity objectives

5.4 Cybersecurity scenarios

5.4.1 Scenario 1: a compliance-based approach in cost-plus

5.4.2 Scenario 2: a semi-participatory approach in cost-plus

5.4.3 Scenario 3: a participatory approach in cost-plus

5.4.4 Scenario 4: experimenting with incentives to enhance the maturity level

5.4.5 Scenario 5: relying on companies’ strategies without relying on metrics

5.4.6 Comparison among scenarios

6 CONCLUSIONS

7 REFERENCES

 

Video presentation of the guidelines

June 17, 2020

A presentation of the guidelines was hosted in the joint USAID/NARUC/ERRA/CEER webinar series on Implications of the Global Pandemic on Tariff Design and Utility Finances. The objective of this webinar series is to examine how regulators can consider the short and long-term implications of the COVID-19 global pandemic on the energy sector, with respect to tariff design and consideration of the financial status of utilities.
The webinar The Regulatory Role in Supporting Cybersecurity Investments was included in this series because experts have warned of an increase in cybersecurity threats as the COVID-19 pandemic creates opportunities for malign actors to increase their cyberattacks. Vulnerabilities have also increased as resources continue to be limited, and software updates become less frequent. As such, it is even more important for regulators and utilities to further cooperate and employ necessary cybersecurity investments during this time, despite the financial and resource constraints. During this portion of the webinar series, NARUC and Elena Ragazzi presented the groundbreaking report, Evaluating the Prudency of Cybersecurity Investments: Guidelines for Energy Regulators, funded by the U.S. Agency for International Development (USAID).

• Agenda of the event 
• Slides
• Recording

 

October 22, 2020 – 9:30 am to 10:30 am EDT

webinar on: The Relationship Between Regulators and Power Utilities: Evaluating the Prudency of Cybersecurity Investments

 

 

Acknowledgements:

This publication is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of the National Association of Regulatory Utility Commissioners (NARUC) and of the authors and do not necessarily reflect the views of USAID or the United States Government.

This publication was produced with funding from the Energy and Infrastructure Division of the Bureau for Europe and Eurasia.

NARUC and Ircres would like to thank the following professionals for their valuable insights and for their time and expertise in designing, reviewing, and editing this document:

Stefano Bracco, Security Officer and Knowledge Manager, European Agency for the Cooperation of Energy Regulators (ACER)

Geoff Marke, Chief Economist, Missouri Office of Public Counsel

Commissioner Ann Rendahl, Washington Utilities and Transportation Commission

Commissioner Dan Scripps, Michigan Public Service Commission

Mohammed Zumla, Head NIS Competent Authority, Office of Gas and Electricity Markets

Former NARUC employees Paul Stack and Crissy Godfrey

Hisham Choueiki and Colleen Borovsky, NARUC

NARUC and Ircres would also like to thank the following national regulatory authorities for their contributions:

Public Services Regulatory Commission of the Republic of Armenia (PSRC)

Georgian National Energy and Water Supply Regulatory Commission (GNERC)

National Agency for Energy Regulation of the Republic of Moldova (ANRE)

National Energy and Utilities Regulatory Commission, Ukraine (NEURC)

 

The project team:

Elena Ragazzi (project leader)

Elena Ragazzi is senior researcher at CNR-Ircres, where she has been working since 1989. She is also an external professor at the Polytechnic University of Turin. She is the author of more than 250 works of applied economics and policy evaluation. She was project leader of several projects, both European and national, coordinating the action of numerous partners, integrating multidisciplinary working groups and mixed groups of researchers and professionals. She organized and chaired special sessions on impact assessment and counterfactual methods. Her research activity in recent years has focused on the theme of the evaluation of public policies and regulation. One of the main research lines targets the power system and, in particular, its protection from cyberattacks. http://www.ircres.cnr.it/images/ragazzi/RagazziENGL.pdf

Alberto Stefanini (expert in the power system and in security standards)

Alberto Stefanini received a full honours degree in Electronic Engineering from the University of Bologna, 1974. He is currently retired and holds a consultancy. Since 2008, he set up several EU projects, included: FM BIASED, ESSENCE, SESAME and GRID. Previously he worked with the Joint Research Centre of the EC on cyber security threats to power system, and with CESI on the diffusion and take-up of research results on the power system. He has been very active with the European framework programme since the ‘70 and contributed to launch a large number of European projects on several subjects: industrial diagnostics, man-machine interfacing, decision support and the life cycle of industrial automation.

Daniele Benintendi (expert in regulation)

Dennis K. Holstein (expert in cybersecurity)

Ugo Finardi (expert in innovation policies)

Ugo Finardi (MSc in Industrial Chemistry, Ph.D. in Materials Sciences and Technologies, discussing a thesis on the Technology Transfer practices in Nanotechnologies) is at present Researcher at CNR-IRCrES, the Research Institute on Sustainable Economic Growth of the National Research Council of Italy. He has been working previously at the University of Torino. His main research interests are: the socio-economic impact of technologies and innovation and of technology transfer; the organisation of universities and public research bodies; bibliometric assessment of the state and evolution of research. He has published more than 70 works among articles and book chapters in international and national journals and books, congress contributions, working papers and research and technical reports.