Priya Mouli - Demystifying Cyber Risk Metrics and ReportingApply to Speak

Automatic Summary

Unlock the Power of Technology Risk Management: An Overview by Priya Mauli

Hello everyone! My name is Priya Mauli, a director based in New York for KPMG’s technology risk consulting practice. I specialize in cybersecurity privacy, data protection, and regulatory compliance - particularly for financial services and technology companies. In this article, we will be delving into the important subject of technology risk management and particularly the concept of risk metrics.

About Technology Risk Management

Technology risk management addresses any possible risk that pertains to data or technology assets in a company. It serves as a broad framework to effectively manage risks across disciplines such as identity and access management, application security, third-party security, and enterprise resilience.

Why Now?

According to a recent survey, most organizations view technology risk as an “internal regulator" or a "sledgehammer," reacting to issues only after they arise or a breach occurs. Additionally, less than 50% perform risk assessments for cloud-connected and mobile devices in their risk assessments, notwithstanding the upward trend in digital transformation. Now, more than ever, proactive risk management is essential, and risk metrics are an effective method in achieving this.

Introduction to Risk Metrics

Before delving into risk metrics, we need to understand risk appetite, which is the type and amount of risk an organization is willing to take. Risk metrics, conversely, serve as measures (numbers, percentages, or trends) of the risk that an organization faces. When these measures reach certain thresholds, trigger actions can be automated to help companies better manage their risks proactively.

The Process of Risk Metrics

Building risk metrics requires:

  1. Available supporting data.
  2. High confidence from key stakeholders in the said data.
  3. A standard language for risk and controls to foster a common understanding across business functions.
  4. Alignment of the metrics with key risks.

Moreover, it is crucial to have an up-to-date data dictionary capturing essential data like KRI formulae, thresholds, data elements, update frequency, and corresponding owners.

Risk Metrics Examples

Some real-world examples of risk metrics could include:

  • Percentage of incidents impacting systems with personally identifiable information.
  • Number of privileged system accounts monitored.
  • Percentage of emergency changes.
  • Procurement rate of collaboration tools such as Zoom or Microsoft Teams.

Challenges in Using Risk Metrics and Overcoming Them

Risk metrics must be viewed collectively to provide a complete picture of true risk exposure. One of the biggest hurdles companies face when dealing with risk metrics is poor data quality. Regular refreshment of the data, proper ownership, a systematic record system, and an interconnected ecosystem are necessary redresses to this issue.

Risk Reporting

As we transition from risk metrics to risk reporting, it is fundamental to understand that reporting ought to be done timely across all organizational levels. Effectively using risk reports encompasses:

  • Making the content relevant and succinct for all involved parties, including the executive leadership and the board.
  • Having a forward-looking agenda that calls attention to potential risks.
  • Having a balanced mix of qualitative data and quantitative data.
  • Automating the risk reporting process.

Concluding Thoughts

Starting with the basics of risk management, we can leverage new technologies at scale once we have our foundations right. In this digital era defined by rapid transformation, it is crucial not to let cyber risk take the back seat. It's time we start thinking about proactive risk management - and risk metrics are the key solution!

Feel free to drop any questions in the comment section below and keep the conversation going. Stay alert, stay proactive!


Video Transcription

OK. Hey, everyone. Uh First of all, um I thank you all for taking the time and joining us for this session today. And a uh special thanks to the women tech conference team for having me.Uh First of all, I hope you and your families are all staying safe and healthy and are all vaccinated. Like many of you, I too cannot wait to just get out and start uh traveling. Um OK, some housekeeping items before we get started. Uh This session is around uh 20 minutes. I will cover the presentation material um in about 15 minutes. Uh So there's at least five minutes for questions and answers. So as questions come along, uh do feel free to type them into the chat window here by way of introduction. Um I am Priya Mauli. I am based out of New York and I'm a director in KPMG S technology risk consulting practice. I cover the areas of uh cybersecurity privacy, um data protection and regulatory compliance. Uh Most of my experience has been with supporting financial services and technology companies um with their cybersecurity strategy. So I've worked with uh the three lines of defense on building really operationalize their tech risk programs.

My recent focus has been leading engagements around continuous controls, monitoring, uh streamlining processes between risk and compliance management and really working with senior management and the board to really think through that middle ground between security risk and agility in the race to adopt disruption.

Uh What I do for fun. Uh I love watching movies in theaters. Uh So with COVID around I uh Binge Watch, uh Netflix and particularly tune into um uh Grey's Anatomy. Uh And I spend my free time writing poems, learning new languages and uh learning about uh wildlife, particularly predators. OK. Um So let's move on to our uh discussion for today. Um um Let me move into the agenda slide. So, um before we jump into the world of risk metrics, I wanted to start off with an introduction of what is technology risk management. And after that, we will move on to risk appetite to set the foundations and then move on to the what, why and the building of risk metrics. And um ultimately, we will move into risk reporting. But what is uh technology risk management? I know the slide has a lot but then simply put it deals with any risk that touches data or technology assets in an organization. Um Technology Risk management is meant to be an umbrella framework to help with managing risk across multiple disciplines, um such as identity and access management, application, security, third party security, enterprise resilience and so on.

So why are we talking about tech risk and uh why now um we are revolution today and based on a recent survey, we're still seeing that for the most part, technology risk is still seen as a sledgehammer, so to speak or as an internal for regulator. Uh as a result of which they are engaged only after an issue surfaces or a breach occurs also uh less than 50% of organizations account for cloud connected devices and mobile device and application management in their risk assessments. Now, this is a shockingly low number given the huge uptick in digital transformation, especially in a post COVID world. But it is important especially now to not let cyber risk take the back seat. Also, hackers are just waiting for the right moment to strike and we've already seen ransomware attacks to this effect. Uh The most recent ones being colonial pipeline and J BS where hackers are really looking beyond the data into targeting the physical assets. And here we see cases in point around targeting the critical infrastructure and supply chain. So it only makes sense to start thinking of proactive risk management and risk measures or metrics is a way to achieve just that. Um With that said, let's enter the world of risk metrics.

Um um OK, so um risk, so, so as I mentioned earlier, uh let's set the foundations right? With looking at risk, appetite. So risk, appetite is the amount and type of risk that an organization is willing to pursue or can retain or can take. Um And what I've seen across my clients is that risk appetite is generally a set of overarching qualitative statements that depend on the company's business context. For example, keeping the customer transaction availability is important for a retail bank with an omnichannel presence.

Um The OCC heightened standards has some guidance around how to set risk appetite within the organization and who within the organization is responsible for setting this and ensuring that it stays current. Um um OCC by the way, stands for office of the Controller of Currency.

Um Essentially what this is is that it's an independent bureau within the United States Department of Treasury that serves to regulate and supervise all national banks and licensed agencies of foreign banks to ensure the safety and soundness of the US banking system. Um So you might be wondering why are we talking about risk appetite as it concerns metrics? Uh Well, we generally want risk metric thresholds to be in line with their risk appetite. Um And this brings me to why we even need metrics. So risk metrics, as you see on the slide are measures, numbers, percentages or trends that serve as a measure of the risk that an organization faces. And this can be tailored at varying levels of granularity. Um and uh risk metrics with thresholds and corresponding trigger actions to find.

Can enable companies stay ahead of the curve by allowing them to get visibility into risks before they occur and hence, enables proactive risk management. Now, it is important to understand that it is these trigger actions when certain thresholds are reached, that can be automated and not the metrics themselves. Um Now let's move on to the Kr I build process the how of risk metrics. Now, this slide lays out four basic steps but note that this is a cycle, right? As Kris need to be refreshed periodically, uh let's talk some ground rules for developing them. First off data should be available to support the metric and key stakeholders such as your risk, control and process owners should have high confidence in this data. Uh It would also be helpful to have a risk and controls taxonomy, namely a domain hierarchy or structuring or framework.

And what this means is that the intent is to really standardize the risk and controls language to enable a common understanding of risk across business functions. Um Another thing to keep in mind is that metrics work best when they are aligned to a key risk and serve as an effective measure of that risk. Uh for example, uh absolute numbers in the case of audit or regulatory findings work better and relative trends such as percentage increase or percentage over a period work better when you're measuring say percentage of emergency changes or uh the change failures. So um Kris could also serve as you know, strategic or aspirational measures, meaning what an organization may want to focus on in the near future. For example, metrics on insider threat, um you can also have metrics around uh you know, uh that could be special purpose or trigger based in cases of a merger or acquisition or divestiture, right. Uh The reason I bring this up is that with COVID, we see a lot of consolidation across industries. Um Another point I want to mention is that metrics could also serve as maturity indicators of your program.

For example, if your organization has a higher number of automated versus manual controls, you know that you are at the higher end of the maturity curve and last but not the least, right. But the most important in my view is to have an up to date data dictionary, capturing the entire um data such as the Kr I formula, uh thresholds, the data elements that constitute the formula where they are sourced, their update frequency owners, um which reports they land on um et cetera.

Now let's go through some examples of Kris to make it real. Uh One metric that comes to mind is um you know, the percentage of incidents impacting systems, housing that meaning personally identifiable information. Um Another metric could be the number of privileged system might wanna monitor.

Um uh And you can see if this is trending up or down. Uh You might also want to use uh composite or complex metrics, for example, in the same privileged accounts. Um case you might want to break it down by employees, meaning full time and part time versus third parties, including your vendors, contractors, business partners and also uh slice it down by their concentration by geography and have the split by the critical application systems and platforms.

They have access to another timely. Kr I that comes to mind is the percentage of unplanned or emergency changes and the percentage increase in the procurement of collaboration tools such as Zoom or Microsoft teams with certain contractual classes uh relaxed. Now I brought these specific examples, recognizing that some firms may be relaxing their thresholds given the current situation. Uh what I'd like to highlight is that and to resume to original thresholds, once things get better, we should be OK. And like I said, the Kr I development process laid out is a cycle and there needs to be an asso governance uh governance process for any changes to the metrics, say proposing new ones, changing formula and related data including ret retire of these metrics. Uh Now that we've covered the how let's discuss some of the common challenges in using metrics and how to make them uh more effective. Um This slide is pretty self explanatory, but three things I'd like to emphasize are this one here, Kris need to be viewed together to give you the visibility into your true risk exposure. For example, you might have a metric around the percentage of assets that are end of life or coming to end of life. Another metric you might have might be around the percentage of assets that you really have. Um uh insight into.

For example, um if you have a number of gaps in your asset inventory reconciliation process, you know, you have a problem. Uh The second most important point is poor da data quality. Now, there is a reason as to why I've blown this up on the slide. I can't emphasize enough on this as this is what most companies struggle with. Not necessarily because they have bad data to start with. But issues start arising when this data is not refreshed regularly when it's not updated by the right owners, when there is no system of record or you know, having a disconnected ecosystem, meaning tools not talking to each other. Um So let's land into the world of risk reporting, right? Because now we've spoken about risk metrics. Let's talk about how the next step, the next steps of how Kris their trends and related data will be used to really drive risk decision. Now, risk reporting is generally considered to um uh yeah, it's generally um considered to be an upper management item, but it is important that reporting happens timely across all organizational levels, right? From the day to day, operational teams to middle management, to leadership and ultimately to the board.

Now there are quite some guiding principles on this slide. But four points I'd like to call out are relevant. Understanding audience is very in terms of fine tuning the content and the level of detail and determining the appropriate frequency of metrics, especially for uh you know, um uh executive leadership in the board. The content should be made available succinctly across devices and platforms uh with drill down abilities into the detail as and when they need it. Um The second point which is point number four here is advanced and aspirational. Now, risk reports should have a forward looking agenda in terms of bringing attention to emerging risks based on industry trends and your internal environment. Um The third point is around numbers and words so well, what I mean to say here is that risk reports should have the right mix of qualitative data, meaning commentary from risk and control owners with respect to certain metric values, your trends, any action items planned and quantitative data, meaning the actual metrics and their values themselves.

Now, the critical element on the slide is around automation of the risk reporting process. Soup to nuts, right from pulling source data to translating them to risk metrics, gathering commentary from stakeholders and pushing them into visualization tools or dashboards using technologies such as you know, a tableau a click view or a Power Bi. And this works best for two reasons.

A it improves data confidence by reducing manual intervention in the data and b the number of man hours saved for every reporting cycle could be huge. And this is based on what I've seen at almost every organization I've worked with and I'm certain you might have to. Now, the other aspect I would touch on is that some of the large banks I work with are seeing more regulatory scrutiny in involvement of their boards in providing credible challenge and weighing in on cyber risk related decisions. And what this means is that the reporting going to the boards, right? Should be timely clear, concise, absorbable and needs to be more transparent from company management, even if it means delivering bad news without any fear of repercussions. Um ok. So now uh uh we've covered the guidance, I talk about the inputs and outputs of risk reports. Um Ju ju this slide ha has a lot to say but a couple of things I'd like to highlight or call your attention attention to are um make sure that when you have risk reports, you cover the entire tech risk universe, meaning across domains, right? Such as I mentioned, application security, third party security enterprise resilience outside of focusing just on compliance issues. The second point I'll touch on is that perform, it is really critical to perform a period analysis by taking your risks, issues, control dictators problems.

However, your organization calls them to understand the ANDRS to really pinpoint those areas needing net new controls or control revamps and Um This can potentially also show you areas where you can tune down your existing controls resulting in cost savings. With that said we will move on to the next slide. Uh This basically has some CEO survey results on stats with respect to what companies are seeing, with respect to some of the common challenges they are facing in seeing the true risk reporting. Now, let's talk about some of the ways to overcome this A is around having a uniform or standardized risk and controls language and taxonomy as I mentioned earlier. And B is having a golden source, meaning having a system of record like a GRC, meaning a governance risk and compliance tool to store all risk and control related data. Um and it might not be your single repository, but it's also important to kind of make sure that this uh system of record that you're using for your risk reports is connected to your other tools like say your Jira issue tracker um and your, you know, service now technology uh tickets uh that you're housing.

OK. So um with that said, uh I wanna part off with some concluding thoughts before we move on to questions. Now, we may sometimes over engineer and sometimes re invent the wheel but uh on risk management. But if you take a step back, it might actually not be that complex to start with. It's important to start with the basics, no doubt. And once you have the foundations, right? Like having a uniform taxonomy enabling a common understanding of risk across business functions in an organization, we can potentially embrace and unleash unleash new technologies at scale. And we've already seen a huge uptick in digital uh transformation in a COVID world. All right.

Well, thank you so much for your time. That's all I had.