Session: Threat Modeling AI and ML for Resilience
Science fiction books and movies are full of general knowledge AI - hyper-smart, independent systems that can think for themselves and even repair themselves in a malfunction occurs. The reality is that while AI and ML can accomplish a dazzling array of sophisticated tasks, they are not sentient and unless some element of self-healing functionality is built in, they are not going to be able to fix themselves if something goes awry. Using a published framework for reference, we will explain what causes AI & ML to fail and enumerate intentional failures that can occur when a criminal or attacker tries to cause the system to malfunction. We’ll also describe unintentional failures, how these systems can falter under normal use. While a lot of attention is given to systems that are broken or manipulated on purpose, it’s very important to look at ways systems fail while being trained and under normal use. Especially in the case of AI & ML. We close with recommendations for building resilience into AI & ML.
Failure Modes Covered:
Perturbation & Adversarial Universal Perturbation
Model Inversion
Data Bias
Reward Hacking
Distributional Shifts
Over/Under-fitting
Bio
Diana Kelley is the CSO2 (Chief Strategy Officer/Chief Security Officer) and co-founder of Cybrize. She also serves on the boards of Cyber Future Foundation, WiCyS, and The Executive Women’s Forum (EWF). Diana was Cybersecurity Field CTO for Microsoft, Global Executive Security Advisor at IBM Security, GM at Symantec, VP at Burton Group (now Gartner), a Manager at KPMG, CTO and co-founder of SecurityCurve, and Chief vCISO at SaltCybersecurity.
Her extensive volunteer work has included serving on the ACM Ethics & Plagiarism Committee, Cybersecurity Committee Advisor at CompTIA, CTO and Board Member at Sightline Security, Advisory Board Chair at WOPLLI Technologies, Advisory Council member Bartlett College of Science and Mathematics, Bridgewater State University, and RSAC US Program Committee.
She is a sought-after keynote speaker, the host of BrightTALK’s The (Security) Balancing Act, co-author of the books Practical Cybersecurity Architecture and Cryptographic Libraries for Developers, has been a lecturer at Boston College's Masters program in cybersecurity, the EWF 2020 Executive of the Year and EWF Conference Chair 2021 and 2022, an SCMedia Power Player, and one of Cybersecurity Ventures 100 Fascinating Females Fighting Cybercrime.