Semantic Adversarial Machine Learning (SAML)

By -

ML is everywhere: Fueled by massive amounts of data, models produced by machine-learning (ML) algorithms, especially deep neural networks, are being used in diverse domains where trustworthiness is a concern, including automotive systems, finance, health care, natural language processing, and malware detection. Of particular concern is the use of ML algorithms in cyberphysical systems (CPS), such as self-driving cars and aviation, where an adversary can cause serious consequences.

Adversarial ML (AML) deals with generating adversarial examples to ML
algorithms (e.g modifying a stop sign slightly so that it is classified as a
yield sign). For a general description of AML see  here

Semantic Adversarial Machine Learning: However, existing approaches to generating adversarial examples and devising robust ML algorithms mostly ignore the semantics and context of the overall system containing the ML component. For example, in an autonomous vehicle using deep learning for perception, not every adversarial example for the neural network might lead to a harmful consequence. Moreover, one may want to prioritize the search for adversarial examples towards those that significantly modify the desired semantics of the overall system. Along the same lines, existing algorithms for constructing robust ML algorithms ignore the specification of the overall
system.

In my recent paper with co-authors (Tommaso Dreossi and Sanjit Seshia from
Berkeley) we argue that the semantics and specification of the overall system has a crucial role to play in this line of research. We present preliminary research results that support this claim.

Interested in reading? See  https://arxiv.org/abs/1804.07045

Note: Have feedback. I would love to hear it. Please email at jha@cs.wisc.edu

Comments

Post a Comment

Popular posts from this blog

Malware Trend in 2007

By -
I read the report IBM Internet Security System X-Force 2007 Trend Statistics. This is a report describing trends for various threats in 2007. This team has been tracking trends since 2000. I found the report to be quite interesting. In the rest of this post, I highlight some of the interesting points from the report and what they mean in the context of malware detection. (I) The X-Force team reports continued growth in Web browser exploitation. This clearly shows that the infection vector is changing to the Web. Earlier the primary infection vectors were email and the network. Therefore, for detecting malware, drive-by-downloads (DBD) and other threats targeted at hacking through the Web browser need a lot of attention. (II) X-Force also reports a marked increase in obfuscated exploits, i.e., exploits that use various code obfuscation techiques (such as encryption). Here is a quote, "X-Force estimated that nearly 80 percent of Web exploits used obfuscation and/or self decryption .
Read more

Model Checking and Security

By -
Model checking is a technique of verifying temporal properties of finite-state systems. One of the attractive features of model checking over other techniques (such as theorem proving) is that if a property is not true, a model checker provides a counter-example which explains why the property is not true. Inventors of model checking, Edmund Clarke, Allen Emerson, and Joseph Sifakis, won the 2008 ACM Turing award (see the announcement here ). I have a personal connection to two of the recipients. Edmund Clarke was my adviser at Carnegie Mellon, and Allen Emerson and I have collaborated on few projects and he has supported me through out my career. In this note I try to summarize various applications of model checking to security. Protocol verification : Protocols in the realm of security (henceforth referred to as security protocols) are very tricky to get correct. For example, flaws in authentication protocols have been discovered several years after they have been published. Techniqu
Read more