Wednesday, January 30, 2008

Case for kernel-level detection

Why kernel-level detection?
These are my thoughts on why malware detection should performed at the
kernel level. In general, the lower in the system hierarchy your
detector resides, the harder it is for an attacker to evade your detector.
For example, if a detector uses system-call interposition, an attacker can
evade this system by directly using kernel calls. For example,
system-call interposition can be done on Windows using the following
package. In my conversations with
a guy from NSA (name withheld for obvious reasons:-)) he confirmed that
new malware they are observing in their lab are using kernel calls directly.
Also, look at the following article


The semantic-gap problem:
A natural question that comes to mind is: why not perform detection at even a lower layer
in the heirarchy? Say the VM layer or even better at hardware. As you move
down in the system hierarchy, you lose some high-level semantics. Let me explain.
Lets say you are doing detection at the VM layer. A high-level event (such as
opening a file) manifests itself as a sequence of events (such as writing to
memory page or an interrupt). In other words, there is a gap between
the events you observe at the VM level and the corresponding high-level event. To my knowledge
the "semantic gap" issue was first articulated in the following paper:

Peter M. Chen, Brian D. Noble, "When virtual is better than real",
Proceedings of the 2001 Workshop on Hot Topics in Operating Systems (HotOS),
May 2001.
The paper can be downloaded at the following site.

As you move down in the hierarchy, the semantic gap problem becomes harder. The
semantic gap problem still exists at the kernel level, but it is more tractable
than at the other layers. Therefore, I think kernel-level detection hits
the "sweet spot". Implementing detectors at kernel level is harder than other
approaches (such as system-call interposition), but then everything good in life takes
effort:-) I strongly believe that detectors that use system-call interposition are very
easy to evade, and so what is the point in having them. The next generation of malware
will definitely use kernel calls directly.