MemCrawler: Discovering Structures in Memory

MemCrawler: Discovering Structures in Memory Ellick M. Chan, Francis M. David, Jeffrey C. Carlyle, Roy H. Campbell Department of Computer Science University of Illinois at Urbana-Champaign {emchan,fdavid,jcarlyle,rhc}@uiuc.edu

1.

OVERVIEW

We present a tool which analyzes the memory of an executing system to identify data structures through the use of semantic constraints. The direct analysis of a system’s memory allows us to reveal important information about the state of the system which may not be available through normal means of evaluation. Our prototype tool, MemCrawler, is focused on analysis of a Linux kernel running in a virtual machine; however, these techniques should be equally applicable to other systems. The Linux kernel already provides several means for inspecting the state of a running system. Entries in the proc and sysfs file systems can be used to gather information about running processes and device drivers, and the kernel’s symbol table is usually available and can be used to locate many data structures in memory; however, not all interesting data structures have a query interface. Additionally, these query interfaces generally only return information about active data structures. Direct analysis of the kernel’s memory can not only reveal important state information about active processes, memory allocation, and network connections that might not be readily available through standard query interfaces, but can also allow us to discover data structures that might be intentionally hidden or that are in freed memory that has not yet been overwritten. Since MemCrawler relies on specified constraints for data structure analysis, it does not require a symbol table for the target kernel. Our current prototype shows promise and is already able to discover process lists and identify about a dozen kinds of typed objects present in the Linux SLAB allocator with a low false-positive rate. MemCrawler operates specifically on non-paged kernel memory, which simplifies the identification and analysis of valid kernel pointers. Memory analysis is based on data structure pattern matching and inferencing. First MemCrawler identifies candidate data structures by matching against constraint templates. Each template describes the C structure layout and constraints for data values. The candidate structures are collected into a pool where the inferencing stage resolves structure ambiguity and filters out false positives from the first round. The inferencing stage relies on data structure semantics to reason about the types and structure of the candidates. Once the process is complete, a list of object addresses and types is created. This information can be used with a graphical debugger such as ddd or gdb to analyze the kernel data. Another use is for forensics and malware analysis. Certain data structures that are discovered such as inodes or process structures reveal what files have been recently active and what processes may have been running. Finally, knowledge of this information can be used for recovery, where automated data structure repair tools [1] use discovered nodes as a starting point. We have constructed a prototype of MemCrawler which can inspect the kernel memory of an active virtual machine session or a passive memory dump. MemCrawler only relies on access to kernel memory, thus making it suitable for analyzing on-disk kernel crash dumps and operation over live debugging interfaces such as JTAG or hardware PCI memory capture cards. Our prototype uses UMLinux, but the technique is generic enough to work with other virtual machines and emulators such as QEMU, Xen, VMWare or Bochs.

As an example, our system is currently able to reconstruct the process list of a running Linux system by scanning for struct task struct process data structures. The constraint template for a process structure ensures that the process: has a valid state of {-1, 0, 1}, a valid priority based on the Linux definition of MAX PRIO, and valid kernel pointers to structures of the correct type for the stack, process parent, process group, address mappings, open file list, and signal handler. Once we accurately identify these structures, we use information from the fields, such as the address mappings and open file information to infer the types of objects that are referenced by this structure. The Linux kernel organizes common data structures into an object allocator called the SLAB allocator. Each SLAB container has many pre-initialized objects of a particular type. Using knowledge of the SLAB allocator’s structure, we are able to identify and reason about a large number of kernel structures by knowing which slab to consult for a particular object type. For example, all process structures belong to the task struct slab area.

2. RELEVANCE We feel that this work is a good match for the audience at ASPLOS because it spans architecture and systems by addressing several major challenges in memory forensics and recovery. The architectural aspects of our work include the use of virtualization technology to provide low level visibility into the memory image of the system software executing at higher levels. This supports system level forensic analysis and information recovery.

3. NOVELTY 1. Automated identification of non-reachable or hidden memory structures using pattern matching and inferencing. 2. Uncovering evidence of malware activity by highlighting suspect memory structures that were discovered. 3. Improving system reliability by using discovered memory structures to guide the repair of corrupted data structures.

4. RELATED WORK Forensic analyzers for kernel memory [2, 3] often refer to known starting points gleaned from symbol files such as Linux’s System.map file to reconstruct the system’s semantic view of process structures, address spaces, network connections and open files. [4] performs simple pattern matching for Windows process structures. Our approach relies on matching data constraints similar to those used in data structure repair [1] without the use of any special markers. Our work also leverages object typing information inherent in the SLAB allocator rather than directly searching for binary object matches, which greatly reduces the risk of false positives and improves identification accuracy.

5. REFERENCES [1] Brian Demsky and Martin Rinard. Automatic Detection and Repair of Errors in Data Structures. In OOPSLA ’03: Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, pages 78–95, New York, NY, USA, 2003. ACM. [2] GMG Systems. KnTTools with KnTList. http://www. gmgsystemsinc.com/knttools/. [3] Mariusz Burdach. Finding Digital Evidence in Physical Memory. Black Hat, 2006. [4] Andreas Schuster. Searching for Processes and Threads in Microsoft Windows Memory Dumps. volume 3. The Proceedings of the 6th Annual Digital Forensics Research Workshop, 2006.