High-Speed Matching of Vulnerability Signatures

16 September 2008

High-Speed Matching of Vulnerability Signatures Nabil Schear * David R. Albrecht † Nikita Borisov † University of Illinois at Urbana-Champaign †

* Department of Computer Science Department of Electrical and Computer Engineering {nschear2, dalbrech, nikita}@illinois.edu

Exploit vs. Vulnerability Signatures • Exploit Signatures – Match a specific example of an exploit

+ fast to match - imprecise, false positives

• Vulnerability Signatures – Match the condition at which the program is vulnerable

+ exploit generic, very precise - expensive

2

Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

HTTP

A05 header attribute attribute attribute attribute

IPP

attribute extra data

3

Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

HTTP

A05 header attribute attribute attribute attribute attribute tag name_len extra data value_len

IPP

name value

4

Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

HTTP

A05 header attribute attribute attribute attribute attribute tag name_len extra data value_len

IPP

Buffer overflow: uint16 name_len used to copy name into 8KB buffer without checks

name value

5

Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

Exploit Signature HTTP

A05 header attribute attribute attribute attribute

IPP

alert tcp any any -> any 631 (content: “|EB 10 5B 4B 33 C9 66 B9 96 03…|”)

attribute 0xA190909090EB105B4B33C966B996 tag name_len 0380340BFDE2FAEB05E8EBFFFFFF extra data value_len value

Shell code stored in name field 6

Example – CUPS/IPP • Now split shell code across two HTTP chunks HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

HTTP Chunk 1

Chunk 2

920 E5 header attribute attribute 0xA190909090EB10 attribute tag name_len 5B4B33C966B99

60380340BFDE2FA value_len EB05E8EBFFFFFF

attribute

value

attribute

attribute extra data

7

Example – CUPS/IPP HTTP/1.1 200 OK Content-Type: ipp Transfer-Encoding: chunked

HTTP

Vulnerability Signature

A05 header attribute attribute attribute attribute attribute tag name_len extra data value_len

IPP

if(name_len > 8192) Exception!

name value

8

Motivation: Matching Performance Throughput (Mbits/s) of vulnerability matchers Protocol

binpac

hand-coded

CUPS/HTTP

5,414

20,340

DNS

71

2,647

IPP

809

7,601

WMF

610

14,013

• Hand-coded 3x to 37x faster! • Many vulnerabilities do not require full protocol parsing 9

Introducing VESPA • A vulnerability signature and protocol parsing architecture • Focus on performance – Hardware acceleration friendly design • Future work: Offload to FPGA, network processor

– Target use in NIC or switch • 1 Gbps+ • Low latency

10

Outline • Parsing Architecture Design – Text Protocols – Binary Protocols

• • • •

Vulnerability Specification Language Performance Evaluation Related Work Conclusions 11

VESPA Design • Couple protocol and vulnerability specifications – maximum parser optimization

• Design Principles – Fast matching primitives – Explicit State Management – Avoid parsing irrelevant message parts

• Basic Idea: Construct matching specs based on primitives and marry to state control functions 12

Protocol State • Core State – Example: HTTP Content-Length header – Define structure and semantics of the message • Always parse

13

Protocol State • Core State – Example: HTTP Content-Length header – Define structure and semantics of the message • Always parse

• Application State – Example: HTTP Accept-Charset header – Only relevant to the application • Skip by default

14

Text Protocols • Often use explicit field labeling – e.g., RCPT TO:

• multi-string matching primitive to flatten irrelevant protocol structure – e.g., search for “HTTP/1.”, “Content-Length:”, “Transfer-Encoding:”, “POST”, and “\r\n\r\n” simultaneously

• Use control logic to drive matching primitive 15

Binary Protocols • Field meaning based on position in message • Binary traversal primitive – – – –

Parses only core fields No full in-memory representation Parses vulnerability relevant fields when desired Implemented with binpac language

16

VESPA Language String Matcher Primitive Spec bool is_post = str_matcher “POST” handler handle_post() %{ is_post = true; }%

• Stores each var as a member of generated C++ class • Extraction function within %{…}%

Handler Spec handle_post() %{ if(is_post) deploy(content_length); }%

• Embedded C++ code • deploy(var) function to control match state • Check vulnerability predicates here 17

Binary Protocols VESPA uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default; handle_name() %{ if(name_len > 8192) // throw exception }%

• VESPA controls: – vulnerability state – predicate evaluation

18

Binary Protocols VESPA uint16 name_len = bin_matcher IPP.binpac:IPP_Attr_Data.name_len handler handle_name() default; handle_name() %{ if(name_len > 8192) // throw exception }%

binpac IPP specification

type IPP_Attr_Data = record { name_len: uint16; name: bytestring &length = name_len &transient; value_len: uint16; value: bytestring &length = value_len &transient; };

• binpac controls protocol binary traversal

19

Modifying binpac for Binary Traversal • Optimized binpac dynamic memory usage – Pre-allocate one of each object that could be parsed in one object – Remove STL vector storage for all array elements

20

Modifying binpac for Binary Traversal • Optimized binpac dynamic memory usage – Pre-allocate one of each object that could be parsed in one object – Remove STL vector storage for all array elements

• Use &pointer attribute to specify objects that must be dynamically created – e.g., DNS name pointers…

21

Evaluation • Focus on vulnerabilities difficult to match with exploit sigs • Tested raw vuln sig matcher/parser performance – Network reassembly and reporting stages studied elsewhere

• Test System – 2.6 GHz AMD Athlon64 – 4GB RAM – Ubuntu Linux 2.6.22-x86-64

22

Tested Vulnerabilities • HTTP/IPP – Negative Content-Length causes integer overflow – uint16 name_len used to store size of 8KB buffer

• DNS – Pointer cycle can cause denial of service

• WMF – Vulnerable feature: allows arbitrary abort procedure to execute malicious code

23

Memory Micro-benchmarks Calls to new/malloc per message

Bytes allocated per message

Protocol

binpac

traversal

Protocol

binpac

traversal

DNS

539

14

DNS

15,812

2,296

IPP

33

6

IPP

1,360

432

WMF

94

6

WMF

3,824

312

• 6x to 40x reduction in number of calls to new • IPP and WMF call new 6x for any file • DNS proportional to num of DNS pointers 24

Memory Micro-benchmarks Calls to new/malloc per message

Bytes allocated per message

Protocol

binpac

traversal

Protocol

binpac

traversal

DNS

539

14

DNS

15,812

2,296

IPP

33

6

IPP

1,360

432

WMF

94

6

WMF

3,824

312

• 6x to 40x reduction in number of calls to new • IPP and WMF call new 6x for any file • DNS proportional to num of DNS pointers 25

String Primitive Micro-benchmarks

• Multi-string matching dominates text performance • VESPA approximates performance of pattern based IDS for simple signatures 26

Parser Performance

• VESPA outperforms binpac by 3 to 5 times

27

Parser Performance

• VESPA DNS considerably faster than binpac – Recall, hand-coded 9x faster than VESPA (2.6 Gbits/s) – Room for improvement in binary traversal 28

Related Work • Pattern Matching – Wu-Manber, Aho-Corasik, flex, pcre, XFA, Protomatching

• Vulnerability Signatures – Shield, GAPA, binpac, NetShield, Prospector

• IDS/IPS – Snort, Bro, SafeCard

29

Conclusions • Key Insight: Vulnerability signatures often do not require full protocol parsing – Specialize protocol parser to signature matching

• Developed VESPA language and architecture – 3-5 times faster than binpac – Performance tied to speed of primitives • Able to hardware accelerate multi-string matching • Improved performance of binary traversal

• Vulnerability signatures can be matched at 1 Gbps+ – Suitable for server NICs, switches, inline IPS

30

Thank you!

Questions?

31