Introduction to computer security

Introduction to

Computer Security International Edition

Michael T. Goodrich Department of Computer Science University of California, Irvine

Roberto Tamassia Department of Computer Science Brown University

PEARSON

Boston

Columbus Indianapolis New York San Francisco Upper Saddle River Cape Town Dubai London Madrid Milan Munich Paris Montreal Toronto Mexico City Sao Paulo Sydney Hong Kong Seoul Singapore Taipei Tokyo

Amsterdam

Delhi

Contents

1

Introduction

1

1.1

2

Fundamental

1.1.2

Concepts Confidentiality, Integrity, Assurance, Authenticity,

1.1.3

Threats and Attacks

1.1.1

and and

Availability Anonymity

1.1.4 1.2

1.5

14 15

1.2.2

Digital Signatures

25

1.2.3

Attacks on

26

1.2.5

1.4

9

Security Principles Cryptographic Concepts 1.2.1 Encryption

1.2.4

1.3

3

19

19

Simple Cryptosystems Cryptographic Hash Functions Digital Certificates

29 31

Access Control Models

33

1.3.1

Access Control Matrices

33

1.3.2

Access Control Lists

34

1.3.3

Capabilities

36

1.3.4

Role-Based Access Control

37

Implementation and Usability Issues

39

1.4.1

Efficiency

1.4.2

Passwords

41

1.4.3

Social

43

1.4.4

Vulnerabilities from

and

Usability

Engineering Programming Errors

Exercises

39

44 46

xiii

xiv

Preface

2

53

Cryptography 2.1 Symmetric Cryptography

2.2

54

2.1.1

Attacks

55

2.1.2

57

2.1.3

Substitution Ciphers One-Time Pads

2.1.4

Pseudo-Random Number Generators

61

2.1.5

The Hill

Cipher and Transposition Ciphers Encryption Standard (AES) 2.1.7 Modes of Operation Public-Key Cryptography

63

2.1.6

The Advanced

65

2.2.1

Modular Arithmetic

72

2.2.2

The RSA Cryptosystem The Elgamal Cryptosystem

76

Key Exchange Cryptographic Hash Functions 2.3.1 Properties and Applications 2.3.2 Birthday Attacks Digital Signatures

81

2.2.3

59

68 72

79

2.2.4

2.3

2.4

2.4.1

2.4.2

83 83 85 87

The RSA Signature Scheme The Elgamal Signature Scheme

88 89

2.4.3 2.5

2.6 3

Using Hash Functions with Digital Signatures Details of AES and RSA Cryptography

90

2.5.1

Details for AES

91

2.5.2

Details for RSA

91

97

Exercises

105

Operating Systems Security 3.1 Operating Systems Concepts 3.1.1 The Kernel and Input/Output

3.2

112 113

3.1.2

Processes

114

3.1.3

The

119

3.1.4

Filesystem Memory Management

3.1.5

Virtual Machines

126

Process Security Inductive Trust from Start 3.2.1 3.2.2

3.3

111

Monitoring, Management,

122

128 to

Finish

and

128

Logging

130

Memory and Filesystem Security

134

3.3.1

Virtual

Memory Security

134

3.3.2

Password-Based Authentication

135

3.3.3

Access Control and Advanced File Permissions

3.3.4

File

3.3.5

Symbolic Links and Shortcuts

Descriptors

....

138 144 146

Preiace 3.4

3.5 4

Application Program Security 3.4.1 Compiling and Linking 3.4.2 Simple Buffer Overflow Attacks

147

3.4.3

Stack-Based Buffer Overflow

150

3.4.4

Heap-Based

157

Buffer Overflow Attacks

147 148

Exercises

160

Malicious Software

167

4.1

Insider Attacks

168

4.1.1

Backdoors

168

4.1.2

Logic

171

Bombs

4.1.3 4.2

Defenses Against Insider Attacks Computer Viruses

174

4.2.1

Virus Classification

176

4.2.2

Defenses

179

4.2.3 4.2.4 4.3

4.4

4.5

4.6 5

xv

Against Viruses Viruses

Encrypted Polymorphic and Metamorphic Viruses

175

180 181

Malware Attacks

182

4.3.1

Trojan Horses Computer Worms

182

4.3.2

4.3.3

Rootkits

189

4.3.4

Zero-Day

4.3.5

Botnets

Privacy-Invasive

184

Attacks

193 194

Software

196

4.4.1

Adware

196

4.4.2

Spyware

198

Countermeasures

202

4.5.1

Best Practices

202

4.5.2

The

4.5.3

The Malware Detection Arms Race

207

4.5.4

Economics of Malware

208

Impossibility of Detecting

All Malware

Exercises

205

209

Network

215

5.1

Security I Network Security Concepts 5.1.1 Network Topology 5.1.2 Internet Protocol Layers

216

5.1.3

221

5.2

Network

The Link

Security Issues

Layer

5.2.1

Ethernet

5.2.2

Media Access Control

5.2.3

ARP

Spoofing

216 217

223

223

(MAC) Addresses

226 227

xvi

Preface 5.3

The Network

Layer

230

5.3.1

IP

230

5.3.2

Internet Control Message Protocol

234

Spoofing Sniffing 5.4 The Transport Layer 5.3.3

IP

236

5.3.4

Packet

238

5.4.1

Transmission Control Protocol

240

5.4.2

User

244

5.4.3

240

(TCP) Datagram (UDP) Network Address Translation (NAT) Protocol

5.4.4

5.5

5.6 6

245

TCP Session Hijacking Denial-of-Service Attacks

247

5.5.1

ICMP Attacks

250

5.5.2

SYN Flood Attacks

252

5.5.3

TCP ACK Attack

Optimistic

5.5.4

Distributed Denial-of-Service

Exercises

250

254 255 256

Network

Security II Application Layer and DNS 6.1.1 A Sample of Application-Layer Protocols 6.1.2 The Domain Name System (DNS)

261

6.1

262

6.1.3

DNS Attacks

270

6.1.4

DNSSEC

277

6.2

6.3

The

6.5

279

6.2.1

Firewall Policies

280

6.2.2

Stateless and Stateful Firewalls

281

Tunneling

284

Secure Shell

(SSH)

6.3.2

IPsec

6.3.3

Virtual Private

285 286

Networking (VPN)

289

Intrusion Detection

291

6.4.1

Intrusion Detection Events

294

6.4.2

Rule-Based Intrusion Detection

297

6.4.3

Statistical Intrusion Detection

298

6.4.4

Port

300

6.4.5

Honeypots Networking Wireless Technologies

304

Wired Equivalent Privacy (WEP) Wi-Fi Protected Access (WPA)

307

Scanning

Wireless

305

6.5.1

306

6.5.2 6.5.3 6.6

263

Firewalls

6.3.1

6.4

262

Exercises

310 314

Preface 7

Browser 7.1

7.2

Security

320

7.1.1

HTTP and HTML

320

7.1.2

HTTPS

326

7.1.3

Dynamic

7.1.4

Sessions and Cookies

Attacks

8.3

8.4

334

Clients

339

Session Hijacking

339 341

7.2.3

Phishing Click-Jacking

7.2.4

Vulnerabilities in Media Content

344

7.2.5

Privacy Attacks Cross-Site Scripting (XSS) Cross-Site Request Forgery (CSRF) Defenses Against Client-Side Attacks

348

Attacks

on

343

Servers

349 356 358 360

7.3.1

Server-Side

Scripting

360

7.3.2

Server-Side

Script Inclusion Vulnerabilities

362

7.3.3

Databases and SQL

7.3.4

Denial-of-Service Attacks

370

7.3.5

Web Server

371

7.3.6

Defenses

372

Injection Attacks

Privileges Against Server-Side Attacks

Exercises

364

374

Physical Security 8.1 Physical Protections 8.2

331

7.2.2

7.2.8

8

on

Content

7.2.1

7.2.7

7.4

319

The World Wide Web

7.2.6

7.3

xvii

379 and Attacks

380

Locks and Safes

381

381

Technology

8.2.1

Lock

8.2.2

Attacks

8.2.3

The Mathematics of Lock

Authentication

on

Locks and Safes

Security

Technologies

386 392 395 395

8.3.1

Barcodes

8.3.2

Magnetic Stripe

8.3.3

Smart Cards

398

8.3.4

RFIDs

403

8.3.5

Biometrics

407

Direct Attacks

Cards

Against Computers

396

412

8.4.1

Environmental Attacks and Accidents

412

8.4.2

Eavesdropping

413

8.4.3

TEMPEST

418

8.4.4

Live CDs

420

8.4.5

Computer Forensics

420

xviii

Preface 8.5

8.6

Special-Purpose Machines

423

8.5.1

Automated Teller Machines

423

8.5.2

Voting Machines

425

Physical Intrusion Detection 8.6.1

8.7

9

427 427

Engineering

Exercises

428

Security Models and Practice

435

9.1

Policy, Models, and Trust

436

9.1.1

9.2

9.3

9.4

9.5

9.6

9.7

Security Policy Security Models

436

9.1.2 9.1.3

Trust

438

9.8

437

Management

Access-Control Models

440

9.2.1

The Bell-La Padula Model

440

9.2.2

Other Access-Control Models

444

9.2.3

Role-Based Access Control

446

Software

Vulnerability Assessment 9.3.1 Static and Dynamic Analysis 9.3.2 Exploit Development and Vulnerability Disclosure Security Standards and Evaluation 9.4.1 Orange Book and Common Criteria 9.4.2 Government Regulations and Standards Administration and Auditing

450 451 ...

454 456 456 458 460

9.5.1

System Administration

460

9.5.2

Network

463

Auditing and Penetration Testing

Kerberos

465

9.6.1

Kerberos Tickets and Servers

465

9.6.2

Kerberos Authentication

466 469

9.7.2

Storage Encryption Disk Encryption

9.7.3

Trusted Platform Module

472

Secure 9.7.1

10

Human Factors and Social

File

Exercises

469 471

474

Applications Security 10.1 Database Security

477 478

10.1.1 Tables and Queries

479

10.1.2

481

Updates

and the Two-Phase Commit Protocol

10.1.3 Database Access Control

483

10.1.4 Sensitive Data

487

10.2 Email

Security

10.2.1 How Email Works

490 490

xix

Preface 10.2.2

Encryption and Authentication

Spam Payment Systems 10.2.3

10.3

497 and Auctions

503

10.3.1 Credit Cards

503

Digital Cash

506

10.3.2

10.4

492

10.3.3 Online Auctions

508

Digital-Rights Management

509

10.4.1

Digital-Media Rights Techniques

510

10.4.2

Digital-Media Rights Practice

513

Licensing Schemes

515

10.4.3 Software

10.4.4

Legal

10.5 Exercises

Issues

517 518