2. Nanofabrication Principles and Practice 17. Tadashi Kawazoe and Motoichi
Ohtsu. 2.1. Adiabatic Nanofabrication 17. 2.2. Nonadiabatic Nanofabrication 19.
Failed Back Surgery Syndrome 88. 8.3.7. Arthritis 89. 8.3.8. Fibromyalgia and Widespread Pain 90. 8.3.9. Neuropathic Pain 90. 8.3.10. Diabetic Neuropathy 91.
Chapter 8: Breaking Down the BMAT ................................................. 139. Chapter 9:
Studying BMAT Questions and Answers . ... Conventions Used in This Book .
Chapter 3: Picking the Right Clarinet for You . ..... Clarinet For Dummies x.
Mastering the Two P's: Practice and ... Exploring the Inner Workings of the
Clarinet .
Chapter 14: Air Force Officer Qualifying Test (AFOQT) Practice Test .........................
....................201. Chapter 15: ..... xii Military Flight Aptitude Tests For Dummies.
viii Nikon 1 J1/V1 For Dummies. Chapter 3: Taking Great Pictures, Automatically.
. . . . . . . . . . . . . . . . .57. Composing a Stronger Image .
Ulrich Krause. References 29. 3. Fire Risk Assessment 33. Javier GarcRa Torrent
and Enrique Querol. 3.1. Introduction 33. 3.2. Experimental Techniques 34.
Chapter 7: Expressing a State of Being with Ser and Estar. ..... Answer Key . .....
An unusual stem-changing -ar verb: Jugar in the present tense ....................... 89.
Chapter 13: Conveying Uncertainty about the Past with the Imperfect Subjunctive.
... Chapter 17: Mastering Subjectivity with the Irregular Present Subjunctive . ...
Part V: Getting Help with the Helping Verb Haber......................197 .... Answ
Chapter 9: Laying Out Your Web Pages with Joomla Templates ..................... ....
Joomla! For Dummies, 2nd Edition xiv. Chapter 2: Getting and Installing Joomla.
because you signed up for email updates via a link on the T2DKP home page. To unsubscribe ... Epigenomic data integrated
65 Mental health at work: psychological interventions. Adrian Neal ... This book
has a companion website available at: www.hodderplus.com/hunters.
In the case of Otto Preminger Institute v Austria, the Court 'could not disregard the fact that the Roman Catholic religion is the religion of the overwhelming ma-.
Chapter 6: automating Data proCessing with oozie. 167. Getting to Know Oozie.
168. Oozie Workflow. 170. Executing Asynchronous Activities in Oozie Workflow.
Contents
Introduction
xvii
AL
Chapter 1: Big Data and the Hadoop Ecosystem Hadoop: Meeting the Big Data Challenge Data Science in the Business World
RI
Big Data Meets Hadoop
1
2 3 5
MA
TE
The Hadoop Ecosystem 7 Hadoop Core Components 7 Hadoop Distributions 10 Developing Enterprise Applications with Hadoop 12 Summary 16
D
Chapter 2: Storing Data in Hadoop
19
HDFS 19
GH
TE
HDFS Architecture Using HDFS Files Hadoop-Specific File Types HDFS Federation and High Availability
20 24 26 32
HBase 34
PY
RI
HBase Architecture HBase Schema Design Programming for HBase New HBase Features
34 40 42 50
CO
Combining HDFS and HBase for Effective Data Storage 53 Using Apache Avro 53 Managing Metadata with HCatalog 58 Choosing an Appropriate Hadoop Data Organization for Your Applications 60 Summary 62 Chapter 3: Processing Your Data with MapReduce
Getting to Know MapReduce MapReduce Execution Pipeline Runtime Coordination and Task Management in MapReduce
63
63 65 68
CONTENTS
Your First MapReduce Application Building and Executing MapReduce Programs
Designing MapReduce Implementations Using MapReduce as a Framework for Parallel Processing Simple Data Processing with MapReduce Building Joins with MapReduce Building Iterative MapReduce Applications To MapReduce or Not to MapReduce? Common MapReduce Design Gotchas
Implementing InputFormat for Compute-Intensive Applications Implementing InputFormat to Control the Number of Maps Implementing InputFormat for Multiple HBase Tables
Reading Data Your Way with Custom RecordReaders Implementing a Queue-Based RecordReader Implementing RecordReader for XML Data
Organizing Output Data with Custom Output Formats Implementing OutputFormat for Splitting MapReduce Job’s Output into Multiple Directories
Writing Data Your Way with Custom RecordWriters
100 106 112
116 116 119
123 124
133
Implementing a RecordWriter to Produce Output tar Files
133
Optimizing Your MapReduce Execution with a Combiner Controlling Reducer Execution with Partitioners
135 139
Implementing a Custom Partitioner for One-to-Many Joins
140
Using Non-Java Code with Hadoop
143
Pipes 143 Hadoop Streaming 143 Using JNI 144
Summary 146 Chapter 5: Building Reliable MapReduce Apps
Unit Testing MapReduce Applications Testing Mappers Testing Reducers Integration Testing
Local Application Testing with Eclipse Using Logging for Hadoop Testing
xii
147
147 150 151 152
154 156
CONTENTS
Processing Applications Logs
160
Reporting Metrics with Job Counters 162 Defensive Programming in MapReduce 165 Summary 166 Chapter 6: Automating Data Processing with Oozie
Getting to Know Oozie Oozie Workflow
167
168 170
Executing Asynchronous Activities in Oozie Workflow Oozie Recovery Capabilities Oozie Workflow Job Life Cycle
173 179 180
Oozie Coordinator Oozie Bundle Oozie Parameterization with Expression Language
181 187 191
Workflow Functions Coordinator Functions Bundle Functions Other EL Functions
192 192 193 193
Oozie Job Execution Model 193 Accessing Oozie 197 Oozie SLA 199 Summary 203 Chapter 7: Using Oozie
205
Validating Information about Places Using Probes Designing Place Validation Based on Probes Designing Oozie Workflows Implementing Oozie Workflow Applications
206 207 208 211
Implementing the Data Preparation Workflow Implementing Attendance Index and Cluster Strands Workflows
212
Implementing Workflow Activities Populating the Execution Context from a java Action Using MapReduce Jobs in Oozie Workflows
Implementing Oozie Coordinator Applications Implementing Oozie Bundle Applications Deploying, Testing, and Executing Oozie Applications Deploying Oozie Applications Using the Oozie CLI for Execution of an Oozie Application Passing Arguments to Oozie Jobs
220
222 223 223
226 231 232 232 234 237
xiii
CONTENTS
Using the Oozie Console to Get Information about Oozie Applications 240 Getting to Know the Oozie Console Screens Getting Information about a Coordinator Job
Overall Implementation Approach A Machine Learning Model, Parameters, and Algorithm Defining a Workflow for an Iterative Process Dynamic Workflow Generation
257 261 262 265
Using the Oozie Java API 268 Using Uber Jars with Oozie Applications 272 Data Ingestion Conveyer 276 Summary 283 Chapter 9: Real-Time Hadoop
Real-Time Applications in the Real World Using HBase for Implementing Real-Time Applications Using HBase as a Picture Management System Using HBase as a Lucene Back End
Summary 365 Chapter 11: Running Hadoop Applications on AWS
Getting to Know AWS Options for Running Hadoop on AWS Custom Installation using EC2 Instances Elastic MapReduce Additional Considerations before Making Your Choice
Understanding the EMR-Hadoop Relationship EMR Architecture Using S3 Storage Maximizing Your Use of EMR Utilizing CloudWatch and Other AWS Components Accessing and Using EMR
Using AWS S3 Understanding the Use of Buckets Content Browsing with the Console Programmatically Accessing Files in S3 Using MapReduce to Upload Multiple Files to S3
Automating EMR Job Flow Creation and Job Execution Orchestrating Job Execution in EMR Using Oozie on an EMR Cluster AWS Simple Workflow AWS Data Pipeline
367
368 369 369 370 370
370 372 373 374 376 377
383 383 386 387 397
399 404 404 407 408
Summary 409 xv
CONTENTS
Chapter 12: Building Enterprise Security Solutions for Hadoop Implementations
What Hadoop Security Doesn’t Natively Provide for Enterprise Applications Data-Oriented Access Control Differential Privacy Encrypted Data at Rest Enterprise Security Integration
Approaches for Securing Enterprise Applications Using Hadoop Access Control Protection with Accumulo Encryption at Rest Network Isolation and Separation Approaches
416 416 417 419 419
419 420 430 430
Summary 434 Chapter 13: Hadoop’s Future
Simplifying MapReduce Programming with DSLs What Are DSLs? DSLs for Hadoop
