How Many IOPS is Enough?

How Many IOPS is Enough? Episode II Tom Coughlin Coughlin Associate & Jim Handy Objective Analysis

© 2012 Coughlin Associates and Objective Analysis

Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors & Sources © 2012 Coughlin Associates and Objective Analysis

3

Our Survey • Survey is ongoing: TinyURL.com/IOPSsurvey • Very few questions: – – – –

IOPS Capacity Latency Applications

• Results to be published in: – SNIA SSSI white paper of top-level findings – Full in-depth report for sale


4

Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources 5

Application Types • Cloud Storage/ServiceVirtualization • Databases • On-Line Transaction Processing (OLTP)© • Video Creation and Distribution • Science & Engineering • Exchange Servers 6

Applications from Survey


7

Cloud Storage/Services-Virtualization Image courtesy of Waring Corp.

• The “IO Blender” – Many streams – Scrambled I/O – Highly random

• Suits SSDs better than HDDs for rapid access • Many VM and VDI systems use flash cache to meet demand speed needs 8


Databases • • • •

Large data sets Random traffic High I/O load Early SSD adopter – Previously used DRAM SSDs

• Some users load entire DB in flash 9


OLTP (On-Line Transaction Processing)

• Verified writes – Write/read back – Doubles I/O load

• No room for errors • Speed is imperative – Delays lose customers

10


Video Creation or Distribution • Large data sets • Multiple video streams – Randomizes access

• High bandwidth required • Expensive talent – Don’t want them sitting around waiting Image courtesy of the US Library of Congress

11


Recording Media Used in Professional Video Cameras


12

Science & Engineering Image courtesy of Wikimedia Commons

• Complex problems – – – –

Genome sequencing CAD/CAM Natural resources Nuclear modeling

• Large data sets • Expensive talent – Don’t want them sitting around waiting 13


Exchange Server • Multiple tasks – e-mail – Scheduling/calenda rs – Data storage

• Scads of users • e-mail chaos – Multiple mailboxes – Asynchronous sends & receives – Spam & virus filters • Strangely we didn’t get any of these users yet Image courtesy of Dell Computer 14


Outline • The Survey • Application Distribution and Attributes • More Survey Results: IOPS, Capacity and Latency • Developing tiers of storage for enterprise (and client) applications • Implications/Projections • Authors and Sources © 2012 Coughlin Associates and Objective Analysis

15

IOPS Needed by Application

83% between 1 K and 1 M IOPS


16

Range of Storage Capacity Needs

Possibly a Bi-Modal Distribution 17

IOPS Required vs. Storage Capacity


18

Maximum IOPS of System Hardware (Before another bottleneck becomes a problem)

91% between 1 K and 1 M IOPS

Good Match to Requirements © 2012 Coughlin Associates and Objective Analysis

19

Minimum Latency of System Hardware (Before another bottleneck becomes a problem)

37% at 10 msec latency

20

IOPS Required vs. Average Storage Latency


21


22

Today’s DRAMs: 6,000 Times HDDs’ Speed

From : HDDs and Flash Memory: A Marriage of Convenience, Coughlin Associates and Objective Analysis


23

107

Memory and Storage Price vs. Bandwidth

1.E+07

106

L1

1.E+06

L2

5 1.E+05

Bandwidth (MB/s)

10

L3

104

1.E+04

DRAM

103

1.E+03

NAND 2 1.E+02

10

101

1.E+01

1.E+00 0

101.E-01-1 10

HDD

Tape

0 1.E+00

3 4 1.E+03 1.E+04 1.E+05 1.E+06 10 10 10 105 106 Price per Gigabyte From Objective Analysis: Are Hybrid Drives Finally Coming of Age? 24 © 2012 Coughlin Associates and Objective Analysis

10

1 1.E+01

10

2 1.E+02

$/GB SSD vs. HDD HDD and Flash are on Parallel Paths $1,000.00 Flash HDD

$100.00 $10.00 $1.00 $0.10 $0.01 2004

2005

2006

2007

2008

2009

2010

Prices: HDD – Price G2, Flash, Objective Analysis 25


Market Expectations Price/GB Roughly Follows IOPS $10,000.00

$/GB

$1,000.00 $100.00 $10.00 $1.00

Lots of IOPs, Lower Capacity, Higher Cost

Lots of TB’s, Low Cost, Lower IOPs Storage

$0.10 1.E+01

1.E+02

1.E+03

1.E+04

IOPS

26

© 2012 Coughlin Associates and Objective Analysis © 2012 Coughlin Associates and Objective Analysis

1.E+05

1.E+06

Device IOPS by Form Factor

MB/s 27



28

Implications/Projections (1) • Increasing SSD adoption – More IOPS is better for most applications

• Fast storage will transform: – Was short-stroked HDDs – Migrating to SSDs

• New enterprise hierarchy – SSD front end – HDD back end 29


Implications/Projections (2) • Bottleneck is no longer storage – Network, software, servers…

• Users focus more attention on IOPS: – Stronger understanding than today – Higher IOPS drives more data/content • More content means more storage

– Result: widespread storage growth • More SSDs • More HDDs • Even more tape!!! 30


Participate in the Survey! • SSDs will be priced by speed and capacity – Users will benefit by knowing their speed & capacity needs – Sellers will too!

• Ongoing 5-minute survey at: TinyURL.com/IOPSsurvey • Initial results encouraging • We need more respondents! © 2012 Coughlin Associates and Objective Analysis

UPCOMING REPORT COMPILES IT MANAGER SURVEY RESULTS • Report to be published in late October

Objective-Analysis.com or TomCoughlin.com/techpapers.htm 408-978-8184 [email protected].


32

Your Presenters Tom Coughlin, President, Coughlin Associates is a highly-respected storage analyst and consultant with over 30 years in the data storage industry in Thomas Coughlin Coughlin Associates

engineering and management at high profile companies. Jim Handy is a widely recognized semiconductor analyst, has over 30 years in the electronics industry.

His background includes marketing and design positions at market-leading suppliers. Jim Handy Objective Analysis © 2012 Coughlin Associates and Objective Analysis

33

Source Documents • How Many IOPS is Enough? – Coughlin Associates & Objective Analysis • HDDs and Flash Memory: A Marriage of Convenience – Coughlin Associates & Objective Analysis • 2012 Digital Storage for Media and Entertainment – Coughlin Associates • Are Hybrid Drives Finally Coming of Age? – Objective Analysis • Two may be Better than One: Why HDD and Flash Belong Together – SNIA SSSI White Paper TomCoughlin.com/techpapers, Objective-Analysis.com


34