Enables stateless computing via Service Profiles. • Efficient scale: Same ....
Manual processes. • Complex handoffs .... X86 2-socket B200 M2. SPECjbb2005.
Cisco Data Centre
Silvo Lipovšek, SE Mostar, oktobar 2013
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
1
Scalability
Spanning-Tree
Active Paths POD Bandwidth
vPC
FabricPath
Single
Dual
16 Way
Up to 10 Tbps
Up to 20 Tbps
Up to 160 Tbps
Layer 2 Scalability Infrastructure Virtualization and Capacity © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
2
• Allow a single device to
use a port channel across two upstream switches
• Eliminate STP blocked
ports
Physical Topology
Virtual Port Channel
• Uses all available uplink
bandwidth • Dual-homed server operate in active-active mode • Provide fast convergence upon link/device failure
Logical Topology
L2 Si
Si
Non-vPC
vPC
Increased BW with vPC © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
3
Consolidation with FCoE • I/O Today:
Today
Parallel LAN/SAN Infrastructure LAN
SAN A
SAN B
Inefficient use SAN of Network LAN A Infrastructure
SAN B
5+ connections per server – higher adapter and cabling costs Adds downstream port costs; cap-ex and op-ex Each connection adds additional points of failure in the fabric Nexus 5000
Longer lead time for server provisioning Multiple fault domains – complex diagnostics Management complexity DCB and FCoE © 2010 Cisco and/or its affiliates. All rights reserved.
Ethernet
FC Cisco Confidential
4
10 GE uplinks 10 GE
10 GE ToR switch
ToR 10G Ethernet switch with high density simplifies transition from 1G to 10G, because it reduces the number of 10G cables to aggregation switches. Less cables, easier moving and adding the servers. Better cooling (cables obstruct the airflow).
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
5
Nexus 7000
Nexus 7000
Distribution Layer
MCEC
Nexus 5000
Nexus 5000 Nexus 2000 FEX
Rack Rack 11
x4
x4
Rack22 Rack
© 2010 Cisco and/or its affiliates. All rights reserved.
Access Layer
x4
Rack 1
x4
Rack Rack 122
Nexus 2000 FEX x4
x4
x4
Rack11 Rack
x4
Rack22 Rack
Rack 1
Rack12 2 Rack Cisco Confidential
6
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
7
Scalability Increased FEX, MAC, VLAN, Multicast scalability
Versatility Layer 3 Routing or Layer 2 Cisco FabricPath ready
Performance Highest density 10G switch with wire-speed performance and low latency
Flexibility Unified Ports gives users the flexibility to customize different personalities for each port
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
8
Performance Highest density line rate, low power per port 10G/40G Ethernet/FCoE switch
Versatility Wide range of deployment options High density FEX aggregator Large Scale Fabrics Leaf-Spine Architecture
Integrated Layer2 and Layer 3 feature set First 40G FCoE Switch !
Latency Low Latency at scale ~ 1 microsecond latency for all ports © 2010 Cisco and/or its affiliates. All rights reserved.
Flexibility Configurable port speeds with true 40GE flows or 4 x 10GE flows Modularity provides future expansion capabilities Cisco Confidential
9
Continuous Market Leadership ü 13,000 Customers since FCS
ü #1 Markets Share Data Center Ethernet ü # 1 Market Share FCoE SAN Switching ü 3000 Channel Partners ü 1000+ FabricPath Customers ü 10,000+ FEX Customers
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
10
VMotion
Problems: • VMotion may move VMs across physical ports—policy must follow • Impossible to view or apply policy to locally switched traffic • Cannot correlate traffic on physical links—from multiple VMs
VLAN 101
VN-Link: • Extends network to the VM • Consistent services • Coordinated, coherent management
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
11
VN-Link (Network View) Boundary of network visibility
• VN-Link provide visibility to the
individual VMs
VNIC
VNIC
Hypervisor
• Policy can be configured per-VM • Policy is mobile within the ESX
cluster • VN-Link refers to a literal link VETH
© 2010 Cisco and/or its affiliates. All rights reserved.
VETH
between a VM VNIC & a Cisco VNLink Switch
Cisco Confidential
12
Cascading of Fabric Extenders Virtualized Deployment
VM-FEX architecture LAN
LAN
Switch
FEX
Logical Switch
Logical Switch
Switch
Switch port extended over cascaded Fabric Extenders to the Virtual Machine
FEX
Hypervisor
Hypervisor vSwitch
VM-FEX
App
App
App
App
App
App
OS
OS
OS
OS
OS
OS
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
13
13
• 10G Ethernet • Unified Fabric, FCoE • FEX • Nexus 1000V • ---------• More RAM • Stateless computing • NIC virtualization
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
14
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
15
Server © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
16
Server
Server
Server
Server Server
Server
Server
Server
Server
Server Server
Server
Server
Server
Rack
Rack
© 2010 Cisco and/or its affiliates. All rights reserved.
Server Server
Cisco Confidential
17
• First generation
Server
Server
Server
Server
Server
Server
Server
Server
Server
Server
Rack-optimized Top of Rack or End of Row switches Cables
• Benefits Space utilization Highly flexible
• Weakness
Server
Server
Cabling Serviceability
Server
Server
Power efficiency
Rack
Rack
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
18
Server
Server
Server
Server
Server Server
Server Server
Server
Server
Server
Server
Server
Server
Server
Server Server
Server
Server
Rack
Rack
© 2010 Cisco and/or its affiliates. All rights reserved.
Server
Server
• Blade servers • Integrated switches • Fixed backplane
Benefits • Space utilization • Cable aggregation • Power efficient
Server
Server
Server
Server
Server
Server
Server
Server
Server
Second generation
Weakness • I/O flexibility • Aggregate management • Large chassis needed to amortize switch/mgmt costs
Cisco Confidential
19
Chassis Management Server
Server
Server
Server
Server
Server
Server
• New management layer
Benefits • Consistency in chassis • Shared chassis infrastructure monitoring
Management © 2010 Cisco and/or its affiliates. All rights reserved.
Server
Server
Server
Server
Server
Server
Server
Weakness • Additional mgmt overhead • Additional cost overhead • Need chassis aggregation management • Artificial aggregation point
Management Cisco Confidential
20
Application Deployment Server
Server
Server
Server
Server
Server
Server
• Single application per OS
Benefits • Isolation • Flexibility • Simplicity
Management © 2010 Cisco and/or its affiliates. All rights reserved.
Server
Server
Server
Server
Server
Server
Server
Weakness • Low utilization • Power & Cooling • Server sprawl
Management Cisco Confidential
21
Software Switch
Management © 2010 Cisco and/or its affiliates. All rights reserved.
Software Switch
Software Switch
Management Cisco Confidential
22
Software Switch
• Advantages Utilization Mobility Availability Familiarity © 2010 Cisco and/or its affiliates. All rights reserved.
Software Switch
Software Switch
• Disadvantages Distributed policies Security Diagnostics Performance Cisco Confidential
23
Infrastructure • Servers & Switches • Physical & Virtual
Challenges
Software Switch
Management © 2010 Cisco and/or its affiliates. All rights reserved.
• • • Software Switch • •
Many points of management Consistent policies Diagnostics Software Switch Training Security
Management Cisco Confidential
24
Mgmt Server
Over the past 10 years • • • •
An evolution of size, not thinking More servers & switches than ever More switches per server Management applied, not integrated
An accidental architecture • Still a 1980’s PC model
Result: Complexity
Management © 2010 Cisco and/or its affiliates. All rights reserved.
• More points of management • More difficult to maintain policy coherence • More difficult to secure Management • More difficult to scale Cisco Confidential
25
Mgmt Server
• Embed management
Mgmt Server
• Unify fabrics • Optimize virtualization • Remove unnecessary switches, adapters, management modules
• Less than 1/3rd the
support infrastructure for a given workload
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
26
Mgmt Server
• A single system that encompasses: Network: Unified fabric Compute: Industry standard x86 Virtualization optimized • Unified management model Dynamic resource provisioning • Efficient Scale Cisco network scale & services Fewer servers with more memory • Lower cost Fewer servers, switches, adapters, cables Lower power consumption Fewer points of management
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
27
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
28
• Single, scalable integrated system • Network + compute virtualization • Dynamic resource provisioning Mgmt
LAN
SAN A SAN B
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
29
Industry Standard innovations that help UCS B-Series Blade servers reduce TCO apply to UCS C-Series Rack-Mount Servers with additional benefits of a familiar rack form factor and built in migration path to Unified Computing.
Unified Fabric – Fabric Extenders
• Simplify I/O Infrastructure and Management • Reduce Support Infrastructure up to 50% – Nics, HBA’s, Chassis Interconnects, Cabling
Embedded – Unified Management
• Single, Highly Available, Point of Management • Reduce management tools, consoles, modules with full interoperability via XML API
Extended Memory Technology
• More Economical footprint for memory intensive workloads and higher consolidation ratios. • Large Dataset workloads on two socket servers.
VN Link – Virtual Interface Card
• I/O consolidation and increased cpu performance • Network policy control and transparency to the VM level.
Dynamic Provisioning Service Profiles
• Faster provisioning - reduced spares • Enables consistent infrastructure policies w/RBAC
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
30
UCS 5108 Blade Server Chassis UCS 6120XP 20 Port Fabric Interconnect
UCS 6140XP 40 Port Fabric Interconnect
I/O Adapters UCS 2104XP Fabric Extender
© 2010 Cisco and/or its affiliates. All rights reserved.
UCS B200-M1 2 Socket Server
UCS B250-M1 Extended Memory Server Cisco Confidential
31
Dynamic Ports Allocation: Lossless Ethernet or Fibre Channel
Unified Port
• Convert protocol support on the same port dynamically • All ports on UCS 6200 Series • 16-port Expansion Module for 6248UP
NATIVE FIBRE CHANNEL
© 2010 Cisco and/or its affiliates. All rights reserved.
LOSSLESS ETHERNET: 1/10GBE, FCOE, ISCSI, NAS
Cisco Confidential
32
2x 1 Link
2x 2 Link
2x 4 Link
2x 8 Links
20 Gbps per Chassis
40 Gbps per Chassis
80 Gbps per Chassis
160 Gbps per Chassis
§ Wire once for bandwidth, not connectivity § Policy-driven bandwidth allocation § Integrates as a single system into your data center © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
33
33
Fabric Port Operation Modes 8 links, Discrète slot 1 slot 2 slot 3 slot 4 slot 5 slot 6 slot 7 slot 8
2 2 0 8
6100/6200
Available bandwidth per blade – 20 Gb (10 Gb per side)
• Statically pinned to Individual fabric links • Deterministic path • No oversubscription, each blade gets 20 Gb © 2010 Cisco and/or its affiliates. All rights reserved.
8 links, Port-channel
2 2 0 8
6200
Available bandwidth per blade – up to 160 Gb (80 Gb per side)
• Statically pinned to Port-channel • Shared bandwidth, better bandwidth utilization.
Cisco Confidential
34
Direct Attach
LAN
FC STORAGE
LAN Switch
Direct Attach
SAN Switch
FCoE Storage
FC Storage
Appliance
1 Link © 2010 Cisco and/or its affiliates. All rights reserved.
2 Link
4 Link
8 Link Cisco Confidential
35
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
36
Base Progression o Host Interfaces: 256 PCIe devices v Can be either vNICs or vHBAs v For OS limits please visit UCS config limits page o Embedded CPU: 30% improvement over 1st Gen ASIC o PCIe Gen-2 x16: Host Bandwidth 64Gbps o Network Bandwidth: 2x40 Gb to single slot B-series blade server v Uses 4x10 EtherChannel, HW 40Gb Capable v vNICs/vHBAs NOT limited to 10Gb
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
37
VNIC
VM
VM
VM
VM
VM
VM
VM
Hypervisor
Hypervisor
Cisco VIC
Cisco VIC
Server
VM
Server
VETH
UCS 6200
vCenter
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
38
B22 M3
B200 M3
B230 M2
B420 M3
B440 M2
Slots
1
1
1
2
2
CPU
E5-2400
E5-2600
E7-2800
E5-4600
E7-4800
Cores
16
16
20
32
40
DIMMs
12
24
32
48
32
Max GB
(with 32GB, coming soon)
384GB
768GB
(with 32GB,)
512GB
(with 32GB, coming soon)
1TB
Disk
2 x 2.5”
2 x 2.5”
2 SSD
4 x 2.5”
4 x 2.5”
Raid
0/1
0/1
0/1
0/1/5/6
0/1/5/6
Integrated I/O
Dual 2 x 10Gb
Dual 2 x 10Gb
No
Dual 20Gb
No
Mezz
1
1
1
2
© 2010 Cisco and/or its affiliates. All rights reserved.
1.5TB
2
Cisco Confidential
39
Storage • Optional Disk usage • SAN settings • LUNs • Persistent Binding
• SAN settings • vSAN
• Firmware • Revisions
• Server
• Network
Identity (UUID)
Uplinks
Adapters
LAN settings
Number
vLAN
Type: FC, Ethernet
QoS
Identity
etc…
Characteristics
Firmware
Firmware Revisions
Revisions Configuration settings © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
40
Storage • Optional Disk usage • SAN settings • LUNs • Persistent Binding
• SAN settings • vSAN
• Firmware • Revisions
• Server
• Network
Identity (UUID)
Uplinks
Adapters
LAN settings
Number
vLAN
Type: FC, Ethernet
QoS
Identity
etc…
Characteristics
Firmware
Firmware Revisions
Revisions Configuration settings © 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
41
§ Server identity no longer has to be tied to physical server hardware
SAN
LAN
Profiles provide identity Seamless server mobility Stateless blades
§ Boot over network (LAN or SAN) Boot order and boot devices are part of the pre-defined logical server profile On-board disks can be used for temp, swap, etc.
Server Name: LS-A UUID: 56 4d cd 3f 59 5b 61… MAC : 08:00:69:02:01:FC WWN: 5080020000075740 Boot Order: SAN, LAN Chassis-1/Blade-5
§ Dynamic Provisioning Complete infrastructure repurposing Integrated with 3rd party tools
© 2010 Cisco and/or its affiliates. All rights reserved.
Chassis-9/Blade-2 Cisco Confidential
42
Automated configuration of bare metal server and it’s network connectivity
HW Traditional Managed as Individual Components UCS B-Series Servers
UCS Service Profile Unified Device Management BIOS Version, BIOS Settings, RAID controller settings, UUID, Server Selection (Explicit or Pool)
UCS Adapters
NIC Firmware version, MAC Addresses, VLANs, QoS Settings, HBA Firmware version, WWNs
UCS 2200 Series Fabric Extender
Fabric Extender is implicitly configured based on Server Slot and physical connectivity to Fabric Interconnect
UCS 6200 Series Fabric Interconnect © 2010 Cisco and/or its affiliates. All rights reserved.
Uplink port configuration, LAN Pinning, SAN Pinning, VLANs, VSANs, DCB Settings Cisco Confidential
43
Unified Computing System Manager
• • • •
Embedded device manager for family of UCS components Enables stateless computing via Service Profiles Efficient scale: Same effort for 1 to 320 blades APIs for integration with new and existing data center infrastructure
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
44
• Comprehensive XML API, standards-based interfaces • Bi-Directional access to physical & logical internals Self Serve portals
Management Tools
Audi6ng Tools UCS CLI
Direct
UCS GUI
3rd Party
Customer
XML API System Status Physical Inventory Logical Inventory
• Broad 3rd party integration support • Faster custom integration for customer use cases • Consistent data and views across ALL interfaces
45
© 2010 Cisco and/or its affiliates. All rights reserved.
Cisco Confidential
45
XML Documents… POST Request:
UCS
© 2010 Cisco and/or its affiliates. All rights reserved.
Response: