Best Practice Guidelines for Deploying EX8200 ... - Juniper Networks

IMPLEMENTATION GUIDE

Best Practice Guidelines for Deploying EX8200 Virtual Chassis Configurations

Although Juniper Networks has attempted to provide accurate information in this guide, Juniper Networks does not warrant or guarantee the accuracy of the information provided herein. Third party product descriptions and related technical details provided in this document are for information purposes only and such products are not supported by Juniper Networks. All information provided in this guide is provided “as is”, with all faults, and without warranty of any kind, either expressed or implied or statutory. Juniper Networks and its suppliers hereby disclaim all warranties related to this guide and the information contained herein, whether expressed or implied of statutory including, without limitation, those of merchantability, fitness for a particular purpose and noninfringement, or arising from a course of dealing, usage, or trade practice.

Copyright © 2013, Juniper Networks, Inc.

1

IMPLEMENTATION GUIDE - Best Practice Guidelines for Deploying EX8200 Virtual Chassis Configurations

Table of Contents Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Design Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 EX8200 Virtual Chassis Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 EX8200 Virtual Chassis Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Virtual Chassis Ports on the XRE200 External Routing Engine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Virtual Chassis Ports on EX8200 Member Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Comparison Between EX4200 and EX8200 Virtual Chassis Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 EX8200 Virtual Chassis High Availability and Resiliency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Hardware Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Control Plane Redundancy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Graceful Routing Engine Switchover. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Nonstop Active Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Data Plane Redundancy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Nonstop Software Upgrade (NSSU). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 The Makeup of an EX8200 Virtual Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 XRE200 External Routing Engines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 EX8200 Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Member ID and Interface Numbering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Network Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Virtual Chassis Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Connecting an XRE200 into an EX8200 Virtual Chassis Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Building Virtual Chassis Configurations over Long Distances . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 New EX8200 Virtual Chassis Configuration Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Upgrade All XRE200s and EX8200 Switches to the Same Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Prepare EX8200 Switches to be Part of a Virtual Chassis Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Create Preprovisioned Virtual Chassis Configuration on Master XRE200. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Connect the EX8200 Switches to the XRE200s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Interconnect the XRE200s. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Convert EX8200 Switch 10GbE Network Ports to Virtual Chassis Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Interconnect the EX8200 Switches via the Converted 10GbE Virtual Chassis Ports. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Virtual Chassis Show Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 Migrating EX8200 Standalone Switches to EX8200 Virtual Chassis Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Upgrade the Switches to Junos OS Version 11.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Revert the Access Switch Links Back to the First EX8208, Now in Virtual Chassis Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Attach the Second EX8208 to the Pair of XRE200s and Configure It in Virtual Chassis Mode . . . . . . . . . . . . . . . . . . . . . . 20 Restore the Access Switch Links Back to the Second EX8208, Now in Virtual Chassis Mode . . . . . . . . . . . . . . . . . . . . . . . 21 Disable VRRP on the EX8200 Switches, Now in Virtual Chassis Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2



EX8200 Virtual Chassis High Availability Best Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 EX8200 Virtual Chassis Failover Convergence Times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 EX8200 Virtual Chassis over Long Distances Configuration Steps. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Physical Connections of XRE200s and Intermediate Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 UFD Configuration on Intermediate Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 XRE200 Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 About Juniper Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

Table of Figures Figure 1: EX8200 Virtual Chassis configuration with XRE200 connecting to every member . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Figure 2: EX8200 four-member Virtual Chassis configuration with full mesh connection between every access switch and line card chassis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Figure 3: Two-member EX8200 Virtual Chassis with XRE200s connected to each other directly over a GbE interface.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 Figure 4: Two-member EX8200 Virtual Chassis over long distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Figure 5: Redundant pair of EX8200 running VRRP and RTG with all traffic flowing over master EX8200 switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Figure 6: All traffic flows through the backup EX8200 switch while master is being prepared to run EX8200 Virtual Chassis configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Figure 7: Single-member EX8200 Virtual Chassis is formed while all traffic flows through the backup EX8200 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Figure 8: All traffic is migrated from backup EX8200 to the single-member EX8200 Virtual Chassis . . . . . . . . . . . . . . . . . 19 Figure 9: All traffic flows through single-member EX8200 Virtual Chassis while backup EX8200 is configured to join EX8200 Virtual Chassis configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Figure 10: Traffic is load-balanced over both EX8200 switches migrated to two-member EX8200 Virtual Chassis . . . . 21 Figure 11: Logical network topology of EX8200 Virtual Chassis configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 12: Two-member EX8200 Virtual Chassis with dual-homed access devices.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Figure 13: A pair of two-member EX8200 Virtual Chassis configurations with dual-homed access devices interconnected via MPLS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Figure 14: Two XRE200s connected to each other via EX2200 over long distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25


3


Introduction With the explosion of video, mobile services, and other real-time applications, network users now impose more stringent requirements and have greater expectations about what networks can deliver. Modern enterprise networks must deploy robust, loop-free, multipath technologies to meet user demands and to satisfy the increased reliance on highly available network infrastructures, avoiding the high cost of downtime. Traditional LAN designs depend on Spanning Tree Protocol (STP) to prevent logical loops in switched networks with redundant links. In addition to being difficult to deploy, manage, and troubleshoot, technologies underutilize costly network capacity, driving up costs. Finally, running STP in a virtualized network with redundant switches requires compute-intensive protocols such as Virtual Router Redundancy Protocol (VRRP) on each switch, limiting the number of simultaneous logical connections that can be supported. Juniper Networks Virtual Chassis technology provides an innovative technique for building highly available and resilient Layer 2 networks without having to rely on protocols like STP or VRRP.

Scope The purpose of this guide is to provide network architects and engineers with best practice guidelines for designing, deploying, and configuring Juniper Networks® EX8200 line of Ethernet switches with Virtual Chassis technology. The guide is divided into three parts: EX8200 Virtual Chassis architectures at the network operational level; deploying Virtual Chassis technology using hardware components and connections; and migration, failover, and nonstop software upgrade (NSSU) scenarios.

Terminology • XRE200: Juniper Networks XRE200 External Routing Engine. • LCC: Line card chassis (used in this document to describe an EX8200 member chassis) • SRE: Switch Fabric and Routing Engine • VCP: Virtual Chassis port • Virtual Chassis Control Protocol • FPC: Flexible PIC Card (i.e., line card module) • Network ports: Non-Virtual Chassis ports that carry only data traffic • PFE: Packet Forwarding Engine

Design Considerations EX8200 Virtual Chassis Configurations Virtual Chassis technology was first introduced on the Juniper Networks EX4200 line of Ethernet switches. Virtual Chassis technology allows up to 10 interconnected EX4200 switches to operate as a single, unified, high bandwidth device. These interconnections can be made using any combination of dedicated high-speed Virtual Chassis ports (VCPs) on the switch’s rear panel or front panel gigabit Ethernet (GbE) or 10GbE fiber links. EX8200 Virtual Chassis technology enables up to four EX8200 chassis to be interconnected as a single logical device. The EX8200 Virtual Chassis architecture consists of redundant external Routing Engines, the XRE200 External Routing Engine, capable of managing up to four chassis connected using 1GbE or 10GbE VCPs and operating as a single chassis. Unlike other virtual system technologies, EX8200 Virtual Chassis technology separates the controller of the virtual system from the chassis Routing Engine. The XRE200s connect to the EX8200 chassis via the 1GbE out-of-band management ports on the Routing Engine modules installed in the modular switch, forming a single Virtual Chassis configuration as shown in Figure 1. These interconnections, known as dedicated VCP links, constitute the control plane connection and do not carry data traffic.

4



Active XRE200

Backup XRE200

EX8200 Virtual Chassis Member Switch




Active XRE200 to internal RE connection (1GbE) - copper Backup XRE to internal RE connection (1GbE) - copper 10GbE line rate intra-Virtual Chassis connection - fiber Inter XRE200 connection - fiber

Figure 1: EX8200 Virtual Chassis configuration with XRE200 connecting to every member Figure 1 shows a fully meshed two-member Virtual Chassis configuration with two XRE200s and a single 10GbE Link Aggregation Group (LAG) between LCCs. In an EX8200 Virtual Chassis configuration, each EX8200 chassis becomes an LCC and are interconnected through EX8200-8XS line cards using either a 10GbE link or a LAG bundle with up to twelve 10GbE line-rate links. This connectivity serves two functions: to allow data traffic between LCCs for single homed access devices; and to pass control traffic between the EX8200 chassis in case of the failure of all dedicated VCP links. Since the EX82008XS line cards use small form-factor pluggable transceivers (SFP+) that can support connections up to 40 km in distance, EX8200-based Virtual Chassis configurations can, for instance, span a large metropolitan area. If the Virtual Chassis members are located in the same or adjacent racks, low-cost direct attach cables (DACs) can be used as the interconnect mechanism. Members of an EX8200 Virtual Chassis configuration can include a mix of the Juniper Networks EX8208 Ethernet Switch (eight-slot) and EX8216 Ethernet Switch (16-slot).

EX8200 Virtual Chassis Ports A Virtual Chassis port (VCP) is any port that is capable of sending and receiving Virtual Chassis Control Protocol traffic to create, monitor, and maintain the Virtual Chassis configuration. There are three types of VCPs on the EX8200—Inter-XRE200, XRE-LCC, and intra-Virtual Chassis. The Inter-XRE200 and XRE-LCC are called “dedicated” VCPs as they carry control traffic, while intra-Virtual Chassis ports carry data traffic between LCCs. In some cases, intra-Virtual Chassis ports may carry data as well as control traffic.

Virtual Chassis Ports on the XRE200 External Routing Engine All GbE ports on the XRE200 Virtual Chassis Control Interface (VCCI) modules are VCPs. Any of the VCPs can be used to connect EX8200 switches to the XRE200 to form a Virtual Chassis configuration and also to connect XRE200s together to provide redundancy within the Virtual Chassis configuration. Any link connecting an XRE200 to an EX8200 switch or to another XRE200, therefore, is a VCP link. No user configuration is required to configure these VCP links. All VCP links on the XRE200 only carry Virtual Chassis Control Protocol traffic.

Virtual Chassis Ports on EX8200 Member Chassis An EX8200 switch in standalone mode has no VCPs. When a standalone EX8200 switch is enabled to function as a Virtual Chassis switch, the management ports on the switch’s Routing Engines are converted into dedicated XRE-LCC


5


Virtual Chassis ports that carry the Virtual Chassis Control Protocol traffic over the XRE-LCC VCP links. No further configuration is required to configure these VCP links. Lastly, the intra-Virtual Chassis ports, which can only reside on the EX8200 switches, can only be configured on the 10GbE EX8200-8XS line cards. VCPs on the 10GbE EX8200-8XS line card are enabled in pairs, i.e., ports that reside on the same Packet Forwarding Engine (PFE). The EX8200-8XS line card offers eight ports—0 through 7—with two contiguous ports residing on the same PFE. If port 0 is enabled as a VCP, Junos OS will automatically enable port 1 as a VCP. Intra-Virtual Chassis links between member switches in Virtual Chassis configuration are automatically configured to form a single LAG; no further user configuration is required. It is possible to configure up to 12 Ethernet ports as VCPs to form a LAG between member switches in a Virtual Chassis configuration. For highest availability, it is recommended to have a two-member LAG at a minimum. However, a four-member LAG with two pairs of port members residing on different line cards is preferred.

Comparison Between EX4200 and EX8200 Virtual Chassis Configurations The EX4200 switch has inherent Routing Engine support for up to 10 switches in a Virtual Chassis configuration, while the EX8200 uses two scalable external routing engines—the XRE200s—that allow four EX8200 chassis to form a Virtual Chassis configuration. One of the XRE200s is a hot-standby backup that takes over in case of failure on the active XRE200. With the EX8200, the dedicated Inter-XRE200 and XRE-LCC VCPs carry control traffic only while the intra-VCPs carry data traffic between LCCs. The intra-VCPs, which are also referred to as VCP-extension (VCPe) ports, carry control traffic only in the event of a dedicated VCP failure. With the EX4200, the same VCP or VCPe ports carry both data and control traffic. Note that EX4200 Virtual Chassis configurations can be built using dedicated high-speed (128 Gbps) VCPs on the switch’s rear panel, or by using front panel GbE or 10GbE fiber links. In an EX8200 Virtual Chassis configuration, mastership is limited to XRE200 members, as the mastership priority setting on the EX8200 chassis themselves is fixed to 0, making them ineligible for mastership election. In an EX4200 Virtual Chassis configuration, any of the members are eligible for mastership. To reduce the amount of traffic between LCCs over the VCP links, Virtual Chassis technology on the EX8200 switch employs an intelligent chassis, local, load-balancing model, which is not present on the EX4200. Figure 2 depicts a fourmember EX8200 Virtual Chassis configuration with full mesh connectivity between all LCCs and access switches. If the source and destination reside on Access Switch 1 and Access Switch 4, the traffic between them will be directly switched via a single node—say Node 4 of the EX8200 Virtual Chassis configuration. The traffic never crosses multiple nodes.

Access Switch 1

Access Switch 2

Access Switch 3

Access Switch 4

Node 1

Node 2

Node 3

Node 4

EX8200 Virtual Chassis Traffic flow between Access Switch 1 and Switch 2 is switched locally

Figure 2: EX8200 four-member Virtual Chassis configuration with full mesh connection between every access switch and line card chassis.

6



Table 1 lists a side-by-side comparison between EX4200 / EX4500 and EX8200 switches with Virtual Chassis technology.

Table 1: EX4200 / EX4500 vs. EX8200 Virtual Chassis Configuration Side-by-Side Comparison Category

EX4200 / EX4500 Virtual Chassis

EX8200 Virtual Chassis

Native Virtual Chassis support

Inherent support for Virtual Chassis by the Forwarding Engine.

Virtual Chassis emulated using XRE200.

Virtual Chassis ports

Fixed Virtual Chassis ports.

Any 10GbE port on 8XS line card can be a Virtual Chassis port.

Virtual Chassis mastership

Any member is eligible to be a master or backup. Routing Engine has an election mechanism to choose master and backup.

Master and backup is fixed to XRE200s.

Virtual Chassis management

All members are managed from the master member.

All members are managed from master XRE200.

Link Aggregation Group

LAG load balancing is hash based.

LAG load balancing is chassis-local.

Virtual Chassis path calculation

Every PFE is a node in the Virtual Chassis topology.

Every chassis is a node in the Virtual Chassis topology.

EX8200 Virtual Chassis High Availability and Resiliency Hardware Redundancy Any network design that is to provide high availability must be built on a solid foundation that offers stability, resiliency, and redundancy. The hardware design of the EX8200 Virtual Chassis configuration separates control and data planes through the use of separate hardware components—Routing Engines for control functions and Flexible PIC Cards (FPCs) for the data traffic. Dual External Routing Engines (XRE200s): In EX8200 Virtual Chassis configurations, Routing Engine functionality is externalized in a special, purpose-built, server-class appliance called the XRE200. With its 2.1 GHz dual-core CPU, 4 GB DRAM, 160 GB RAID hard disk, and dual redundant power supplies, the XRE200 supports control plane processing requirements for large-scale systems such as EX8200 Virtual Chassis configurations, and also provides an extra layer of availability and redundancy. Two XRE200s, running in active/hot standby mode, are required in an EX8200 Virtual Chassis configuration to provide for data, control, and management plane redundancy. To achieve high availability (HA) and resiliency goals, the EX8200 Virtual Chassis must connect in a fully meshed topology. The master XRE200 plays the role of the protocol master and takes care of interface creation and management. All control protocols such as OSPF, Internet Group Management Protocol (IGMP), Link Aggregation Control Protocol (LACP), 802.3ah, Virtual Chassis Control Protocol, etc., as well as all management plane functions, run or reside on the master XRE200. Junos OS control plane HA features such as graceful Routing Engine switchover (GRES), nonstop active routing (NSR), and nonstop bridging (NSB) available in Junos OS 11.3, are enabled on both XRE200s. In the event of an active XRE200 failure, the standby XRE200 takes over and Junos OS HA features ensure that the state of the Virtual Chassis, L2/L3 protocols, and forwarding information are not lost. Dual Switch Fabric and Routing Engines (SREs): Redundant Switch Fabric and Routing Engines (SREs) are supported on the EX8200 switches in both the standalone and Virtual Chassis configuration modes. One Routing Engine functions as the master, while the other is a hot standby should the master fail. In the EX8200 Virtual Chassis configuration, dual Routing Engines are installed in each of the LCCs. The master Routing Engine in the LCCs provides chassis control operations such as FPC power budgeting, power supply unit (PSU) management, line card management, and environmental monitoring and control. It also provides network port connectivity and data plane functionality while maintaining the PFE forwarding tables. Lastly, it acts as a conduit or a relay agent for communications between the master XRE200 and the FPCs. The Junos OS GRES feature is enabled by default on the SREs in the EX8200 Virtual Chassis configuration, so that the backup SRE stays in sync with the master Routing Engine in terms of line card states, PFE forwarding tables, and so forth. If the master becomes unavailable, the backup SRE takes over the functions that the master SRE performs.


7


Control Plane Redundancy Graceful Routing Engine Switchover In dual XRE200 and SRE configurations, the GRES feature leverages the Junos OS software’s separation of control and data plane functionality to provide continuous packet forwarding even when one Routing Engine fails. GRES preserves interface and kernel information so that traffic is not interrupted. However, GRES alone does not preserve the control plane. Even though the data traffic continues to flow through the device (switch or router) during and after switchover between Routing Engines, the protocol timers expire, the neighbor relationship between routers is dropped, and traffic is stopped at the neighbor device. Neighboring devices detect that the EX8200 Virtual Chassis switch has experienced a restart and react to the event in a manner prescribed by individual protocol specifications. To preserve all control plane functions during an XRE200 switchover, GRES must be combined with either graceful restart protocol or nonstop active routing (NSR) and nonstop bridging (NSB). Any updates to the master XRE200 are replicated to the backup XRE200 as soon as they occur. If the kernel on the master XRE200 stops operating, the master XRE200 experiences a hardware failure, or the administrator initiates a manual switchover, mastership switches to the backup XRE200.

Nonstop Active Routing NSR enables a routing device with redundant Routing Engines to switch from a primary Routing Engine to a backup Routing Engine without alerting peer nodes/neighbors that a change has occurred. NSR uses the same infrastructure as GRES to preserve interface and kernel information. NSR preserves routing information and protocol sessions by running the routing protocol process on both Routing Engines, so tracing options are replicated on the backup Routing Engine as well. In addition, NSR preserves TCP connections maintained in the kernel. Stateful replication of routing table adjacency information on standby Routing Engines uses routing protocol messages such as routing updates, hello messages, and adjacency states so the standby can immediately take up adjacencies without the disruption of protocol adjacencies. Therefore, the switchover is transparent to neighbors. Configuration of NSR requires that GRES be enabled on the XRE200, with the default setting disabled. With Junos OS 11.3, NSR supports the following routing protocols: • RIP, OSPF, IS-IS, BGP, IGMP and Bidirectional Forwarding Detection (BFD) (for all IPv4 unicast protocols) • OSPFv3 and RIPng IGMP snooping, Physical Interface Module (PIM), and Multicast Listener Discovery (MLD) will have NSR capabilities at a later time. These features can still be configured when NSR is enabled but will be excluded from NSR.

Data Plane Redundancy Multi-LCC LAGs: Link aggregation on Ethernet networks is a simple yet effective way to address bandwidth limitations and lack of resilience. EX8200 Virtual Chassis technology supports combining up to 12 individual interfaces as a single bundle known as a Link Aggregation Group (LAG). Connecting access switches to the EX8200 Virtual Chassis configuration over LAGs whose members are connected to different LCC members ensures a redundant multipath for the data plane. All links are active at the same time and traffic is load-balanced across them using the intelligent chassis local load-balancing model explained earlier in this guide. This reduces the amount of traffic between LCCs over the VCP links while ensuring resiliency.

Nonstop Software Upgrade (NSSU) Nonstop software upgrade (NSSU) provides a mechanism for upgrading Junos OS on Juniper Networks EX Series Ethernet Switches (in standalone mode as well as Virtual Chassis mode) using a single command-line interface (CLI) command with minimal traffic disruption. NSSU is available on EX8200 switches with redundant Routing Engines and on EX8200 Virtual Chassis configurations with redundant XRE200 External Routing Engines. NSSU leverages the underlying HA features such as GRES and NSR to enable upgrading the Junos OS version running on a switch or in a Virtual Chassis configuration with no disruption to the control plane and sub-second interruption to the data plane. In addition, NSSU upgrades line cards one at a time, permitting traffic to continue to flow through the line cards that are not being upgraded. By configuring LAGs such that the member links reside on different line cards, it is possible to achieve sub-second traffic disruption when performing an NSSU.

8



The Makeup of an EX8200 Virtual Chassis As mentioned in the previous sections, the EX8200 Virtual Chassis configurations are formed using two XRE200 External Routing Engines and up to four EX8200 chassis with dual SREs.

XRE200 External Routing Engines The function of each hardware device in a Virtual Chassis configuration is determined by that device’s role. The master role in an EX8200 Virtual Chassis configuration is assigned to an XRE200 External Routing Engine only. One of the XRE200s will take the role of Junos OS master Routing Engine and the other one will act as the Junos OS backup Routing Engine. The functions of the XRE200s include: • Master XRE200 controls most Routing Engine functions for all switches in the Virtual Chassis configuration. • Master XRE200 provides a single point for viewing and managing all functionality for all devices in the Virtual Chassis configuration. • Backup XRE200 maintains a state of readiness to take over the master role if the master fails.

EX8200 Chassis All EX8200 switches in an EX8200 Virtual Chassis configuration are assigned a line card role. Each switch is referred to as a line card chassis, or LCC. The LCCs are excluded from running the chassis control protocols; their primary function is to forward data traffic. The only role available for the switches in an EX8200 Virtual Chassis configuration is a line card role.

Member ID and Interface Numbering An EX8200 Virtual Chassis configuration contains member IDs 0 through 9. Member IDs 0 through 7 must be assigned to EX8200 LCCs only. Member IDs 8 and 9 must be assigned to XRE200 External Routing Engines only. Table 2 summarizes the EX8200 Virtual Chassis member IDs and roles.

Table 2: EX8200 Virtual Chassis Member IDs and Roles Device

Role

Member IDs

EX8208 or EX8216 switch

Line card

0-7

XRE200

Master or backup Routing Engine

8-9

Network Ports Interface numbering of the network ports on the EX8200 Virtual Chassis configuration follows the standard Junos OS interface nomenclature: type-//, for example xe-0/0/0. The FPC numbers are based on the member ID of the respective switch member and can be noncontiguous in a twomember Virtual Chassis configuration. The value for PIC is always 0. Table 3 shows the FPC and interface numbering for EX8216 or EX8208 members for EX8200-8-XS line card modules.

Table 3: EX8216 or EX8208 FPC and Interface Numbering Member ID

FPC Numbering

Network Port Interface Range

0

0 through 15 for EX8216

xe-0/0/0 to xe-15/0/7


xe-0/0/0 to xe-7/0/7


xe-16/0/0 to xe-31/0/7


xe-16/0/0 to xe-23/0/7


xe-32/0/0 to xe-47/0/7


xe-32/0/0 to xe-39/0/7


xe-48/0/0 to xe-63/0/7


xe-48/0/0 to xe-55/0/7

1

2

3


9


Virtual Chassis Ports Interface numbering of the VCPs on the EX8200 Virtual Chassis configuration varies with the type of the VCPs and the member switches on which the ports reside. When an EX8200 switch in standalone mode is enabled to function as a Virtual Chassis member, the management ports on the SREs installed in the switch are converted into the dedicated XRE-LCC VCPs. All VCPs on the SRE follow the nomenclature vcp-/, for example vcp-0/0 for a port that resides on SRE0. Intra-Virtual Chassis ports that reside on EX8200-8XS line cards follow nomenclature vcp-//, for example vcp-2/0/0.

Connecting an XRE200 into an EX8200 Virtual Chassis Configuration The GbE interfaces on the active XRE200 (up to eight) can be used to connect to the active Routing Engines in each of the EX8200 chassis participating in the Virtual Chassis configuration. Similarly, the GbE interfaces on the standby XRE200 (again, up to eight) can be used to connect to the standby Routing Engines in each of the EX8200 chassis in the Virtual Chassis configuration. The two XRE200s can also be connected to each other directly over any available GbE interface (Figure 3).

Active XRE200

Standby XRE200

EX8200 Virtual Chassis Switch


EX Series (10GbE)

EX Series (10GbE)

Intra-XRE connection for HA (1GbE) Active XRE to Internal Routing Engine connection (1GbE) Standby XRE to Internal Routing Engine connection (1GbE) 10GbE LAG intra-Virtual Chassis connection 10GbE links Traffic flow from access to access and access to core/WAN

Figure 3: Two-member EX8200 Virtual Chassis with XRE200s connected to each other directly over a GbE interface.

Building Virtual Chassis Configurations over Long Distances In large campus or data center environments where the distance between XRE200s and the EX8200 chassis exceeds the maximum reach of a Cat5 or Cat6 cable, dedicated low-end Layer 2 switches such as the Juniper Networks EX2200 Ethernet Switch can be deployed in each location to act as media converters, allowing the XRE200s to be connected over fiber links. At a later time, the XRE200s will have SFP-based interface modules that will eliminate the need for media converters. Currently, a port pair consisting of an RJ-45 and an SFP port on each L2 switch is required for every long-distance connection. All pair ports dedicated to supporting a long-distance connection are assigned to the same static VLAN. This simple configuration enables users to easily deploy EX8200 Virtual Chassis configurations in a wide variety of environments.

10



The intermediate switches must be configured in L2 mode only with Jumbo frame support enabled and no STP or L3 protocols running. All ports that form the VCP connections must belong to the same VLAN on the intermediate switches. It is recommended that the Unidirectional Failure Detection (UFD) protocol be run on the intermediate switches to detect any inter-switch link failures and disable the VCPs to ensure fast convergence times. Also, IEEE 802.3ah must be configured on the intermediate switches to allow fiber cut detection.

Active XRE200

1GbE BASE-T

L2 switch A 1GbE BASE-T

1GbE BASE-T

Standby XRE200

L2 switch B

1GbE BASE-T


1GbE BASE-T

1GbE BASE-T


Intra-XRE connection for HA (1GbE) Active XRE to Internal Routing Engine connection (1GbE) Standby XRE to Internal Routing Engine connection (1GbE) 10GbE LAG intra-Virtual Chassis connection

Figure 4: Two-member EX8200 Virtual Chassis over long distance The detailed steps required to set up the XRE200s and the intermediate switches for this deployment are provided later in the “EX8200 Virtual Chassis over Long Distances Configuration Steps” section.

New EX8200 Virtual Chassis Configuration Steps The creation of a full mesh two-member EX8200 Virtual Chassis configuration involves the following steps. Please refer to Figure 1 for the physical topology network diagram. 1. Upgrade all XRE200s and EX8200 switches to the same version 2. Prepare the EX8200 to be part of a Virtual Chassis configuration 3. Create preprovisioned Virtual Chassis configuration on the master XRE200 4. Connect the EX8200 switches to the XRE200s 5. Interconnect the XRE200s 6. Convert the EX8200 10GbE network ports to Virtual Chassis ports 7. Interconnect the EX8200 switches via the converted 10GbE VCPs

Upgrade All XRE200s and EX8200 Switches to the Same Version To form an EX8200 Virtual Chassis configuration, all XRE200s and the EX8200 LCCs must be upgraded to the same Junos OS version. An EX8200 two-member Virtual Chassis configuration is supported on Junos OS 10.4 or later. As of this writing, it is recommended to upgrade both the EX8200 LCCs and the XRE200s to Junos OS 11.1. If running Junos OS 10.4, the NSSU on the EX8200 is also available. Please refer to the Junos OS 10.4 release notes to confirm if NSSU is supported without any caveats or bugs. Alternately, the “standard” (non-NSSU) method of upgrading must be performed. This may impact network traffic when switching over SREs during the upgrade process.


11


The upgrade steps are as follows: 1. Disable GRES and NSR: deactivate chassis redundancy graceful deactivate routing-options nonstop 2. Upgrade backup SRE (assuming that Routing Engine 1 is acting as backup: request system software add re1 no-validate reboot 3. From CLI request Routing Engine master switch: request chassis routing-engine master switch 4. Upgrade old master Routing Engine 5. Re-enable GRES and NSR For more details on upgrading EX8200 switches, please refer to the appropriate technical documentation.

Prepare EX8200 Switches to be Part of a Virtual Chassis Configuration Use the following command to convert all EX8200s to Virtual Chassis mode: root> set chassis virtual-chassis This will reboot the RE(s) Do you want to continue ? [yes, no] yes

Create Preprovisioned Virtual Chassis Configuration on Master XRE200 A preprovisioned configuration must be used to configure an EX8200 Virtual Chassis deployment. Non-provisioned configurations are not supported. 1. Log into the master external Routing Engine and specify the preprovisioned configuration mode: [edit virtual-chassis] root@external-routing-engine# set preprovisioned 2. Specify all members for the Virtual Chassis, listing each switch’s or external Routing Engine’s serial number, member ID, and role: [edit virtual-chassis] root@external-routing-engine# set member member-id serial-number serial-number role role For the EX8200 line card chassis, the “Backplane” serial number displayed in the output of “show chassis hardware” (not shown here) must be used.

12

[edit virtual-chassis] root@external-routing-engine# root@external-routing-engine# root@external-routing-engine# root@external-routing-engine# root@external-routing-engine# root@external-routing-engine#

set set set set set set

member member member member member member

8 9 0 1 2 3

serial-number serial-number serial-number serial-number serial-number serial-number

062010000001 062010000005 BT0908500271 BT0908500274 xxxxxxxxxxxx xxxxxxxxxxxx



Connect the EX8200 Switches to the XRE200s Connect the XRE200-LCC VCPs to build the links that will carry control or Virtual Chassis Control Protocol traffic between the XRE200s and the line card chassis SREs. 1. Connect the management port of master SRE of each line card chassis to the Virtual Chassis ports on the master XRE200 2. Connect the management port of backup SRE of each line card chassis to the Virtual Chassis ports on the backup XRE200

Interconnect the XRE200s Connect the Inter-XRE200 VCPs to allow synchronization and control between XRE200 traffic to traverse directly instead of using XRE200-LCC VCPs. 1. Connect one of the VCPs on the master SRE to one of the VCPs on the backup XRE200.

Convert EX8200 Switch 10GbE Network Ports to Virtual Chassis Ports With Junos OS 10.4, only line-rate 10GbE ports on the EX8200-8XS line card support VCPs. The 10GbE network ports are converted to VCPs in pairs; for example, interfaces xe-0/0/0 and xe-0/0/1 are both converted to VCPs even though the CLI request is to convert only one of them. From the master XRE200 operational mode, configure VCPs for both EX8200 Virtual Chassis members. root@ external-routing-engine > request virtual-chassis vc-port set fpc-slot 0 pic-slot 0 port 0 member 0 This will also convert FPC slot 0 PIC slot 0 Port 1 to virtual chassis port. Do you want to continue ? [yes,no] (no) yes root@ external-routing-engine > request virtual-chassis vc-port set fpc-slot 0 pic-slot 0 port 0 member 1 This will also convert FPC slot 0 PIC slot 0 Port 1 to virtual chassis port. Do you want to continue ? [yes,no] (no) yes It is highly recommended that both links between the line card chassis that form an automatic VCP LAG be connected to have multiple VCP LAGs. root@ external-routing-engine > request virtual-chassis vc-port set fpc-slot 6 pic-slot 0 port 0 member 0 This will also convert FPC slot 6 PIC slot 0 Port 1 to virtual chassis port. Do you want to continue ? [yes,no] (no) yes root@ external-routing-engine > request virtual-chassis vc-port set fpc-slot 6 pic-slot 0 port 0 member 1 This will also convert FPC slot 6 PIC slot 0 Port 1 to virtual chassis port. Do you want to continue ? [yes,no] (no) yes

Interconnect the EX8200 Switches via the Converted 10GbE Virtual Chassis Ports Connect the intra-VCP links that carry data traffic between LCCs. As mentioned previously, in some cases such as the failure of Inter-XRE200 VCPs, the intra-VCPs carry data as well as control traffic.

Virtual Chassis Show Commands EX8200 Virtual Chassis status and topology information can be obtained from the following operational commands: - - Show virtual-chassis - - Show virtual-chassis vc-port - - Show chassis lccs


13


root@XRE200-3> show virtual-chassis Preprovisioned Virtual Chassis Virtual Chassis ID: a72e.0aab.8526 Virtual Chassis Mode: Enabled List Member ID Interface 0 (FPC 0-15)

Mastership

Status

Serial No

Model

Prsnt

BT0908500271 ex8208

priority

Neighbor Role

0

Linecard

0/0/0 0/0/1 6/0/0 6/0/1 1 (FPC 16-31)

Prsnt

BT0908500274 ex8208

0

Linecard

0/0/0 0/0/1 6/0/0 6/0/1 8 (FPC 128-143) Prsnt

062010000001 ex-xre

129

Backup*

9 (FPC 144-159) Prsnt

062010000005 ex-xre

129

Master

ID 8 1

vcp-0/0 vcp-

1

vcp-

9 1

vcp-0/1 vcp-

1

vcp-

9 0

vcp-0/0 vcp-

0

vcp-

8 0

vcp-0/1 vcp-

0

vcp-

9 1 0 8 1 0

vcp-0/0 vcp-1/0 vcp-1/1 vcp-0/0 vcp-1/0 vcp-1/1

{master:9} root@XRE200-3> root@XRE200-3> show virtual-chassis vc-port member0: -------------------------------------------------------------------------Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface Slot/PIC/Port vcp-0/0 Dedicated -1 Up 1000 8 vcp-1/1 vcp-0/1 Dedicated -1 Up 1000 9 vcp-1/1 0/0/0 Configured 3 Up 10000 1 vcp-0/0/0 0/0/1 Configured 3 Up 10000 1 vcp-0/0/1 6/0/0 Configured 3 Up 10000 1 vcp-6/0/0 6/0/1 Configured 3 Up 10000 1 vcp-6/0/1 member1: -------------------------------------------------------------------------Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface Slot/PIC/Port

14



vcp-0/0 vcp-0/1 0/0/0 0/0/1 6/0/0 6/0/1

Dedicated Dedicated Configured Configured Configured Configured

-1 -1 3 3 3 3

Up Up Up Up Up Up

1000 1000 10000 10000 10000 10000

0 0 0 0

9 8

vcp-1/0 vcp-1/0 vcp-0/0/0 vcp-0/0/1 vcp-6/0/0 vcp-6/0/1

member8: -------------------------------------------------------------------------Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface Slot/PIC/Port vcp-0/0 Dedicated -1 Up 1000 9 vcp-0/0 vcp-1/0 Dedicated -1 Up 1000 1 vcp-0/1 vcp-1/1 Dedicated -1 Up 1000 0 vcp-0/0 vcp-1/2 Dedicated -1 Down 1000 vcp-1/3 Dedicated -1 Down 1000 member9: -------------------------------------------------------------------------Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface Slot/PIC/Port vcp-0/0 Dedicated -1 Up 1000 8 vcp-0/0 vcp-1/0 Dedicated -1 Up 1000 1 vcp-0/0 vcp-1/1 Dedicated -1 Up 1000 0 vcp-0/1 vcp-1/2 Dedicated -1 Down 1000 vcp-1/3 Dedicated -1 Down 1000 {master:9} root@XRE200-3> root@XRE200-3> show chassis lccs Slot State Uptime 0 Online 22 hours, 29 minutes, 50 seconds 1 Online 22 hours, 30 minutes, 4 seconds 2 Present 3 Present 4 Present 5 Present 6 Present 7 Present {master:9} root@XRE200-3>


15


Migrating EX8200 Standalone Switches to EX8200 Virtual Chassis Configurations This section shows the sequence of steps required to migrate from a redundant standalone EX8200 configuration to a Virtual Chassis configuration while in production to minimize the impact on active traffic flows. In this test network, there are two traffic flows—a routed “L3” flow between tester ports 601/1 and 601/2 and a bridged/ switched “L2” flow between tester ports 601/3 and 601/4 connected to edge switches. The impact to the traffic flows has been noted for each step. The figure below shows the initial network configuration in the EX8200 standalone mode. The topology uses redundant trunk group (RTG) LAGs and VRRP to achieve loop-free forwarding paths. Switch EX8208-0 is the VRRP master and LAGs between EX4200-0 and EX4200-1 are the RTG primary links.

EX8208-0

OSPF Area 0

.10

EX8208-1

V30

xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4

.11

V200 – L2 VLAN V110, V111 – VR IP .1

Active on V30 Passive on V110-V111

8202-0 is VRRP Master

V110 V200

V111 V200

EX4200-0

EX4200-1 RTG

AE to 8208-0 is primary Uplinks are non-revertive

TestPort 601/1

10.1.110.61/24

TestPort 601/3 10.1.200.63/24

TestPort 601/2

10.1.111.62/24

TestPort 601/4 10.1.200.64/24

Figure 5: Redundant pair of EX8200 running VRRP and RTG with all traffic flowing over master EX8200 switch Migrating a pair of EX8200 standalone switches running VRRP to an EX8200 Virtual Chassis configuration involves the following steps: 1. Upgrade the switches to Junos OS version 11.1 2. Detach the first EX8200 switch from the access switches and the peer EX8200 switch 3. Attach the first EX8200 switch to the pair of XRE200s and configure it in Virtual Chassis mode 4. Revert the access switch links back to the first EX8200 switch, now in Virtual Chassis mode 5. Attach the second EX8200 switch to the pair of XRE200s and configure it in Virtual Chassis mode 6. Restore the access switch links back to the second EX8200 switch, now in Virtual Chassis mode 7. Disable VRRP on the EX8200 switches, now in Virtual Chassis mode

16



Upgrade the Switches to Latest Junos OS Version See step 1 from the above section. Detach the First EX8208 from the Access Switches and the Peer EX8208 1. First increase the priority of EX8208-1 so that it preempts EX8208-0 and becomes the VRRP master. This will have about one second impact on L3 traffic flows. set interfaces vlan unit 110 family inet address 10.1.110.11/24 vrrp-group 110 priority 250 set interfaces vlan unit 111 family inet address 10.1.111.11/24 vrrp-group 111 priority 250 2. On EX8208-0, disable edge facing Aggregated Ethernet (AE) interfaces and “commit” the changes. This will have a sub-second impact on traffic flows. Next, disable the ae2 interface which interconnects the two chassis; this will have zero impact on traffic flows. set interface ae0 disable set interface ae1 disable commit set interface ae2 disable commit 3. On EX8208-1, disable the ae2 interface so that the VCP does not “come up” when configured on EX8208-0 in Step 2.

set interfaces ae2 disable

EX8208-0

EX8208-1 xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4 ae2

V110, V111, V200, V30 xe-0/0/0 ae0

xe-1/0/0

V110 V200

xe-5/0/0 xe-4/0/0

V111 V200

ae1

EX4200-0

ae0

EX4200-1

Figure 6: All traffic flows through the backup EX8200 switch while master is being prepared to run EX8200 Virtual Chassis configuration Attach the First EX8208 to the Pair of XRE200s and Configure in Virtual Chassis Mode 1. From the console, load the factory default configuration on EX8208-0 and enable it to function in Virtual Chassis mode: root@8200-0>load factory-default root@8200-0>set system root plain root@8200-0>commit and-quit root@switch>set chassis virtual-chassis


17


2. Configure the master XRE200 (i.e., XRE200-8): • The XRE200s must run the same version of software as the EX8208s. • Create preprovisioned Virtual Chassis configuration on XRE200-8 for all members. For detailed steps, see “Create Preprovisioned Virtual Chassis Configuration on Master XRE200” above. • Interface me0 IP addressing from EX8208-0 can optionally be reused. • Physically connect the XRE200s and verify that they are both “present.” 3. Configure the EX8200 switches via XRE200 configuration mode: • Include the EX8208-1 configuration even though it is not yet physically connected. • Note that ports on the second chassis start with FPC 16. • There are now two LAGs total in the Virtual Chassis. Because the RTG on the EX4200s is non-revertive, traffic should not be received on these ports even though the links are “up.” • VLAN: There is no need for an inter-chassis VLAN. • IP: The IP addressing schemes do not change. • VRRP: Even though the EX8200 Virtual Chassis is one logical switch, this is required in the interim because the hosts will continue to use the VRRP media access control (MAC) address, which is locally cached. to avoid duplicate IP addresses between the EX8200 Virtual Chassis and EX8208-1. • More details and a configuration example can be found here: www.juniper.net/techpubs/en_US/junos/topics/ task/configuration/virtual-chassis-ex8200-cli.html. • Physically connect the XRE200s and EX8208-0 Routing Engine management ports (which are now converted into VCPs). • From XRE200 operational mode, configure VCPs on EX8208-0 that interconnect the two chassis. Note that ports on the EX8200-40XS line card are not supported for VCP operation. request virtual-chassis vc-port set fpc-slot 0 pic-slot 0 port 3 member 0 request virtual-chassis vc-port set fpc-slot 5 pic-slot 0 port 3 member 0

XRE200-8

XRE200-9 vcp-1/0

vcp-1/1

vcp-1/1

vcp-0/0

vcp-0/1 xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4

EX8208-0 Line Card 0

ae0

EX4200-0

EX8208-1 VCP

V110 V200

ae1

V111 V200 EX4200-1

Figure 7: Single-member EX8200 Virtual Chassis is formed while all traffic flows through the backup EX8200

18



Revert the Access Switch Links Back to the First EX8208, Now in Virtual Chassis Mode 1. Manually revert links on the EX4200s back to EX8208-0 via the CLI so that traffic flows through Virtual Chassis Line Card 0. 2. On the EX4200s, disable the ae1 interfaces connected to EX8208-1. Use an application such as SecureCRT, ClusterSSH, or PuTTY Connection Manager to execute a CLI command for multiple sessions simultaneously. This will have a sub-second impact on the traffic flows. set interface ae1 disable commit 3. Leave the ae1 ports disabled, as they will be reconfigured in the next step.

XRE200-8

XRE200-9 vcp-1/0

vcp-1/1

vcp-1/1

vcp-0/0

vcp-0/1 xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4


EX8208-1 VCP

ae0

EX4200-0

V110 V200

ae1

ae1

V111 V200

ae1

EX4200-1

Figure 8: All traffic is migrated from backup EX8200 to the single-member EX8200 Virtual Chassis


19


Attach the Second EX8208 to the Pair of XRE200s and Configure It in Virtual Chassis Mode

XRE200-8

XRE200-9 vcp-1/0

vcp-1/1

vcp-1/2

vcp-1/1

vcp-1/2

vcp-0/0

vcp-0/1

vcp-0/0

vcp-0/1

xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4


EX8208-1 VCP

V110, V111

ae0

V110 V200

EX4200-0

ae1

V111 V200 EX4200-1

Figure 9: All traffic flows through single-member EX8200 Virtual Chassis while backup EX8200 is configured to join EX8200 Virtual Chassis configuration 1. From the console, load the factory default setting on EX8208-1 and enable it to function in Virtual Chassis mode: root@8200-1> root@8200-1> root@8200-1> root@switch>

load factory-default set system root plain commit and-quit set chassis virtual-chassis

After EX8208-1 reboots, make the physical connections between the XRE200s and Routing Engine management ports on EX8208-1. From XRE200 operational mode, configure VCPs on EX8208-0 that interconnect the two chassis: request virtual-chassis vc-port set fpc-slot 0 pic-slot 0 port 3 member 1 request virtual-chassis vc-port set fpc-slot 5 pic-slot 0 port 3 member 1

20



Restore the Access Switch Links Back to the Second EX8208, Now in Virtual Chassis Mode

XRE200-8

XRE200-9 vcp-1/0

vcp-1/1

vcp-1/2

vcp-1/1

vcp-1/2

vcp-0/1

vcp-0/0

vcp-0/1

Virtual Chassis vcp-0/0

xe-0/0/3 xe-0/0/4 xe-5/0/3 xe-5/0/4


ae0


VCP

V110 V200

ae0

EX4200-0

V111 V200 EX4200-1

Figure 10: Traffic is load-balanced over both EX8200 switches migrated to two-member EX8200 Virtual Chassis 1. On each EX4200 switch, move the ports connected to EX8208-1 (ae1) into the LAG that is connected to EX8208-0 (ae0). 2. Remove RTG-related settings from the configuration: delete interfaces ae1

set interfaces xe-0/1/1 ether-options 802.3ad ae0 set interfaces xe-1/1/1 ether-options 802.3ad ae0 delete vlan v110 interface ae1.0 delete vlan v200 interface ae1.0 delete ethernet-switching-options redundant-trunk-group delete protocols rstp interface ae1.0

3. This re-enables the ports that were previously disabled in Step 3. This will have zero to sub-second impact on all traffic flows.


21


Disable VRRP on the EX8200 Switches, Now in Virtual Chassis Mode As mentioned earlier, one of the major advantages of the Virtual Chassis technology is to eliminate VRRP to provide high availability and resiliency. So VRRP must be disabled on the EX8200s running in Virtual Chassis mode. However, the existing VRRP virtual MAC address is cached in the access device’s ARP tables, so they will continue to forward traffic with the destination MAC of the VRRP virtual MAC address. To refresh the ARP tables of the end devices, VRRP must be deleted and its virtual IP addresses must be migrated to the routed VLAN interfaces (RVIs). Next, to force the RVI interfaces to send out a gratuitous ARP with the new MAC address of the newly configured interface, the RVI interfaces must be disabled and re-enabled. As soon as the end devices receive the G-ARP messages, they will refresh their ARP tables with the new MAC address of the RVI interface and start forwarding traffic to the new destination MAC address. This may result in a traffic loss of 1 to 2 seconds. Step-by-step procedure to disable VRRP:

a) Delete VRRP on RVI: edit interfaces vlan unit 110 family inet delete address 10.1.110.11/24 vrrp-group 110 virtual-address 10.1.110.1

b) Configure VRRP virtual IP (VIP) address on the RVI: set address 10.1.110.1/24 exit

c) Disable the RVI and commit: set interfaces vlan unit 110 disable

d) Enable the RVI and commit: delete interfaces vlan unit 110 disable

The figure below depicts the logical network topology of the EX8200 Virtual Chassis configuration after the migration is completed.

EX8208

V110 V200

.1

V111 V200 ae1

ae0

ae0

EX4200-0

TestPort 601/1

10.1.110.61/24

EX4200-1

TestPort 601/3 10.1.200.63/24

TestPort 601/2

10.1.111.62/24

TestPort 601/4 10.1.200.64/24

Figure 11: Logical network topology of EX8200 Virtual Chassis configuration

22



EX8200 Virtual Chassis High Availability Best Practices EX8200 Virtual Chassis configurations are highly resilient, with no single point of failure. The EX8200 Virtual Chassis architecture ensures that no single failure of an element—a chassis, a line card, a Routing Engine, or an interconnection—can render the entire Virtual Chassis inoperable. This is achieved by the combination of hardware and software architectures discussed in the early part of this document. To achieve a highly resilient, fully redundant EX8200 Virtual Chassis configuration with no single point of failure in hardware, control plane, and data plane, the following features must be configured: 1. Fully meshed redundant EX8200 Virtual Chassis configuration with four-member, multi-FPC, intra-Virtual Chassis interconnections as depicted in Figure 12 below.

Active XRE200

Standby XRE200

xe-2/0/0 xe-2/0/1 xe-4/0/4 xe-4/0/5

EX8216-0 Line Card 0 xe-4/0/0 ae0

xe-7/0/0

EX8216-1 Line Card 1 xe-20/0/0 ae1

xe-23/0/0

EX4500-0

EX4500-1

Intra-XRE connection for HA (1GbE) Active XRE to internal Routing Engine connection (1GbE) Standby XRE to internal Routing Engine connection (1GbE) 10GbE LAG intra-Virtual Chassis connection 10GbE Links

Figure 12: Two-member EX8200 Virtual Chassis with dual-homed access devices. 2. GRES on both XRE200s: set chassis redundancy graceful-switchover 3. NSR for routing protocols: set routing-options nonstop-routing 4. Multi-LCC LAGs for access switches. In Figure 12, EX4500-0 and EX4500-1 are connected to the EX8200 Virtual Chassis configuration via multi-LCC LAGs:

set set set set

interfaces interfaces interfaces interfaces

xe-4/0/0 ether-options 802.3ad ae0 xe-20/0/0 ether-options 802.3ad ae0 xe-7/0/0 ether-options 802.3ad ae1 xe-23/0/0 ether-options 802.3ad ae1

5. LACP on the LAGs in slow periodic mode: set interfaces ae0 aggregated-ether-options lacp active set interfaces ae0 aggregated-ether-options lacp periodic slow 6. xSTP and VRRP are completely disabled. 7. The above mentioned steps allow users to achieve a sub-second traffic loss of around 650 to 850 milliseconds during an NSSU operation.


23


EX8200 Virtual Chassis Failover Convergence Times This section describes the convergence times in the event of different types of failures for the topology shown in Figure 13. The configuration highlights of the EX8200 Virtual Chassis systems are as described in the “Best Practices” section above.

EX8208-1

MPLS

L3 LAG

L3 LAG EX8200 Virtual Chassis #1

Test Port #1

VCP-LAG

Test Port #2

LAG LAG

EX8208-2

LAG

Test Port #3

EX8200 Virtual Chassis #2 VCP

LAG LAG

Top of Rack Virtual Chassis #1



Top of Rack Virtual Chassis #4 Dual Homed Servers

Dual Homed Servers Test Port #4

Figure 13: A pair of two-member EX8200 Virtual Chassis configurations with dual-homed access devices interconnected via MPLS Table 4: High Availability and Resiliency Convergence Test Results for Figure 13

24

Event

Result

Disconnect/restore single LAG member from top-of-rack Virtual Chassis #1 switch to EX8200 Virtual Chassis #1

Disconnect: sub-second packet loss Restore: zero packet loss

Disconnect/restore LAG members so that all traffic is passing through VCP ports on EX8200 Virtual Chassis1

Disconnect: sub-second packet loss Restore: zero packet loss

Pull out line card on EX8200 Virtual Chassis1 on which one of the LAG members resides

Zero packet loss

Pull out RE0 from EX8216-0 (Member0 of EX8200 Virtual Chassis #1)

Zero packet loss

Restore RE0 from EX8216-0

Zero packet loss

Power off/on XRE200-1 (master) of EX8200 Virtual Chassis #1 LACP (slow interval) configured on all link bundles

Power Off: sub-second packet loss Power On: zero packet loss

Power on/off XRE200-0 (backup) of EX8200 Virtual Chassis #1 No LACP configured on all link bundles (static)

Zero packet loss

Pull out/restore SCB 0 of EX8216-0 (member0)

Sub-second packet loss Zero packet loss

Power off/on member0 Power on EX8216-1

Sub-second packet loss Sub-millisecond packet loss

Manual switchover of master Routing Engine (XRE200-0 > XRE200-1, request chassis Routing Engine master switch)

Sub-second packet loss

Nonstop upgrade EX8200 Virtual Chassis

Sub-second packet loss



EX8200 Virtual Chassis over Long Distances Configuration Steps Physical Connections of XRE200s and Intermediate Switches

XRE200-8 vcp-0/0 ge-0/0/0 vcp-0/1 vcp-1/0

Switch A EX2200

ge-0/0/1

ge-0/1/0 ge-0/1/0 ge-0/1/2 ge-0/1/2 ge-0/1/3 ge-0/1/3

Switch B EX2200

ge-0/0/2

To LCC0 Master

To LCC0 Backup

UFD on Switch A:

ge-0/0/0 tracks ge-0/1/0 ge-0/0/1 tracks ge-0/1/1 ge-0/0/2 tracks ge-0/1/2

XRE200-9 ge-0/0/0 vcp-0/0

ge-0/0/1 ge-0/0/2

To LCC1 Master

vcp-0/1 vcp-1/0

To LCC1 Backup

UFD on Switch B:

ge-0/0/0 tracks ge-0/1/0 ge-0/0/1 tracks ge-0/1/1 ge-0/0/2 tracks ge-0/1/2

Figure 14: Two XRE200s connected to each other via EX2200 over long distance 1. Assume that ports ge-0/0/0 and ge-0/0/1 on Switch A in Figure 5 are connected to ports vcp-0/0 and vcp-0/1 on the active XRE200 and are the inter-XRE and XRE-LCC1 VCPs, respectively. Also, port ge-0/0/2 on Switch A is connected to port vcp-1/0 on LCC0. 2. Assume that ports ge-0/0/0 and ge-0/0/1 on Switch B are connected to ports vcp-0/0 and vcp-0/1 on the standby XRE200. Also, port ge-0/0/2 on Switch B is connected to port vcp-1/0 on LCC1. 3. Assume that fiber ports ge-0/1/0, ge-0/1/1, and ge-0/1/2 on Switch A and Switch B are interconnected. 4. Therefore, on Switches A and B, the copper ports ge-0/0/0 through ge-0/0/2 pair up with their respective fiber ports ge-0/1/0 through ge-0/1/2 for Unidirectional Failure Detection protocol (UFD) configuration.

Configuration on Intermediate Switches Switch A Configuration and Verification 1. Configure ports ge-0/0/0 and ge-0/1/0 in the same VLAN, say VLAN 10: root@SwitchA> root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#

configure set vlans vlan10 vlan-id 10 set interfaces ge-0/0/0 unit 0 family ethernet-switching set interfaces ge-0/1/0 unit 0 family ethernet-switching set vlans vlan10 interface ge-0/0/0 set vlans vlan10 interface ge-0/1/0

2. Configure ports ge-0/0/1 and ports ge-0/1/1 in the same VLAN, say VLAN 20: root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#

set set set set set

vlans vlan20 vlan-id 20 interfaces ge-0/0/1 unit 0 family ethernet-switching interfaces ge-0/1/1 unit 0 family ethernet-switching vlans vlan20 interface ge-0/0/1 vlans vlan20 interface ge-0/1/1

3. Configure ports ge-0/0/2 and ports ge-0/1/2 in the same VLAN, say VLAN 30: root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#


set set set set set


25


4. Configure three UFD groups so that any failures of the fiber links will trigger the UFD protocol to disable the corresponding copper Virtual Chassis link: root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#

edit protocols uplink-failure-detection set group Inter-XRE link-to-monitor ge-0/1/0 set group Inter-XRE link-to-disable ge-0/0/0 set group XRE-Master-RE link-to-monitor ge-0/1/1 set group XRE-Master-RE link-to-disable ge-0/0/1 set group XRE-Backup-RE link-to-monitor ge-0/1/2 set group XRE-Backup-RE link-to-disable ge-0/0/2 commit

5. Verify the UFD configuration: root@SwitchA> show Group : Uplink : Downlink : Failure Action : Group Uplink Downlink Failure Action

: : : :

uplink-failure-detection Inter-XRE ge-0/1/0 ge-0/0/0 Inactive XRE-Master-RE ge-0/1/1 ge-0/0/1 Inactive

Group : XRE-Backup-RE Uplink : ge-0/1/2 Downlink : ge-0/0/2 Failure Action : Inactive 6. Configure 802.3ah on all of the fiber ports: root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#

set protocols oam ethernet link-fault-management interface ge-0/1/0 set protocols oam ethernet link-fault-management interface ge-0/1/1 set protocols oam ethernet link-fault-management interface ge-0/1/2 commit

7. Configure Jumbo frames on all interfaces.

root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA# root@SwitchA#

set interface set interface set interface set interface set interface set interface commit

ge-0/0/0 ge-0/0/1 ge-0/0/2 ge-0/1/0 ge-0/1/1 ge-0/1/2

mtu mtu mtu mtu mtu mtu

9216 9216 9216 9216 9216 9216

Switch B Configuration and Verification 1. Configure ports ge-0/0/0 and ge-0/1/0 in the same VLAN, say VLAN 10: root@SwitchB> root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB#

configure set vlans vlan10 vlan-id 10 set interfaces ge-0/0/0 unit 0 family ethernet-switching set interfaces ge-0/1/0 unit 0 family ethernet-switching set vlans vlan10 interface ge-0/0/0

root@SwitchB# set vlans vlan10 interface ge-0/1/0

26



2. Configure ports ge-0/0/1 and ports ge-0/1/1 in the same VLAN, say VLAN 20: root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB#

set set set set set


3. Configure ports ge-0/0/2 and ports ge-0/1/2 in the same VLAN, say VLAN 30: root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB#

set set set set set


4. Configure three UFD groups so that any failures of the fiber links will trigger the UFD protocol to disable the corresponding copper Virtual Chassis link: root@SwitchB# edit protocols uplink-failure-detection [edit protocols uplink-failure-detection] root@SwitchB# set group Inter-XRE link-to-monitor ge-0/1/0 root@SwitchB# set group Inter-XRE link-to-disable ge-0/0/0 root@SwitchB# set group XRE-Master-RE link-to-monitor ge-0/1/1 root@SwitchB# set group XRE-Master-RE link-to-disable ge-0/0/1 root@SwitchB# set group XRE-Backup-RE link-to-monitor ge-0/1/2 root@SwitchB# set group XRE-Backup-RE link-to-disable ge-0/0/2 root@SwitchB# commit 5. Verify the UFD configuration: root@SwitchB> show uplink-failure-detection Group Uplink Downlink Failure Action

: : : :

Inter-XRE ge-0/1/0 ge-0/0/0 Inactive

Group Uplink Downlink Failure Action

: : : :

XRE-Master-RE ge-0/1/1 ge-0/0/1 Inactive

Group Uplink Downlink Failure Action

: : : :

XRE-Backup-RE ge-0/1/2 ge-0/0/2 Inactive

Note that the above configurations on Switches A and B ensure that fiber breaks cause the copper ports to be disabled and not vice versa. Therefore, a copper port failure or an XRE200 reboot may result in up to one minute of traffic loss. 6. Configure 802.3ah on all of the fiber ports: root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB#


set protocols oam ethernet link-fault-management interface ge-0/1/0 set protocols oam ethernet link-fault-management interface ge-0/1/1 set protocols oam ethernet link-fault-management interface ge-0/1/2 commit

27


7. Configure Jumbo frames on all interfaces.

root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB# root@SwitchB#

set interface set interface set interface set interface set interface set interface commit

ge-0/0/0 ge-0/0/1 ge-0/0/2 ge-0/1/0 ge-0/1/1 ge-0/1/2

mtu mtu mtu mtu mtu mtu

9216 9216 9216 9216 9216 9216

XRE200 Configuration 1. Enable GRES and failover on loss of keepalive on both XRE200s: root@xre200-8# show chassis redundancy failover on-loss-of-keepalives; graceful-switchover; root@xre200-9# show chassis redundancy failover on-loss-of-keepalives; graceful-switchover;

Conclusion EX8200 switches with Virtual Chassis technology address the fundamental requirements of a core or collapsed core/ aggregation switch, delivering a solution for implementing a network fabric in campus and data center environments. EX8200 Virtual Chassis configurations support multipathing and eliminate the inefficiencies associated with Spanning Tree Protocol (STP), providing a highly resilient system, and simplifying management and control plane operations at scale. Virtual Chassis technology on the EX8200 modular platforms also reduces the bandwidth inefficiencies associated with STP to accelerate network convergence and simplify network architecture. Virtual Chassis technology on the high capacity EX8200 line of Ethernet switches delivers a cost-effective solution that simultaneously achieves the following objectives: • Extending the core switching capacity beyond the EX8216 switch’s limit of 2.5 Tbps • Protecting the network against any single point of failure • Fully utilizing redundant paths of the network by eliminating the need for Spanning Tree Protocol and VRRP Implicitly, the deployment of a Virtual Chassis configuration reduces the number of devices that need to be managed, with all of the costs for power and maintenance that come with them. The simple steps and recommendations outlined in this guide will help you take full advantage of these benefits.

28



About Juniper Networks Juniper Networks is in the business of network innovation. From devices to data centers, from consumers to cloud providers, Juniper Networks delivers the software, silicon and systems that transform the experience and economics of networking. The company serves customers and partners worldwide. Additional information can be found at www.juniper.net.

Corporate and Sales Headquarters

APAC and EMEA Headquarters

To purchase Juniper Networks solutions,

Juniper Networks, Inc.

Juniper Networks International B.V.

please contact your Juniper Networks

1194 North Mathilda Avenue

Boeing Avenue 240

Sunnyvale, CA 94089 USA

1119 PZ Schiphol-Rijk

representative at 1-866-298-6428 or

Phone: 888.JUNIPER (888.586.4737)

Amsterdam, The Netherlands

or 408.745.2000

Phone: 31.0.207.125.700

Fax: 408.745.2100

Fax: 31.0.207.125.701

authorized reseller.

www.juniper.net Copyright 2013 Juniper Networks, Inc. All rights reserved. Juniper Networks, the Juniper Networks logo, Junos, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United States and other countries. All other trademarks, service marks, registered marks, or registered service marks are the property of their respective owners. Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice.

8010081-003-EN

Jan 2013


Printed on recycled paper

29

Best Practice Guidelines for Deploying EX8200 ... - Juniper Networks

Best Practice Guidelines for Deploying EX8200 ... - Juniper Networks

Suggest Documents

Best Practices and Guidelines for Deploying the Oracle VM Blade ...

EMQN Best Practice Guidelines for molecular and

Evidence-Based Best-Practice Guidelines for

EAZA Best Practice Guidelines for Gorilla

placer deposits - CIM best practice guidelines

Best Clinical Practice Gynaecological Cancer Guidelines 2009

EAZA Best Practice Guidelines Red Panda

CMSUK Standards & Best Practice Guidelines 2nd Edition

best practice guidelines: wound management in

EAZA Best Practice Guidelines Red Panda

CMSUK Standards & Best Practice Guidelines 2nd Edition

National Best Practice Guidelines Consultation ... - Informing Families

CMSUK Standards & Best Practice Guidelines 2nd Edition

Design Guidelines for Deploying Closed Loop Systems

Canadian Best Practice Guidelines - Gold Centre

Guidelines for deploying and implementing scheduling ... - CiteSeerX

Guidelines for deploying and implementing scheduling ... - CiteSeerX

Guidelines for Constructing and Deploying Common

Juniper Flow Monitoring - Juniper Networks [PDF]

Release Notes - Juniper Networks

Juniper Networks Inc.xlsx

Juniper Networks Evaluation Agreement

A systematic review of clinical practice guidelines and best practice ...

EMQN best practice guidelines for the molecular ... - ePrints Soton