network services in a unified manner, through seamless insertion and automation .... Note: Cisco ACI supports RHI (routing) only for external devices. Figure 1.
Citrix Systems, Inc.
Deploying NetScaler ADCs in Cisco Application Centric Infrastructure (ACI)
Contents Introduction.................................................................................................................................................... 3 Policy-Based Automation Framework ....................................................................................................... 3 Policy-Based Service Insertion ................................................................................................................. 3 Benefits of Using Citrix NetScaler ADCs in Cisco ACI ............................................................................. 4 Deployment Modes of NetScaler ADCs in Cisco ACI ............................................................................... 4 Inline Mode ............................................................................................................................................ 4 Anywhere Mode .................................................................................................................................... 5 NetScaler Device Package Supported Features ...................................................................................... 6 Limitation ................................................................................................................................................... 7 Deploying the NetScaler ADC in Cisco ACI .................................................................................................. 8 Prerequisites ............................................................................................................................................. 8 Importing a Device Package ..................................................................................................................... 9 Registering the Device .............................................................................................................................. 9 Prerequisites ....................................................................................................................................... 10 Creating and Deploying a Service Graph ............................................................................................... 13 Applying the Service Graph Template to Endpoint Groups ................................................................ 14 Managing the NetScaler in Cisco ACI ......................................................................................................... 17 Modifying Attributes of the Deployed Service Graph at the EPG Level .................................................. 17 Deleting the Service Graph Template ..................................................................................................... 19 Monitoring NetScaler Device Health ....................................................................................................... 19 Monitoring Service Graph Health ............................................................................................................ 20 Customizing or Importing Function Profiles ................................................................................................ 20 Sample POC Kit on GitHub ......................................................................................................................... 23 Troubleshooting .......................................................................................................................................... 23 APIC Fault Reports ................................................................................................................................. 23 Logs Generated by Device Package ...................................................................................................... 23 Debug.log ............................................................................................................................................ 24 Apic.log ................................................................................................................................................ 24 Periodic.log.......................................................................................................................................... 25 FAQs ........................................................................................................................................................... 27
Introduction As businesses quickly move to make the datacenter more agile, the application centric automation and virtualization of both hardware and software infrastructure become increasingly important. Cisco Application Centric Infrastructure (ACI) supplies the critical link between business-based requirements for applications and the infrastructure that supports them. The Citrix NetScaler application delivery controller (ADC) connects infrastructure and applications and makes their configuration available to the Cisco Application Policy Infrastructure Controller (APIC) through integration. Citrix NetScaler and Cisco ACI enable datacenter and cloud administrators to holistically control L2-L7 network services in a unified manner, through seamless insertion and automation of best-in-class NetScaler services into next-generation datacenters built on Cisco's ACI Architectures. A NetScaler ADC leverages the Cisco Application Policy Infrastructure Controller (APIC) to programmatically automate network provisioning and control on the basis of application requirements and policies for both datacenter and enterprise environments. Cisco APIC addresses the two main requirements for achieving the application centric data center vision:
Policy-based automation framework Policy-based service insertion technology
Policy-Based Automation Framework A policy-based automation framework enables the Cisco APIC to dynamically provision and configure resources according to application requirements. As a result, core services such as firewalls and Layer 4 through 7 services can be consumed by applications, and these services can be made ready to use in a single automated step. Being application centric, the APIC allows the creation of application profiles, which define the Layer 4 through 7 services consumed by a given datacenter-tenant application. A NetScaler ADC provides L4-L7 services such as load balancing, application acceleration, and application security. Integration between the Cisco APIC controller and the NetScaler ADC is achieved through a NetScaler device package. Imported by the APIC controller, the device package enables REST-based API integration and allows the APIC controller to perform detailed feature-level configuration of the NetScaler.
Policy-Based Service Insertion The Cisco APIC solution automates the steps of routing network traffic to the correct services on the basis of application policies. L4-L7 resources can be dynamically provisioned and configured according to application requirements on a per tenant basis. The Cisco APIC offers APIs or a graphical drag and drop GUI for easy creation of L4-L7 Service Graphs that specify network traffic routing. Any of the L4-L7 ADC features available in the NetScaler device package can be included in a Service Graph definition, allowing comprehensive NetScaler integration with the Cisco APIC. Policy-based service insertion automates the steps of routing network traffic to the correct services as specified by application policies. The automated addition, removal, and reordering of services allows administrators to quickly change the resources allocated to an application, without the need to rewire and reconfigure the network or relocate the services. For example, if a business decides to use the load balancing feature of a modern ADC, administrators can simply redefine the policy for the services that should be used for the related applications. The Cisco APIC can dynamically distribute new policies to the infrastructure and service nodes in minutes, without requiring manual changes to the network. Once created, a Service Graph can be assigned to an Application Profile and contracted to a data center tenant, thereby defining the network traffic flow for that specific application and tenant.
Benefits of Using Citrix NetScaler ADCs in Cisco ACI The unique Cisco ACI and Citrix NetScaler joint solution improves data center operations and application deployment, using the Cisco APIC as the central policy-control and management station, and Cisco ACI service-insertion technology to direct traffic to the appropriate service nodes. The main benefits include:
Central point of network control with ADC service policy coordination and automation: The Cisco APIC acts as a point of configuration management and automation for NetScaler ADCs (both MPX appliances and VPX virtual appliances), tightly coordinates the ADC service delivery with the network automation, and provides end-to-end telemetry and visibility of service-aware applications and tenants. Scalable and elastic architecture for NetScaler ADCs: Cisco ACI defines a policy-based service insertion mechanism for both physical and virtual ADC appliances, providing full lifecycle service management based on workload instantiation and decommissioning. Investment protection: Cisco ACI and Cisco APIC are fully compatible with existing ADC networks, preserving existing service operation models and using open standards protocols.
Deployment Modes of NetScaler ADCs in Cisco ACI A NetScaler ADC resides between the clients and the servers, so that client requests pass through it and the server response pass through it or bypasses it based on the mode you have deployed the NetScaler. In a typical installation, virtual servers configured on the ADC provide connection points that clients use to access the applications behind the ADC. In this case, the ADC owns public IP addresses that are associated with its virtual servers, while the real servers are isolated in a private network. It is also possible to operate the ADC in a transparent mode as an L2 bridge or L3 router, or even to combine aspects of these and other modes. Note: NetScaler L2 (Go-through) mode is not applicable to Cisco ACI deployment. A NetScaler appliance logically residing between clients and servers can be deployed in either of two modes:
Inline Anywhere
Inline Mode In inline mode, multiple network interfaces of the NetScaler ADC are connected to a leaf node of the Cisco ACI fabric, and the NetScaler ADC is logically placed between the clients and the servers that are in different subnets respectively. The appliance has a separate network interface for client networks and a separate network interface for server networks. It is possible for the servers to be in a public network and the clients to directly access the servers through the appliance, with the appliance transparently applying the L4-L7 features. Usually, virtual servers are configured to provide an abstraction of the real servers. Traffic from client passes through the ADC to access a load balanced server. Client requests at the fabric are forwarded to the NetScaler ADC, and the NetScaler ADC uses the configured load balancing method to select the server. Consider an example of a load balancing setup, in the Cisco ACI fabric, that uses a NetScaler ADC called NS1, which is deployed in inline mode. NS1 is connected to leaf node L1 of the Cisco ACI fabric. Load balancing virtual server LBVS1 on NS1 is used to load balance servers S1 and S2 in the Cisco ACI fabric. Servers S1 and S2 belong to same subnet, 192.0.2.0/24. NetScaler NS1 is connected to L1 through two interfaces. The first link is dedicated to client-side connections and the second link is dedicated to server-side connections.
Subnet IP (SNIP) address SNIP1 (192.0.2.10) is configured on NS1 for enabling NS1 to communicate with servers S1 and S2. LBVS1 is accessible through the first link. Using routing protocols, NS1 advertises routes for LBVS1 and SNIP1 to the Cisco ACI fabric. Similarly, the fabric advertises routes for S1 and S2 to NS1. Services SVC-S1 and SVC-S2 on NS1 represent servers S1 and S2, respectively. Note: Cisco ACI supports RHI (routing) only for external devices. Figure 1. Inline Deployment Mode
Following is the traffic flow in this example: 1. Client CL1 sends a request packet to LBVS1. The request packet has: Source IP = IP address of the client Destination IP = IP address of LBVS1 (203.0.113.15) 2. LBVS1 of NS1 receives the request packet. 3. LBVS1's load balancing algorithm selects server S2. 4. NS1 opens a connection between SNIP1 and S2, and then sends the request packet from SNIP1 to S2. The request packet has: Source IP address = SNIP1 (192.0.1.10) Destination IP address = IP address of S2 (192.0.2.20) 5. S2’s response reaches CL1 through NS1.
Anywhere Mode In Anywhere mode, single or multiple network interface of the ADC is connected to one of the leaf node in a subnet of the Cisco ACI fabric. Anywhere mode can simplify network changes needed for NetScaler ADC installation in some environments. Client requests received on the fabric are forwarded to the ADC, and the ADC uses the configured load balancing method to select the server. Consider an example of a load balancing setup, in the Cisco ACI fabric, that uses a NetScaler ADC called NS1, which is deployed in Anywhere mode. NS1 is connected to leaf node L1 of the Cisco ACI fabric.
Load balancing virtual server LBVS1 on NS1 is used to load balance servers S1 and S2 in the Cisco ACI fabric. Servers S1 and S2 belong to same subnet, 192.0.2.0/24. Only one interface of NS1 is connected to L1. SNIP address SNIP1 (192.0.1.10) is configured on NS1 and is used by NS1 to communicate with servers S1 and S2. Using routing protocols, NS1 advertises routes for LBVS1 and SNIP1 to the Cisco ACI fabric. Similarly, the fabric advertises routes for S1 and S2 to NS1. Services SVC-S1 and SVC-S2 on NS1 represent servers S1 and S2, respectively. Note: Cisco ACI supports RHI (routing) only for external devices. Figure 2. Anywhere Deployment Mode
Following is the traffic flow in this example: 1. Client CL1 sends a request packet to LBVS1. The request packet has: Source IP = IP address of the client Destination IP = IP address of LBVS1 (203.0.113.15) 2. LBVS1 of NS1 receives the request packet. 3. LBVS1's load balancing algorithm selects server S2. 4. NS1 opens a connection between SNIP1 and S2, and then sends the request packet from SNIP1 to S2. The request packet has: Source IP address = SNIP1 (192.0.1.10) Destination IP address = IP address of S2 (192.0.2.20) 5. S2’s response reaches CL1 through NS1.
NetScaler Device Package Supported Features Citrix has introduced a new notion of function-definition, which includes the complete configuration details of a particular feature, such as Load Balancing. Cisco APIC mandates feature definitions. These definitions are easy to use and they simplify configuration. The entire NetScaler features set is included in the various functions definitions, although not all features are currently supported.
The NetScaler device package includes the following features:
Load Balancing SSL Offload AAA Application Firewall Cache Redirection Compression Content Switching DataStream Domain Name Service Global Server Load Balancing Integrated Caching
Note: NetScaler device package supports NetScaler SDX mixed mode deployment but as an out-of-band configuration. You can download the device package from the Citrix web site.
Limitation
You must take extra precaution when removing a NetScaler ADC's configuration object from Cisco APIC. It is important to remove an object's bindings first, before you delete the object, because the device package does not display any error message for a failed deletion. For example, if you delete a virtual server without unbinding the services bound to it, the NetScaler ADC displays an error message informing you that you need to first unbind the service from the virtual server. However, the device package does not display such an error message. You cannot modify an existing binding. To change a binding, the administrator must remove the existing binding and create a new one. The following NetScaler feature configurations are out-of-band. They cannot be performed through Cisco APIC: o High availability o Management network that is used to communicate between APIC to NetScaler device. This includes Subnet IP address (SNIP), VLAN, Interfaces, and NetScaler management IP address (NSIP) bindings. o SSL certificates o System user accounts and Role-Based-Access (RBA) policies Citrix NetScaler SDX configuration is not supported through APIC.
Deploying the NetScaler ADC in Cisco ACI Use Cisco APIC to deploy a NetScaler ADC in Cisco ACI.
Prerequisites Make sure that:
You have conceptual knowledge of Cisco ACI components and Citrix NetScaler ADCs. o For more information about Cisco ACI and its components, see the product documentation at http://www.cisco.com/c/en/us/support/cloud-systemsmanagement/application-policy-infrastructure-controller-apic/tsd-products-support-serieshome.html. o For more information about the Citrix NetScaler ADCs, see the Citrix NetScaler product documentation at http://docs.citrix.com/. All the required components of Cisco ACI, including Cisco APIC in the datacenter, are set up and configured. For more information about Cisco ACI and its components, see the product documentation at http://www.cisco.com/c/en/us/support/cloud-systems-management/applicationpolicy-infrastructure-controller-apic/tsd-products-support-series-home.html. The NetScaler ADCs is deployed in the datacenter and has network connectivity to Cisco ACI. You are cautious when providing configuration data. NetScaler features are configured as function definitions in APIC, so make sure that: o You provide the mandatory data for all the required entities for a given function. o After configuring an object, you do not change attributes that cannot be modified (for example, serviceType of lbvserver in the load balancing function). o You are familiar with all the required parameters for a given object, such as lbvserver. For an object that has a composite key, merely providing a unique name is not sufficient to create the object.
To deploy NetScaler ADC in Cisco ACI by using Cisco APIC: 1. Configure the NetScaler ADCs for Management Access. You need to configure the management IP address (NSIP) and management VLAN (NSVLAN, VLAN of NSIP), and specify the default gateway on the deployed NetScaler ADCs that are to be integrated with Cisco ACI. Also, make sure that you configure the high availability and SSL certificates related configurations. These configurations are made through the user interfaces of the NetScaler ADCs. For more information, see the Citrix NetScaler product documentation at: http://docs.citrix.com/en-us/netscaler.html. 2. Download the NetScaler ADC Device package. A NetScaler device package provides the APIC with information about NetScaler ADCs, including what NetScaler ADCs are and what they are capable of. A NetScaler device package is a zip file containing the following parts: Device Model. An XML file that contains the following: o Device properties (for example, model and NetScaler software version) o Functions provided by NetScaler ADCs (for example, load balancing) o Configuration parameters of each function o Device configuration parameters o Function Profiles Device script. A Python script that integrates the APIC and the NetScaler ADC. The APIC events are mapped to function calls defined in the device script. Functional profile. A profile of parameters with default values that are specified by Citrix. The administrator can configure a function to use these default values.
Device-level configuration parameters. A configuration file specifying the values of the parameters that are required by a NetScaler ADC. The configuration can be shared by one or more of the graphs that use the NetScaler ADC. 3. Import the NetScaler Device Package into Cisco ACI. For detailed instructions, see Importing a Device Package. 4. Register the NetScaler ADC with the Cisco ACI. For detailed instructions, see Registering the Device. 5. Create and deploy a service graph template. For detailed instructions, see Creating and Deploying a Service Graph.
Importing a Device Package Cisco APIC uses a device package to communicate with NetScaler. Download the device package from the Citrix web site and import the device package to APIC. To import device package to APIC by using the APIC GUI: 1. On the menu bar, click L4-L7 Services tab and select the Packages panel. 2. In the Navigation pane, right-click on L4-L7 Device Types and select Import Device Package. 3. In the Import Device Package dialog box, click Browse to select the downloaded NetScaler device package. 4. Click Submit. After successfully importing the device package to APIC, in the Navigation pane, you can view the details of the device package by clicking Citrix-NetScaler-1.0.
Important: After you import the device package, make sure that there are no faults in APIC. You can view the faults by clicking the Faults tab in the Device Types window.
Registering the Device You need to register the device, in this case the NetScaler ADC, so that it can communicate with the Cisco ACI. You need to configure the basic settings of the device configuration, such as configuration management IP addresses, and credentials. You must also physically connect the device to the fabric, and power on the device.
Note: Make sure that you make a note of:
The connection interfaces and IP addresses that are used for management and data-path connectivity. Leaf-switch details: NetScaler IP addresses, ports, interfaces, and so on.
Prerequisites Make sure that you have configured all the Cisco ACI related entities: Tenant, Application-profile, endpoint groups (EPGs) and so on. To register the device by using the APIC GUI: 1. 2. 3. 4. 5.
6. 7. 8. 9.
On the menu bar, click Tenants > All Tenants. In the Work pane, double click the tenant’s name. In the Navigation pane, select tenant_name > L4-L7 Services > L4-L7 Devices. In the Work pane, select Actions > Create L4-L7 Devices. In the Create L4-L7 Devices dialog box, in the General section, perform the following: a. Select the Managed check box. b. In the Name field, enter a name for the device. c. In the Service Type drop-down list, select ADC. d. In the Device Type field, select Physical. Note: Make sure that for VMware ESX, select Virtual and associate the respective Virtual Machine Manager (VMM) domain. e. In the Physical Domain drop-down list, select the physical domain. f. In the Mode field, select Single Node or HA Cluster, depending on your requirement. g. In the Device Package drop-down list, select Citirix-NetScaler-1.0. h. In the Model drop-down list, select the device model. For example, NetScaler-MPX, or NetScaler-VPX. In the Connectivity section, select Out-Of-Band in the APIC to Device Management Connectivity field. In the Credentials section, specify the user name and password for access to the device. In the Device 1 section, complete the management related configuration. In the Cluster section, complete the management related configuration for the cluster.
10. Click Next. The Device Configuration page displays a list of possible features and parameters for the package you are using. It includes a tab with the Basic parameters displayed, and an All Parameters tab that displays all the available parameters of your device package (including the basic parameters). Note: The NetScaler device package does not support some device-level configuration, but you can configure the following cluster-level settings: NTP SNMP Feature Turn or/off Mode Turn on/off
11. On the Device Configuration page, in the Feature section, select the feature that you want to use and configure the parameters related to the feature, and click Update. 12. Click Finish. 13. In the Work pane, review the configuration details and click Submit. Important: After you register the device, make sure that there are no faults in APIC. You can view the faults by clicking the Faults tab in the Work pane.
Creating and Deploying a Service Graph You have to use Cisco APIC service graph templates to create and deploy the NetScaler ADCs. Cisco ACI treats services as an integral part of an application. Any services that are required are treated as a service graph that is instantiated on the Cisco ACI fabric from the APIC. You need to define the service for the application, and service graphs identify the set of network or service functions that are needed by the application. After the graph is configured in the APIC, the APIC automatically configures the services according to the service function requirements that are specified in the service graph. The APIC also automatically configures the network according to the needs of the service function that is specified in the service graph, which does not require any changes in the service device. A service graph is represented as two or more tiers of an application with the appropriate service function inserted between them. A service graph is inserted between the source and destination EPGs by a contract. To create a service graph by using the APIC GUI: 1. On the menu bar, choose Tenants > All Tenants. 2. In the Work pane, double click the tenant's name. 3. In the Navigation pane, select tenant_name > L4-L7 Services > L4-L7 Service Graph Templates. 4. In the Work pane, select Actions > Create a L4-L7 Service Graph Template. 5. In the Create a L4-L7 Service Graph Template dialog box, in the Device Clusters section, select a device cluster and perform the following: a. In the Graph Name field, enter the name of the service graph template. b. In the Graph Type field, select Create A New One. c. From the Device Cluster section, drag the device and drop it between the consumer endpoint group and provider endpoint group to create a service node. d. In the device_name information section, do the following: i. In the ADC field, select One-Arm or Two-Arm, depending on how NetScaler is deployed in the fabric. ii. In the Profile drop-down list, select the function profile provided in the device package.
6. Click Submit.
7. In the Navigation pane, click the service graph template. The screen presents a graphic topology of the service graph template. Note: Cisco APIC supports the notion of connectors, and these connectors are visible in the ADCCluster node. The connectors define the network traffic direction and the device script that dynamically binds the allocated VLAN to a virtual IP (VIP) or subnet IP (SNIP) address, depending on whether the connection is external or internal. VLANs are also bound to specific interfaces used for inbound and outbound traffic.
Applying the Service Graph Template to Endpoint Groups You need to apply the created service graph template to the endpoint groups (EPGs) to deploy the NetScaler ADCs in Cisco ACI. Prerequisites Make sure that you configured EPGs when you configured the appliance profile. To apply the service graph template to EPGs: 1. On the menu bar, choose Tenants > All Tenants. 2. In the Work pane, double click the tenant's name. 3. In the Navigation pane, choose tenant_name > L4-L7 Services > L4-L7 Service Graph Templates > template_name. 4. In the Work pane, choose Actions > Apply L4-L7 Service Graph Template. 5. In the Apply L4-L7 Service Graph Template To EPGs dialog box, in the EPG Information section, complete the following fields: a. In the Consumer EPG/External Network drop-down list, select the consumer endpoint group. b. In the Provider EPG/External Network drop-down list, select the provided endpoint group.
6. In the Contract Information section, complete the appropriate fields. The contract information is specific to Cisco APIC and is configured as part of the security policies associated with the EPGs. 7. Click Next. 8. In the Device Clusters section, select a device cluster. 9. In the Graph Template drop-down list, select the service graph template that you created. 10. In the Connector section, do the following: a. In the Type field, select General. b. In the BD drop-down list, select the bridge domain. Connector details are part of the bridge domain that is part of the Cisco APIC infrastructure model. c. In the Cluster Interface drop-down list, select the appropriate cluster interface for the selected bridge domain.
The Cisco APIC uses the selected bridge domains for data path traffic between the NetScaler ADC device and the fabric as required by the selected service graph template.
11. Click Next. 12. On the Parameters screen, on the Required Parameters tab, enter the names and values, as appropriate, for all of the required parameters. The Cisco APIC GUI allows you to filter the parameters on the basis of features (for example, load balancing). You can view and set all the mandatory parameters on the Required Parameters tab, and you can view and set all the other parameters related to the feature on the All Parameters tab.
13. Click Finish. Important: After you apply the service graph template, make sure that there are no faults in the deployed graph. You can view the faults by clicking the Faults tab in the Work pane.
Also, you can verify the configuration using NetScaler or CLI.
Managing the NetScaler in Cisco ACI Using the Cisco APIC GUI, you can:
Modify attributes related to the deployed service graph template, at the EPG level. Delete the deployed service graph template. Monitor the NetScaler device health. Monitor the deployed service graph template health.
Modifying Attributes of the Deployed Service Graph at the EPG Level After you have deployed the service graph template, you can edit the parameters related to the deployed service graph at the EPG level. To edit the parameters of the deployed servicer graph at the EPG Level: 1. On the menu bar, select Tenants > All Tenants. 2. In the Work pane, double click the tenant's name. 3. In the Navigation pane, expand tenant_name > Application Profiles > app_profile_name > Application EPGs > created_epg > L4-L7 Service Parameters.
4. Click the Switch To Edit Mode button.
5. In the Edit L4-L7 Service Parameters dialog box, do the following: a. In the Contract Name drop-down list, select the contract. b. In the Graph Name drop-down list, select the graph. c. In the Node Name drop-down list, select the node. d. In the Features section, select the feature that you want to edit and, on the Basic Parameters or All Parameters tabs, edit the values of the parameters related to the feature.
e. Click Submit.
Deleting the Service Graph Template You can delete the service graph template by using the Cisco APIC GUI. To delete a service graph by using the APIC GUI: 1. On the menu bar, select Tenants > All Tenants. 2. In the Work pane, double click the tenant's name. 3. In the Navigation pane, choose tenant_name > L4-L7 Services > L4-L7 Service Graph Templates. 4. Right-click on the service graph template that you want to delete, and then click Delete.
Monitoring NetScaler Device Health After you configure a service graph template and attach the graph to an endpoint group (EPG) and a contract, you can monitor NetScaler devices at the tenant level. The Cisco APIC monitors a NetScaler device by periodically polling for device health. It also collects relevant statistical information from the device and uses that information to calculate the device's health score on a scale from 0 to 100, where 0 indicates that the device is down and 100 indicates that it is in good health. You can also monitor what devices are in use, which VLANs are configured for a NetScaler device, the parameters passed to the device, the statistics of the device, and the health of the device. To monitor NetScaler device by using the APIC GUI: 1. 2. 3. 4.
On the menu bar, choose Tenants > All Tenants. In the Work pane, double click the name of the tenant whose service graph you want to monitor. In the Navigation pane, expand tenant_name > L4-L7 Services > Deployed Devices. Select the deployed NetScaler device and click the Health tab.
Note: For detailed NetScaler specific monitoring details, use the NetScaler GUI.
Monitoring Service Graph Health After you configure a service graph and attach the graph to an endpoint group (EPG) and a contract, you can monitor the service graph instance. The Cisco APIC monitors the service graph template by periodically polling for the health of the deployed service graph, and it collects various statistical information about the deployed service graph (for example, vserver, service group, and service group member). The Cisco APIC calculates the health score for the graph on a scale of 0 to 100, where 0 indicates that the services are down and 100 indicates that they are in good health. You can also view the state of a graph instance, functions of a graph instance, resources allocated to a function, and parameters specified for a function. To monitor the service graph template by using the APIC GUI:
On the menu bar, choose Tenants > All Tenants. In the Work pane, double click the name of the tenant whose service graph you want to monitor. In the Navigation pane, expand tenant_name > L4-L7 Services > Deployed Devices. Select the deployed service graph template and click the Health tab.
Customizing or Importing Function Profiles A function profile is an instance of the function definition, with default values assigned to various attributes for various entities in the definition. You can use function profiles to customize the configurations of any applications that use common ADC services, such as load balancing. The NetScaler device package provides built-in function profiles for all the function definitions listed in the device package, as shown below.
You can customize the existing built-in function profiles or import function profiles from the local file system. To customize a built-in function profile:
1. 2. 3. 4. 5.
On the menu bar, select Tenants > All Tenants. In the Work pane, double click the tenant's name. In the Navigation pane, choose tenant_name > L4-L7 Services > Function Profiles. In the Work pane, choose Actions > Create L4-L7 Services Function Profile. In the Create L4-L7 Services Function Profile dialog box, perform the following: a. In the Name field, enter a name for the function profile. b. In the Description field, enter a brief description of the function profile. c. In the Profile Group drop-down list, select the function profile group in which you want the function profile be listed. d. Select the Copy Existing Profile Parameters checkbox. e. In the Profile drop-down list, select the built-in function profile that you want to customize. f. In the Features section, select the feature that you want to edit and, on the Basic Parameters or All Parameters tab, customize the parameters related to the feature. g. Click Submit.
The customized function profile appears under tenant_name > L4-L7 Services > Function Profiles.
To import a function profile from the local file system: 1. On the menu bar, select Tenants > All Tenants. 2. In the Work pane, double click the tenant's name. 3. In the Navigation pane, choose tenant_name > L4-L7 Services > Function Profiles. 4. You can use an existing function profile group or create a new function group. If you want to create a new function group, in the Work pane, choose Actions > Create Profile Group. 5. Right-click the previously existing or newly created function profile group and click Post. 6. In the Post dialog box, click Browse and select the function profile file in the local file system. 7. Click Post.
The imported function profile file appears under the function profile group. For more information on the behavior of function profile, see Cisco Product Documentation.
Sample POC Kit on GitHub You can use the sample XML payloads with scripts on GitHub to deploy various functional definitions of NetScaler through Cisco APIC APIs. See https://github.com/citrix/netscaler_aci_poc_kit.
Troubleshooting You can troubleshoot any failures that might arise during deployment of the NetScaler device package in Cisco ACI by using:
The fault reports generated by Cisco APIC. The following logs generated by the device package: o debug.log o apic.log o periodic.log
APIC Fault Reports When you deploy a NetScaler device package in Cisco ACI, the Cisco APIC reports any failures. You can view the fault reports at any level of the APIC (for example, device, tenant, EPGs, or service graph). The screen shot below shows a fault report at the device level. For more information on faults, see http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1x/faults/guide/b_APIC_Faults_Errors/b_IFC_Faults_Errors_chapter_01.html Select any APIC entity and click the Faults tab to display the faults reported by APIC for that entity.
Logs Generated by Device Package The NetScaler device package generates configuration-related logs and monitoring-related logs. The generated logs are located at /data/devicescript/Citrix.NetScaler.1.0/logs as shown below.
Note: Cisco APIC runs in clusters of three nodes, and log details are captured only on the active node. You might have to check more than one APIC node to determine which one is capturing the logs.
Debug.log The Cisco APIC triggers various configuration events, such as serviceModify. It passes device and configuration payloads to the NetScaler ADC. These payloads are processed by the device script provided in the device package, and then the device script initiates various NITRO requests to NetScaler. The debug.log reports all the NITRO requests, and the responses from the device script to the NetScaler device. For any specific configuration issue, you can investigate the corresponding NITRO request and the response that the device script received from the device. You can also compare the logged details with the entries in the ns.log file on the NetScaler. Following is a sample log entry in the debug.log file. You could use it to trace configuration related issues: 2016-01-06 02:16:31.807981 DEBUG Thread-19 395166 [10.102.102.62, 8102] Add Attr col = {'ipv46': '10.2.2.2.', 'servicetype': 'HTTP', 'port': '80', 'lbmethod': 'ROUNDROBIN', 'name': 'testLbVserver_1'} 2016-01-06 02:16:31.808045 DEBUG Thread-19 395167 [10.102.102.62, 8102] ++++++++++++++++ This is to add NITRO Object ++++++++++++++++++ …. 2016-01-06 02:16:31.842175 DEBUG Thread-19 395169 [10.102.102.62, 8102] ------ add Nitro object ------------- Response = { "errorcode": 1110, "message": "Invalid IP address [10.2.2.2.]", "severity": "ERROR" }
… 2016-01-06 03:16:42.260617 DEBUG Thread-6 410970 [10.102.102.62, 8138] ++++++++++++++ ServiceAudit response = {'faults': [([(0, '', 52849), (4, 'lbvserver', 'lbvserver')], 1110, 'Invalid IP address [10.2.2.2.] SEVERITY:ERROR')], 'state': 2}
Apic.log The apic.log file captures all configuration requests from Cisco APIC and the request payload. The following is a sample of the request, payload, and response: request: serviceAudit{ 'args': ({ (0, '', 52849): { 'ackedstate': 0, 'ctxName': 'cokectx1', 'dn': u'uni/vDev-[uni/tn-coke_SDX2/lDevVip-ADCCluster1]-tn-[uni/tn-coke_SDX2]-ctx-cokectx1', 'state': 2, 'tenant': 'coke_SDX2', 'transaction': 0, 'txid': 10083, 'value': { (1, '', 9350): { 'absGraph': 'WebGraph',
'ackedstate': 0, 'rn': u'vGrp-[uni/tn-coke_SDX2/GraphInst_C-[uni/tn-coke_SDX2/brc-webCtrct1]-G-[uni/tn-coke_SDX2/AbsGraphWebGraph]-S-[uni]]', 'state': 2, 'transaction': 0, 'value': { (3, 'LoadBalancing', 'Node1'): { 'ackedstate': 0, 'state': 2, 'transaction': 0, 'value': { (2, 'external', 'outside'): { 'ackedstate': 0, 'state': 2, 'transaction': 0, 'value': { (9, '', 'ADCCluster1_outside_2785280_32773'): { 'ackedstate': 0, 'state': 0, 'target': 'ADCCluster1_outside_2785280_32773', 'transaction': 0 }, …. 2016-01-0603: 16: 42.261865DEBUGThread-6410971[ 10.102.102.62, 8138 ]result: serviceAudit{ 'result': { 'faults': [ ([ (0, '', 52849), (4, 'lbvserver', 'lbvserver') ], 1110, 'Invalid IP address [10.2.2.2.] SEVERITY:ERROR') ], 'state': 2 }, 'stats': { 'max': 37.48120903968811, 'num': 94, 'last': 34.02421307563782, 'avg': 34.25977123798208, 'min': 33.137107133865356 }
Periodic.log The periodic.log file captures all the monitoring related information. The Cisco APIC monitors the health of the device and service graph by periodically polling the device and service graph. These request details are captured in the periodic.log. Following is an example: 2016-01-0423: 46: 33.381518DEBUGThread-444084[ 10.102.102.62, 7092 ]request: serviceHealth{
'args': ({ (0, '', 52849): { 'ctxName': 'cokectx1', 'dn': u'uni/vDev-[uni/tn-coke_SDX2/lDevVip-ADCCluster1]-tn-[uni/tn-coke_SDX2]-ctx-cokectx1', 'state': 2, 'tenant': 'coke_SDX2', 'value': { (1, '', 9350): { 'absGraph': 'WebGraph', 'rn': u'vGrp-[uni/tn-coke_SDX2/GraphInst_C-[uni/tn-coke_SDX2/brc-webCtrct1]-G-[uni/tn-coke_SDX2/AbsGraphWebGraph]-S-[uni]]', 'state': 2, 'value': { (3, 'LoadBalancing', 'Node1'): { 'state': 2, 'value': { (2, 'external', 'outside'): { 'state': 2, 'value': { (9, '', 'ADCCluster1_outside_2785280_32773'): { 'state': 0, 'target': 'ADCCluster1_outside_2785280_32773' } } }, (2, 'internal', 'inside'): { 'state': 2, 'value': { (9, '', 'ADCCluster1_inside_2785280_49154'): { 'state': 0, 'target': 'ADCCluster1_inside_2785280_49154' } } }, (4, 'external_network', 'external_networkwebCtrct1WebGraph'): { 'connector': 'outside', 'state': 0, 'value': { (6, 'external_network_key', 'external_network_key'): { 'state': 0, 'target': 'network_webCtrct1WebGraph/snip2_webCtrct1WebGraph' } } }, … …. 2016-01-04 23:46:33.574321 DEBUG Thread-4 44123 [10.102.102.62, 7092] result: serviceHealth {'result': {'devs': 'ADC1', 'faults': [], 'health': [([(0, '', 52849), (1, '', 9350),
(3, 'LoadBalancing', 'Node1')], 0)], 'state': 0}, 'stats': {'max': 0.5484399795532227, 'num': 287, 'last': 0.2926321029663086, 'avg': 0.35803680968201534, 'min': 0.25844407081604004}} … …. 2016-01-06 03:30:53.851591 DEBUG Thread-16 411217 [10.102.102.63, 8146] result: deviceHealth {'result': {'faults': [], 'health': [([], 95)], 'state': 0}, 'stats': {'max': 0.5235550403594971, 'num': 1240, 'last': 0.44126415252685547, 'avg': 0.2513603793036553, 'min': 0.11344313621520996}}
FAQs
What is a fault? What is a function definition? What is the compatibility matrix between NetScaler Device Package and NetScaler Versions? What is the compatibility matrix between Cisco APIC and Device Package versions? What is the difference between inline and anywhere mode? What is the difference between one-arm and two-arm configurations that are pushed to the NetScaler? Does Cisco ACI store the configurations that APIC pushes to NetScaler appliances? Can I use APIC to perform an upgrade or downgrade of the NetScaler firmware? Can I use APIC to initiate a high-availability failover? Does Cisco APIC create dynamic VLANs for each virtual IP (VIP) address even if some of the VIPs are on the same subnet? What kind of monitoring support does APIC provide for a NetScaler appliance and its entities? Can I set up some configurations out-of-band while the NetScaler appliance is being managed through APIC? What are cluster and device configurations? What entities are present at each level? Is NetScaler SDX mixed mode design supported, that is, some instances are managed by APIC and others are managed manually/out-of-band? What features are not supported for APIC integration?
What is a fault? In Cisco APIC, a fault is a mechanism that reports failures in operations and the possible causes for the failures. The NetScaler device package constructs an appropriate fault whenever it encounters any NetScaler specific problem during deployment or while collecting the monitoring data. For more information about APIC faults, see: http://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1x/faults/guide/b_APIC_Faults_Errors/b_IFC_Faults_Errors_chapter_01.html What is a function definition? A Function definition is a collection of configurable NetScaler entities for a specific feature, such as Load Balancing. The function definition includes parameter values for a specific configuration of the feature. For more information, see Cisco ACI product documentation. NetScaler device package 129.62 provides 12 function definitions that simplify using Cisco APIC to configure a NetScaler ADC. In addition to feature-parameter values, these function definitions include the networking details, policies, and bindings that make the NetScaler data-path ready for the feature.
What is the compatibility matrix between NetScaler Device Package and NetScaler Versions? NetScaler Device package 10.1.129.62 supports NetScaler 10.1 features and functionalities. The device package is forward compatible, that is, the NetScaler version can be 10.1 or above. However, use with a later version (for example, NetScaler 10.5) is restricted to features available in release 10.1. Device Package
NetScaler Version
10.1 Build #129.62
10.1 and above
Released
What is the compatibility matrix between Cisco APIC and Device Package versions? The following is the current compatibility matrix: APIC Version 1.1(xx)
Device Package #129.62
Status Released
What is the difference between inline and anywhere mode? Inline mode uses two different interfaces. Traffic flows into one VLAN interface and out the other. Anywhere mode uses the same interface for all traffic. What is the difference between one-arm and two-arm configurations that are pushed to the NetScaler? The differences are as follows:
In one-arm mode only one SNIP address is created, but in two-arm mode two SNIP addresses are created. When you deploy L4-L7 devices in one-arm mode, each VLAN or interface is associated with both the consumer and the provider. In two-arm mode, one VLAN or interface is associated with the consumer, and another is associated with the provider. Does Cisco ACI store the configurations that APIC pushes to NetScaler appliances? Yes. ACI stores the pushed configurations. Can I use APIC to perform an upgrade or downgrade of the NetScaler firmware? No. A NetScaler firmware upgrade or downgrade can be done only out-of-band. Can I use APIC to initiate a high-availability failover? No. HA-failover initiation must be done out-of-band. Does Cisco APIC create dynamic VLANs for each virtual IP (VIP) address even if some of the VIPs are on the same subnet? No. What kind of monitoring support does APIC provide for a NetScaler appliance and its entities? The Cisco APIC monitors a NetScaler device and the deployed service graph by periodically polling for device and service graph health. For more information, see Monitoring NetScaler Device Health and Monitoring Service Graph Health. Can I set up some configurations out-of-band while the NetScaler appliance is being managed through APIC? You must not make any out-of-band modifications of NetScaler configurations supported by a device package. APIC might trigger a configuration audit that removes the out-of-band configuration. Is NetScaler SDX mixed mode design supported, that is, some instances are managed by APIC and others are managed manually/out-of-band? Yes. What are cluster and device configurations? What entities are present at each level? Cisco APIC supports configuration classifications at the following two levels:
Cluster o o o o Device o o
SNMP NTP Configuration Mode Feature Enable/Disable Rest of the Configuration, including Global, Policy, Network, Singleton, and all other configuration entities, such as load balancing virtual servers. Binding objects are parts of object definitions. For example, lbvserver_servicegoup_binding is part of an lbvserver definition.
What features are not supported for APIC integration? The following NetScaler features are not supported for APIC integration:
High availability SSL certificate management License management
The following table lists the command-line interface (CLI) commands that are not supported in APIC integration. Group AAA APPFLOW APPFLOW APPFLOW APPFLOW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPFW APPQOE APPQOE AUTHEN AUTHEN AUTHEN AUTHOR AUTHOR AUTOSCALE BASIC
Resource Name aaasession appflowaction appflowpolicy appflowpolicylabel appflowcollector appfwcustomsettings appfwhtmlerrorpage appfwarchive appfwarchive appfwprofile appfwprofile appfwlearningdata appfwsignatures appfwsignatures appfwpolicylabel appfwpolicy appfwxmlerrorpage appqoecustomresp appqoecustomresp authenticationvserver authenticationvserver authenticationvserver authorizationpolicy authorizationpolicylabel autoscalepolicy locationdata
Operation kill rename rename rename rename export update export import archive restore export import update rename rename update import update enable disable rename rename rename rename clear
CACHE CACHE CACHE CACHE CACHE CACHE CMP CMP CMP CR CR CR CS CS CS CS CS CS DB DNS DNS DNS DNS DNS DNS DNS GSLB GSLB GSLB GSLB GSLB GSLB LB LB LB LB LB LB LB LB LB LB LB LB
cachecontentgroup cachecontentgroup cacheobject cacheobject cachepolicy cachepolicylabel cmpaction cmppolicylabel cmppolicy crvserver crvserver crvserver csvserver csvserver csvserver csaction cspolicy cspolicylabel dbsmonitors dnskey dnsnameserver dnsnameserver dnsproxyrecords dnszone dnszone dnspolicylabel gslbldnsentries gslbconfig gslbservice gslbvserver gslbvserver gslbvserver lbpersistentsessions vserver vserver servicegroup servicegroup servicegroup lbmonitor lbmonitor service service service lbgroup
expire flush expire flush rename rename rename rename rename enable disable rename enable disable rename rename rename rename restart create enable disable flush sign unsign rename clear sync rename enable disable rename clear enable disable enable disable rename enable disable enable disable rename rename
LB LB LB LB LB LB NETWORK NETWORK NETWORK NETWORK NETWORK NETWORK NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS NS
lbvserver lbvserver lbvserver server server server route6 route Interface Interface Interface bridgetable nspbr nspbr nsacl nsacl nsacl nslimitsessions nsacls6 nsacls6 nsacls6 nstimer rnat6 nssurgeq nspbr6 nspbr6 nspbr6 nspbr6 nspbr6 nsdhcpip nsacl6 nsacl6 nsacl6 rnat nssimpleacl6 nssimpleacl6 nspbrs nspbrs nspbrs arp nsip nsip nssimpleacl nssimpleacl
enable disable rename enable disable rename clear clear clear enable disable clear enable disable enable disable rename clear clear apply renumber rename clear flush renumber enable disable clear apply release enable disable rename clear clear flush renumber clear apply send enable disable clear flush
NS NS NS NS NTP NTP OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL OPERATIONAL
nd6 nsacls nsacls nsacls ntpsync ntpparam reboot nsconfig nsconfig nsconfig nstrace nstrace shutdown systemsession systembackup
clear renumber clear apply enable/ disable set/unset reboot clear save diff start stop shutdown kill create/ restore/ remove
OPERATIONAL
systementitydata
rm
OPERATIONAL
nsaptlicense
update
OPERATIONAL OPERATIONAL OPERATIONAL RESPONDER RESPONDER RESPONDER RESPONDER RESPONDER REWRITE REWRITE REWRITE SNMP
reporting techsupport callhome responderaction responderpolicylabel responderpolicy responderhtmlpage responderhtmlpage rewritepolicy rewriteaction rewritepolicylabel snmpgroup
enable/ disable show set rename rename rename import update rename rename rename add / rm/ set/ unset
SNMP SNMP SNMP SPILLOVER
snmpmib snmpengineid snmpoption spilloverpolicy
set set set rename
SPILLOVER SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL SSL STREAM TD TD TD TRANSFORM TRANSFORM VPN VPN VPN WI WI
spilloveraction sslfipssimtarget sslfipssimtarget sslcert sslrsakey sslcertkey sslcertkey sslcertkey sslcrl ssldsakey sslpkcs8 sslfipssimsource sslfipssimsource ssldhparam snmpalarm snmpalarm sslfipskey sslfipskey sslfipskey sslcertreq sslfips sslwrapkey sslpkcs12 streamsession nstrafficdomain nstrafficdomain nstrafficdomain transformpolicylabel transformpolicy vpnvserver vpnvserver vpnvserver wipackage wipackage
rename enable init create create link unlink update create create convert enable init create enable disable create import export create update create convert clear clear enable disable rename rename enable disable rename install uninstall
