Oracle Real Application Cluster (RAC) SERIES 1 - Calgary Oracle ...

Oracle Clusterware 11GR2 Presented By : Qari Kamran Siddique Senior Database Consultant CGI

What is CLUSTER Enables Servers to Communicate with each other as a COLLECTIVE UNIT. A software that make clustered hardware to run multiple instances against ONE database . Database files are stored on disks that are either physically or logically connected to each node. Cluster Software hides the structure. Disks are available for read and write by all nodes. Operating system is the same on each machine. This architecture enables users and applications to benefit from the processing power of multiple machines. In case of crash of one node or instance, application can still access to the surviving node.

Benefits Scalability of applications Use of less expensive commodity hardware Ability to fail over Ability to increase capacity over time by adding servers Ability to program the startup of applications in a planned order Ability to monitor processes and restart them if they stop Resource Control

More Benefits Eliminate unplanned downtime due to hardware failures. Reduce or eliminate planned downtime for software maintenance. Increase throughput for cluster-aware applications Reduce the total cost of ownership

Basic RAC Components Oracle 10g R1,R2,11g R1 Oracle Clusterware Shared Storage Oracle RAC Database

Basic RAC Components (Oracle 11g R2) Grid Infrastructure RAC Database

Oracle Clusterware Hardware Concepts and Requirements One or more servers connected with each other with a network, called “INTERCONNECT” At least two network interface cards: one for a public network and one for a private network The interconnect network is a private network using a switch (or multiple switches) that only the nodes in the cluster can access No not support using crossover cables At least two network interfaces for the public network, bonded to provide one address At least two network interfaces for the private interconnect network Oracle Clusterware supports NFS, iSCSI, Direct Attached Storage DAS), Storage Area Network (SAN) storage, and Network Attached Storage (NAS).

Oracle Clusterware Hardware Concepts and Requirements (Continue) Consider the I/O requirements of the entire cluster when choosing your storage subsystem. At least one local disk that is internal to the server This disk is used for the operating system and Oracle Software binaries Increase HA by providing safe side for binary corruption Allows rolling upgrades, which reduce downtime.

Oracle Clusterware Operating System Concepts and Requirements (Product Certification)

Software Concepts  Voting Disks  Oracle Clusterware uses voting disk files to determine      

which nodes are members of a cluster. Can be configured on Oracle ASM ,or on shared storage ( Raw volumes). In case of ASM, redundancy level defines number of voting disks Without ASM => Minimum THREE voting disks for HA Use external redundancy Do not use more than five voting disks The maximum number of voting disks that is supported is 15.

Software Concepts  Oracle Cluster Registry (OCR)  Store and manage information about the components

    

that oracle clusterware controls , e.g; Rac Database, listeners, virtual IP addresses (VIPs), services, applications. Can be configured on Oracle ASM ,or on shared storage ( Raw volumes) stores configuration information in a series of key-value pairs in a tree structure. multiple OCR locations (multiplexing) should be defined You can have up to five OCR locations Each OCR location must reside on shared storage that is accessible by all of the nodes in the cluster

Software Concepts

Virtual Internet Protocol Address (VIP)

Oracle RAC requires a virtual IP address for each server in the cluster. It is an unused IP address on the same subnet as the Local Area Network (LAN). This address is used by applications to connect to the RAC database (NOT 11G R2). If a node fails, the Virtual IP is failed over to another node in the cluster to provide an immediate node down response to connection requests.

Oracle Clusterware Network Configuration Concepts Grid Infrastructure through the self-management of the network requirements for the cluster. Oracle Clusterware 11g release 2 (11.2) supports the use of dynamic host configuration protocol (DHCP) for all private interconnect addresses, as well as for most of the VIP addresses. DHCP provides dynamic configuration of the host's IP addresses. Addition of the Oracle Grid Naming Service (GNS) to the cluster=========11gR2Clusterware

Oracle Clusterware Network Configuration Concepts (Continue) Grid Naming Service (GNS)  Linked to the corporate Domain Name Service (DNS)  Clients can easily connect to the cluster.  Requires DHCP service on the public network.  Obtain an IP address on the public network for the GNS VIP.  DNS uses the GNS VIP to forward requests to the cluster .  Delegate a subdomain in the network to the cluster  Subdomain forwards all requests for addresses in the subdomain to the

GNS VIP.

Grid Naming Service (GNS Reference DNS and DHCP Setup Example for Grid Infrastructure GNS [ID 946452.1]

Network Configuration Concepts (Continue)  Single Client Access Name (SCAN)  Virtual hostname to provide for all clients connecting to the cluster (as 

  



opposed to the vip hostnames in 10g and 11gR1). Domain name registered to at least one and up to three IP addresses, either in the domain name service (DNS) or the Grid Naming Service (GNS). By default, the name used as the SCAN is also the name of the cluster. For installation to succeed, the SCAN must resolve to at least one address. Do not configure SCAN VIP addresses in the hosts file. But if you use the hosts file to resolve SCAN name, you can have only one SCAN IP address If hosts file is used, Cluster Verification Utility failure at end of installation.

Network Configuration Concepts (Continue)  DNS

Round Robin resolution to three addresses – RECOMMENDED  Add/remove nodes without reconfiguring clients  Adds location independence for the databases, so that client configuration does not have to depend on which nodes are running a particular database.  local listener LISTENER on all nodes to listen on local VIP, and SCAN listener LISTENER_SCAN1 (up to three cluster wide) to listen on SCAN VIP(s) system/manager@cgi1-scan:1521/apps jdbc:oracle:thin:@cgi-scan:1521/apps

Network Configuration Concepts (Continue) Sample TNS entry for SCAN TEST. CGI.COM = (DESCRIPTION= (ADDRESS=(PROTOCOL=tcp)(HOST=SCAN-TEST.CGI.COM)(PORT=1521)) (CONNECT_DATA=(SERVICE_NAME=11GR2TEST.CGI.COM)) )

Sample TNS entry without SCAN TEST.CGI.COM = (DESCRIPTION= (ADDRESS_LIST= (ADDRESS=(PROTOCOL=tcp)(HOST=TEST1-vip.CGI.COM)(PORT=1521)) (ADDRESS=(PROTOCOL=tcp)(HOST=TEST2-vip.CGI.COM)(PORT=1521)) ) (CONNECT_DATA=(SERVICE_NAME=11GR2TEST.CGI.COM)) )

Network Configuration Concepts (Continue) The node VIP and the three SCAN VIPs are obtained from the DHCP server when using GNS. If a new server joins the cluster, then Oracle Clusterware dynamically obtains the required VIP address from the DHCP server, updates the cluster resource, and makes the server accessible through GNS. $ srvctl config scan SCAN name: cgi-scan, Network:192.168.182.0/255.255.255.0/ SCAN VIP name: scan1, IP: /192.168.182.109 SCAN VIP name: scan2, IP: /192.168.182.110 SCAN VIP name: scan3, IP: /192.168.182.108

Node Name

Instance Database Name Name

cginode1

cgirac1

cginode2

cgirac2

cgi.dbservices.ca

Node Name

Public IP

Private IP

cginode1

192.168.1.151

192.168.2.1 192.168.1.153

cginode2

192.168.1.152

192.168.2.2 192.168.1.154

SCAN NAME

IP

SCAN VIP1

192.168.2.201

SCAN VIP2

192.168.2.202

SCAN VIP3

192.168.2.203

VIP

Oracle Clusterware startup sequence

Do not worry.. that is the Clusterware's job! image from the “Oracle Clusterware Administration and Deployment Guide)”

Oracle Grid Infrastructure Grid HOME Grid Infrastructure home => Oracle ASM + Oracle Clusterware Single Oracle home for both OCR and voting disk files can be placed either on Oracle ASM,or on a cluster file system or NFS system Installing Oracle Clusterware files on raw or block devices is no longer supported

Oracle Grid Infrastructure Grid HOME Oracle Clusterware and Oracle ASM are installed into a single home directory , which is called Grid Home # su - grid ORACLE_SID=+ASM1; export ORACLE_SID ORACLE_BASE=/u01/app/grid; export ORACLE_BASE # Specifies the directory containing the Oracle Grid Infrastructure software.

ORACLE_HOME=/u01/app/11.2.0/grid export ORACLE_HOME

Oracle Automatic Storage Management Cluster File System (Oracle ACFS) new multi-platform, scalable file system and storage management solution provides dynamic file system resizing improved performance provides storage reliability through the mirroring and parity protection Oracle ASM provides.

Cluster Time Synchronization Service Ensures that there is a synchronization service in the cluster. Network Time Protocol (NTP) is not found during cluster configuration, then CTSS is configured to ensure time synchronization.

Mandatory OS Users and Groups Oracle Inventory Group (typically, oinstall) => Must be the primary group for Oracle Software installation owners. Oracle Software Owner => Typically oracle OSDBA group => typically, dba for Database authentication. (SYSDB A + SYSAM)

Recommended Approach for OS Users and Groups Reference (Oracle Grid Infrastructure installation guide Grid Infrastructure software owner => GRID Oracle RAC Software owner => ORACLE Separate group for Oracle ASM => OSASM group Members of this group would connect to ASM by using sysasm O/S authentication ASM Database Administrator group (OSDBA) => Members of the OSDBA group for Oracle ASM are granted read and write access to files managed by Oracle ASMThe Oracle Automatic Storage Management Group (typically asmadmin) OSOPER for Oracle ASM group (typically asmper) => Member of this group are granted access to a subset of the SYSASM privileges.

Example of Creating Role-allocated Groups, Users, and Paths # groupadd -g 1000 oinstall # groupadd -g 1020 asmadmin # groupadd -g 1021 asmdba # groupadd -g 1031 dba1 # groupadd -g 1041 dba2 # groupadd -g 1022 asmoper # useradd -u 1100 -g oinstall -G asmadmin,asmdba grid # useradd -u 1101 -g oinstall -G dba1,asmdba oracle1 # useradd -u 1102 -g oinstall -G dba2,asmdba oracle2 # mkdir -p /u01/app/11.2.0/grid # mkdir -p /uo1/app/grid # chown -R grid:oinstall /u01 # mkdir -p /u01/app/oracle1 # chown oracle1:oinstall /u01/app/oracle1 # mkdir -p /u01/app/oracle2 # chown oracle2:oinstal

Oracle Base Directory path # mkdir -p /u01/app/11.2.0/grid #chown grid:oinstall /u01/app/11.2.0/grid #chmod -R 775 /u01/app/11.2.0/grid # mkdir -p /u01/app/oracle #chown -R oracle:oinstall /u01/app/11.2.0/oracle #chmod -R 775 /u01/app/11.2.0/oracle

Storage Options

What’s Next !!! Administering Oracle Clusterware, ASM and RAC databases Oracle RAC Backup and Recovery RAC Services RAC , Oracle Clusterware and ASM tuning Adding and Deleting RAC Nodes Patch Management in RAC Oracle Clusterware Cloning Application high availability with clusterware Oracle Clusterware utilities usage Whole clusterware stack upgrade to 11g R2 RAC + Clusterware + ASM …tips & tricks…………………..and Troubleshooting

Questions ???