Oracle Real Application Clusters (RAC) requires all the nodes to run the same Operating System binary in a cluster. All nodes must be the same architecture (I.E. All nodes must be either 32 bit or all nodes must be 64 bit or all nodes must be HP-UX PARISC since you cannot mix PARISC with Itanium).
Oracle RAC does support a cluster with nodes that have different hardware configurations. An example is a cluster with 3 nodes with 4 CPUs and another node with 6 CPUs. This can easily occur when adding a new node after the cluster has been in production for a while. For this type of configuration, customers must consider some additional features to get the optimal cluster performance. The servers used in the cluster can be from different vendors; this is fully supported as long as they run the same binaries. Since many customers implement Oracle RAC for high availability, you must make sure that your hardware vendor will support the configuration. If you have a failure, will you get support for the hardware configuration?
The installation of Oracle Clusterware expects the network interface to be the same name on all nodes in the cluster. If you are using different hardware, you may need to work with your operating system vendor to make sure the network interface names are the same name on all nodes (IE eth0).
Customers implementing uneven cluster configurations need to consider how they will balance the workload across the cluster. Some customers have chosen to manually assign different workloads to different nodes. This can be done using database services however it is often difficult to predict workloads and the system cannot dynamically react to changes in workload. Changes to workload require the DBA to modify the service. You will also need to consider how you will survive failures in the cluster. Will the service levels be maintained if the larger node in the cluster fails? Especially in a small cluster, the impact of losing a node could impact the ability to continue processing the application workload.
The impact of the different sized nodes depends on how much difference there is in the size. If there is a large difference between the nodes in terms of memory and CPU size, than the "bigger" nodes will attract more load, obviously, and in the case of failure the "smaller" node(s) will become overpowered. In such a case, static routing of workload via services e.g. batch and certain services, which can be suspended/stopped if the large node fails and the cluster has significantly reduced capacity, may be advisable. The general recommendation is that the nodes should be sized in such a way that the aggregated peak load of the large node(s) can be absorbed by the smaller node(s), i.e. smaller node should have sufficient capacity to run the essential services alone. Another option is to add another small node to the cluster on demand in case that the large one fails.
It should also be noted especially if there is a large difference between the sizes of the nodes, the small nodes can slow down the larger node. This could be critical one if the smaller node is very busy and must serve data to the large node.
To help balance workload across a cluster, Oracle RAC 10g Release 2 and above provides the Load Balancing Advisory (LBA). The load balancing advisory runs in an Oracle RAC database and monitors the work executed by the service on all instances where the service is active in the cluster. The LBA provides recommendations to the subscribed clients about the state of the service and where the client should direct connection requests. Setting the GOAL on the service activates the load balancing advisory. Clients that can utilize the load balancing advisory are Oracle JDBC Implicit Connection Cache, Oracle Universal Connection Pool for Java, Oracle Call Interface Session Pool, ODP.NET Connection Pool, and Oracle Net Services Connection Manager. The Oracle Listener also uses the Load Balancing Advisory if CLB_GOAL parameter is set to SHORT (recommended Best Practice if using an integrated Oracle Client mentioned here). If CLB_GOAL is set to LONG (default), the Listener will load balance the number of sessions for the service across the instances where the service is available. See the Oracle Real Application Clusters Administration and Deployment Guide for details on implementing services and the various parameter settings.
Thursday, March 26, 2009
Wednesday, July 9, 2008
Cluster Managed Services
With Oracle RAC 10g, Oracle introduced Cluster Managed Services. This allows a dba to manage the workload running in their Oracle RAC database. Within the database, there are several attributes to a service which allow you to define performance characteristics for the workload. In an Oracle RAC environment, you can manage where the service runs by managing the parameters stored in the Oracle Clusterware resource. You can also monitor performance by service.
Many features of the Oracle Database are integrated with services. Oracle Streams, creates a service for each queue. This allows the queue to be accessed by the application independent of which instance the capture and apply processes are running. These services are created automatically by the database when you create the queue and should not be manaually dropped (or your queue may not work any more). The Database will clean up the service when you delete the queue. Oracle Scheduler can automatically assign a job class depending on the service used. Users connected to a service can automatically be mapped to a resource manager group. Oracle XMLDB will create its own services. These services are managed by the database and can not be stopped/started by the dba like user created services. A simple rule of thumb you can use, is if you did not explicitly create the service, you cannot manage the service.
The attributes for a service in the database include name, goal, clb_goal, and TAF policy. The goal is used to take advantage of the load balancing advisory. In an Oracle RAC database, we keep track of the work in the cluster for a given service. Based on the current workload in the cluster, Oracle RAC recommends how to distribute incoming connections across the active instances providing a given service. Clients who listen to the load balancing advisory can then allocate work across the cluster based on the current workload. Clients integrated with the load balancing advisory are Oracle JDBC Implicit Connection Cache, Oracle ODP.Net Connection Pool, Oracle OCI Session Pool, Oracle Connection Manager, and Oracle Listener. The clb_goal is the connection load balancing goal which tells the listener what type of connections it is dealing with. Short is for applications using the load balancing advisory (IE an Oracle Connection Pool) or applications that connect to the database for short sessions. Long is for applications that connect to the database for long periods (such as Oracle Forms applications) or when using connection pools that are not integrated with the load balancing advisory. The addition of the TAF policy to the service makes it much easier to administer TAF. All users connecting with the service will automatically get the TAF policy, you do not have to edit the client connection string. The Service TAF policy will override the client TAF policy. You must use DBMS_SERVICE pl/sql package to edit these attributes. For goal and clb_goal, the EM DBControl cluster managed services page will use PL/SQL under the covers to edit these attributes.
If your application is using XA transactions, there is a special attribute on the service in the database. This is a DTP service (DTP for distributed transaction processing). When using XA transactions with Oracle RAC 10g, you must direct all branches of the global transaction to a single instance in the cluster. Oracle RAC will ensure a DTP service is only active on one instance and when the service fails over, we will ensure that the first level of recovery has completed before activating the service on another instance. This type of service is sometimes referred to as singleton. This attribute is set with the DBMS_SERVICE package or using the EM DBControl Cluster Managed Services Page.
If you are using Fast Connection Failover for OCI based clients (OCI and ODP.NET), you must set the AQ_HA_NOTIFICATIONS attribute for the service. This tells the database to publish the FAN HA events to the internal queue for OCI. This attribute is set with the DBMS_SERVICE package or using the EM DBControl Cluster Managed Services Page.
In a cluster, you can define a service to run on a subset of the Oracle RAC instances. A service has its preferred instances (instances where it should run during normal operations) and available instances (instances where it should run if a preferred instance fails). If a preferred instance fails, the service is moved to an available instance. The cluster will try to keep the service running on the same number of instances as defined in the preferred attribute. We refer to this as the cardinality of the service. If the service is running on an available instance, and the preferred instance is restarted, you will have to manually relocate the service back to the preferred instance. This can be automated through a server callout, see the OTN Sample Code Page for RAC for examples.
For more details on services and using the different attributes, see the whitepaper Automatic Workload Management with Oracle RAC 10g or Automatic Workload Management with Oracle RAC 11g available on OTN (otn.oracle.com/rac). Also read the Oracle Real Applciation Clusters Administration and Deployment Guide.
Many features of the Oracle Database are integrated with services. Oracle Streams, creates a service for each queue. This allows the queue to be accessed by the application independent of which instance the capture and apply processes are running. These services are created automatically by the database when you create the queue and should not be manaually dropped (or your queue may not work any more). The Database will clean up the service when you delete the queue. Oracle Scheduler can automatically assign a job class depending on the service used. Users connected to a service can automatically be mapped to a resource manager group. Oracle XMLDB will create its own services. These services are managed by the database and can not be stopped/started by the dba like user created services. A simple rule of thumb you can use, is if you did not explicitly create the service, you cannot manage the service.
The attributes for a service in the database include name, goal, clb_goal, and TAF policy. The goal is used to take advantage of the load balancing advisory. In an Oracle RAC database, we keep track of the work in the cluster for a given service. Based on the current workload in the cluster, Oracle RAC recommends how to distribute incoming connections across the active instances providing a given service. Clients who listen to the load balancing advisory can then allocate work across the cluster based on the current workload. Clients integrated with the load balancing advisory are Oracle JDBC Implicit Connection Cache, Oracle ODP.Net Connection Pool, Oracle OCI Session Pool, Oracle Connection Manager, and Oracle Listener. The clb_goal is the connection load balancing goal which tells the listener what type of connections it is dealing with. Short is for applications using the load balancing advisory (IE an Oracle Connection Pool) or applications that connect to the database for short sessions. Long is for applications that connect to the database for long periods (such as Oracle Forms applications) or when using connection pools that are not integrated with the load balancing advisory. The addition of the TAF policy to the service makes it much easier to administer TAF. All users connecting with the service will automatically get the TAF policy, you do not have to edit the client connection string. The Service TAF policy will override the client TAF policy. You must use DBMS_SERVICE pl/sql package to edit these attributes. For goal and clb_goal, the EM DBControl cluster managed services page will use PL/SQL under the covers to edit these attributes.
If your application is using XA transactions, there is a special attribute on the service in the database. This is a DTP service (DTP for distributed transaction processing). When using XA transactions with Oracle RAC 10g, you must direct all branches of the global transaction to a single instance in the cluster. Oracle RAC will ensure a DTP service is only active on one instance and when the service fails over, we will ensure that the first level of recovery has completed before activating the service on another instance. This type of service is sometimes referred to as singleton. This attribute is set with the DBMS_SERVICE package or using the EM DBControl Cluster Managed Services Page.
If you are using Fast Connection Failover for OCI based clients (OCI and ODP.NET), you must set the AQ_HA_NOTIFICATIONS attribute for the service. This tells the database to publish the FAN HA events to the internal queue for OCI. This attribute is set with the DBMS_SERVICE package or using the EM DBControl Cluster Managed Services Page.
In a cluster, you can define a service to run on a subset of the Oracle RAC instances. A service has its preferred instances (instances where it should run during normal operations) and available instances (instances where it should run if a preferred instance fails). If a preferred instance fails, the service is moved to an available instance. The cluster will try to keep the service running on the same number of instances as defined in the preferred attribute. We refer to this as the cardinality of the service. If the service is running on an available instance, and the preferred instance is restarted, you will have to manually relocate the service back to the preferred instance. This can be automated through a server callout, see the OTN Sample Code Page for RAC for examples.
For more details on services and using the different attributes, see the whitepaper Automatic Workload Management with Oracle RAC 10g or Automatic Workload Management with Oracle RAC 11g available on OTN (otn.oracle.com/rac). Also read the Oracle Real Applciation Clusters Administration and Deployment Guide.
Friday, May 9, 2008
Questions from the Expert Panel at IOUG
The Oracle RAC SIG (www.oracleracsig.org) held an expert panel at their recent Collaborate 08 conference in Denver. Here are the questions (along with the answers)
- Do you support Virtualization - Today Oracle RAC is supported on most hardware virtualization offering such as IBM LPAR, HP VPAR, SUN GLOBAL CONTAINERS, SUN E25K Partitions. Check certify on Oracle Metalink for details. Today Oracle RAC is not certified on any software virtualization such as VMWARE or Oracle VM.
- How large a cluster can I have? Oracle RAC supports up to 100 instances and Oracle Clusterware supports up to 100 nodes in a cluster. How many nodes you implement depends on your requirements for the cluster. Remember if I only have 2 nodes in the cluster, I lose 1, I have lost 50% of my processing power and 50% of my users will be affected. If I have a 4 node cluster, I loase a node, I will only affect 25% of my users and have lost only 25% of my processing power. Scalability of the application depends on how well written the application is (as in all situations!). Many customers today run in 4, 8, 16 or more nodes.
- Should I consolidate multiple applications in a cluster? Yes, cluster consolidation is a great way to get started on the grid and share resources amoung apps. It is easy to adjust the number of instances running for a given application within the cluster. Many customers take advantage of this method today.
- How do I upgrade from 9i RAC to Oracle RAC 10g? There are many options open for upgrade depending on how long an outage you can take and how big your database is. Check the Oracle documentation for details. There is also a new upgrade companion available in Metalink. If you would like to remove 3rd party clusterware from the system, you should completely de-install before installing Oracle Clusterware.
- How much traffic on the interconnect is Oracle Clusterware vs Oracle RAC? Oracle Clusterware sends small messages so it is a fairly small bandwidth user. Oracle RAC usage depends on the application and how much data is shared between SGAs. Depending on the blocksize of your database, the database transfered is messages as well as data blocks.
- Should I use connection load balancing, what should be set up? In an Oracle RAC environment, yes you should use SQL*NET connection load balancing. If you use DBCA to create your database, it will set it up for you. This will load balance the client connections across the instances that are currently running the service. If you want better load balancing and fast failover of connections, have a look at the OTN whitepaper linked below
Thursday, April 24, 2008
Welcome
After several people telling me I should start blogging, I thought I should give it a try. Unfortunately life may prove to busy to keep it up however I will see how it goes!
It is always good to start with something you know so since my world is all about Oracle RAC, I figured I would start there. This will be dedicated to talking about things that people ask about based on the premise that if people are asking about it, it must be of concern to many of you out there! Feel free to ask questions or suggest topics for future.
Barb :)
It is always good to start with something you know so since my world is all about Oracle RAC, I figured I would start there. This will be dedicated to talking about things that people ask about based on the premise that if people are asking about it, it must be of concern to many of you out there! Feel free to ask questions or suggest topics for future.
Barb :)
Subscribe to:
Posts (Atom)