Here is some tuning tips for GPFS and Oracle RAC
1) Do not use the "dio" mount option for the GPFS file system
2) If using RAID devices, configure a single LUN for each RAID device. Do not create LUNs across RAID devices for use by GPFS as this will ultimately result in significant loss in performance as well as making the removal of a bad RAID more difficult. GPFS will stripe across the multiple LUNs (RAIDs) using its own optimized method.
3) For Oracle RAC databases, set the GPFS file system block using the "mmcrfs" command and the "-B" option, to a large value using the following guidelines:512 KB is generally suggested.
4) GPFS Threads
Use the following guidelines to set the GPFS "worker threads" to allow the maximum parallelism of the Oracle AIO threads, and the GPFS "prefetch threads" to benefit Oracle sequential I/O.
On a 64-bit AIX kernel:
GPFS worker threads can be <= 548.
GPFS worker threads + GPFS prefetch threads <= 550.
When requiring GPFS sequential I/O, set the prefetch threads between 50 and 100 (the default is 64), and set the worker threads to have the remainder.
5) For Oracle RAC node recovery to work correctly, GPFS must be configure to be automatically loaded at boot time and automatically mount the GPFS file systems. Use the following two GPFS commands to configure this:
root@raven:64bit /> mmchconfig autoload=yes
root@raven:64bit /> mmchfs /dev/oragpfs -A yes
mmchfs: 6027-1371 Propagating the changes to all affected nodes.
This is an asynchronous process.
References : Oracle Metalink Document : 302806.1