Tuesday, January 19, 2010

ORA-27061: waiting for async I/Os failed on AIX

When you see this error and the database crashes , hangs or does not shutdown/startup then there is a problem with Async I/O servers on AIX.

Syptomps

The following messages in the alert log
-------------------------------------------------------------------
.............
Mon Oct 8 22:57:05 2007
Errors in file /dwrac/BDUMP/dwhp_p009_1941690.trc:
ORA-27061: waiting for async I/Os failed <<<<
IBM AIX RISC System/6000 Error: 22: Invalid argument

Users affected:
* systems running aix 5.3 with the bos.rte.aio fileset at the 5.3.0.60 or 5.3.0.61 level.
* this problem is known to affect oracle.

Problem description:
When the maximum requests (maxreqs) is exceeded, lio_listio will update the control blocks, but may not return an error to indicate that some i/os were not started. this results in an unexpected einval from aio_nwait later when these i/os are waited for.

* errors similar to the following may be logged by oracle:
* ora-27061: waiting for async i/os failed
* ibm aix risc system/6000 error: 22: invalid argument



There are 3 actions to try :

1) Increase the number of aios and the maxreqs

Check aio the current settings:
aioo -a
or
lsattr –El aio0

Set maxreqs at least 8192.

chdev -P -l aio0 -a maxreqs=8192
or
aioo -p -o maxreqs=8192 (AIX 5.3 and up)

If this does not resolve the problem increase the number of min/max aio servers

chdev -P -l aio0 -a maxservers=[new value] -a minservers=[new value]

To make changes persistance on reboot

aioo -p -o maxservers=[new value] -o minservers=[new value]

To apply new values online (AIX 5.3 and up)

2) Decrease the parallel_min_servers and parallel_max_servers on the failing instance.

FOR RAC

ALTER SYSTEM SET parallel_min_servers = [NUM] SCOPE=BOTH SID='MYINSTANCE1';
ALTER SYSTEM SET parallel_max_servers = [NUM] SCOPE=BOTH SID='MYINSTANCE1';

FOR SINGLE DB
ALTER SYSTEM SET parallel_min_servers = [NUM] SCOPE=BOTH;
ALTER SYSTEM SET parallel_max_servers = [NUM] SCOPE=BOTH;

3) IBM FIX for AIX 5.3 ML6 and UP
Install fix for apar iz03260



REFERENCES
Metalink Note 464768.1
http://www-01.ibm.com/support/docview.wss?uid=isg1IZ03260

No comments:

Post a Comment