ASM rebalance operation never end [message #523458] |
Mon, 19 September 2011 00:19 |
luc_tran
Messages: 31 Registered: October 2010 Location: Viet Nam
|
Member |
|
|
Hi all,
I had added a new disk into disk group DATA1 with rebalance power of 5, it ran as estimated for about 2 days long, I can see the estimated time down to zero but the process never end.
The command used:
SQL> alter diskgroup DATA1 add disk '<path>' rebalance power 5;
Below is the disk status:
SQL> select group_number, disk_number, total_mb, free_mb, mount_status, header_status, state
from v$asm_disk where group_number=1 order by disk_number;
GROUP_NUMBER DISK_NUMBER TOTAL_MB FREE_MB MOUNT_S HEADER_STATU STATE
------------ ----------- ---------- ---------- ------- ------------ --------
1 0 1191626 314236 CACHED MEMBER NORMAL
1 1 1191626 314230 CACHED MEMBER NORMAL
1 2 1191626 314232 CACHED MEMBER NORMAL
1 3 1191602 314229 CACHED MEMBER NORMAL
1 4 1191626 314210 CACHED MEMBER NORMAL
1 5 1191626 314218 CACHED MEMBER NORMAL
1 6 1191626 314218 CACHED MEMBER NORMAL
1 7 1191602 314223 CACHED MEMBER NORMAL
1 8 1191626 314216 CACHED MEMBER NORMAL
1 9 1191626 314202 CACHED MEMBER NORMAL
1 10 1191626 314230 CACHED MEMBER NORMAL
GROUP_NUMBER DISK_NUMBER TOTAL_MB FREE_MB MOUNT_S HEADER_STATU STATE
------------ ----------- ---------- ---------- ------- ------------ --------
1 11 1191602 314237 CACHED MEMBER NORMAL
1 12 1191626 314229 CACHED MEMBER NORMAL
1 13 1191626 314221 CACHED MEMBER NORMAL
1 14 1191626 314220 CACHED MEMBER NORMAL
1 15 1191602 314225 CACHED MEMBER NORMAL
16 rows selected.
Disk 15 is the new one, current rebalance status:
SQL> select group_number, operation, state, power, est_minutes from v$asm_operation;
GROUP_NUMBER OPERA STAT POWER EST_MINUTES
------------ ----- ---- ---------- -----------
1 REBAL RUN 5 0
I've checked and follow the status for a few days more, it still running with EST_MINUTES=0. I think it was hang and never end.
During the next few weeks, I've manually restarted the rebalance process by modifying its power to 6, 8, 10
SQL> alter diskgroup DATA1 rebalance power 10;
After every restart, the diskgroup rebalance again but has the same issue, never end.
Here is the content in the alert log
NOTE: starting rebalance of group 1/0x9d2529a0 (DATA1) at power 5
Starting background process ARB0
Fri Sep 16 12:01:11 2011
ARB0 started with pid=15, OS id=21066
Starting background process ARB1
Starting background process ARB2
Fri Sep 16 12:01:11 2011
ARB1 started with pid=16, OS id=21068
Starting background process ARB3
Fri Sep 16 12:01:11 2011
ARB2 started with pid=17, OS id=21070
Starting background process ARB4
Fri Sep 16 12:01:11 2011
ARB3 started with pid=18, OS id=21072
Fri Sep 16 12:01:11 2011
ARB4 started with pid=19, OS id=21074
NOTE: assigning ARB0 to group 1/0x9d2529a0 (DATA1)
Fri Sep 16 12:21:23 2011
NOTE: assigning ARB1 to group 1/0x9d2529a0 (DATA1)
Fri Sep 16 12:21:41 2011
NOTE: assigning ARB2 to group 1/0x9d2529a0 (DATA1)
NOTE: assigning ARB3 to group 1/0x9d2529a0 (DATA1)
NOTE: assigning ARB4 to group 1/0x9d2529a0 (DATA1)
As you can see, the rebalance process started 3 days ago, hanging till now
SQL> select group_number, operation, state, power, est_minutes from v$asm_operation;
GROUP_NUMBER OPERA STAT POWER EST_MINUTES
------------ ----- ---- ---------- -----------
1 REBAL RUN 5 0
Could you please tell me how to solve this problem, thanks a lot!
Luc
[Updated on: Mon, 19 September 2011 00:45] by Moderator Report message to a moderator
|
|
|
|
|
|