Flash disk replacement in Exadata

Steps to replace flash disk in exadata:

The exadata has the compute nodes and the storage/cell node attached to it. In case of a damage of the flash modules(fmod) on flash cards on Exadata cellnodes, the flash cache will not function normally and there will be performance problems.

Idenfify on which cell node the flash disk is failed. For the flash disk replacement we need to bring down the that particular cell nodes.

Login to compute node and check which cell node has faulty disk.

1. To know which cell node has faulty disk:

#dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root 'cellcli -e list physicaldisk'

2. After idenfifying the cell node then login to it.

#cellcli -e list physicaldisk --> to know whether normal disk is failed or flash disk is failed

You will see something like this

(FLASH_1_0 15557M04E3N warning - poor performance)

3. If flash disk failed then 

#cellcli -e list flashcache detail --> to know degradedCelldisks, effectiveCacheSize and status of the faulty disk.

4. Use cellcli utility and make all griddisks inactive from that cell node

CellCLI> alter griddisk all inactive

5. check the status of the griddisks:

CellCLI> list griddisk attributes name,asmmodestatus,asmdeactivationoutcome --> you will see offline status

6. Bring down the cell node.

#init 0

7. Hand it over the cell node for flash disk replacement

8. Once the disk replacement done and cell node is up then login to it and check disk status:

#cellcli -e list physicaldisk  --> you will see normal disk and flash disk status as "normal"

9. Check the flash cache.

#cellcli -e list flashcache detail 

10. Reactivate the griddisks:

CellCLI> alter griddisk all active

11. check status of griddisks

# CellCLI> list griddisk attributes name,asmmodestatus,asmdeactivationoutcome --> All should be online

Regards,

Kiren Jadhav