Labels

hpunix (63) marathi kavita (52) linux (21) solaris11 (11) AWS (5) numerology (5)
Showing posts with label command. Show all posts
Showing posts with label command. Show all posts

Monday, July 13, 2026

Dynamically Expanding CPU Resources for a Production Zone


Oracle Solaris Resource Pools: Dynamically Expanding CPU Resources for a Production Zone

One of the powerful capabilities of Oracle Solaris is the ability to allocate dedicated CPU resources to critical workloads using Resource Pools and Processor Sets. Recently, I had to increase the CPU allocation for a clustered production zone without causing any downtime.

This article walks through the process of reviewing the existing configuration, backing up the resource pool configuration, modifying the processor set, and activating the changes online.


Environment Overview

The server hosts multiple Solaris zones:

root@solaris-node01# zoneadm list -cv

ID NAME STATUS PATH BRAND IP
0 global running / solaris shared
1 appzone01 running /zones/appzone01 solaris10 excl
2 prodcluster01 running /zones/prodcluster01 solaris excl
3 clonezone01 running /zones/clonezone01 solaris excl
4 clonezone02 running /zones/clonezone02 solaris excl


Reviewing the Production Zone Configuration

The target zone was prodcluster01.

root@solaris-node01# zonecfg -z prodcluster01 info

zonename: prodcluster01
zonepath: /zones/prodcluster01
brand: solaris
autoboot: true
pool: prodpool
scheduling-class: FSS

[max-shm-memory: 64G]
[max-shm-ids: 4096]
[max-sem-ids: 4096]

net:
physical: vnic0
net:
physical: vnic1

attr:
name: cluster
type: boolean
value: true
``

Notable configuration items:

  • Dedicated Resource Pool
  • Fair Share Scheduler (FSS)
  • Cluster-enabled zone
  • 64 GB shared memory allocation
  • Multiple network interfaces for cluster traffic

Reviewing Current Pool Allocations

Before making any resource changes, check the current pool distribution.

root@solaris-node01# poolstat

pset
id pool size used load
1 prodpool 32 0.00 2.93
0 pool_default 80 0.00 1.38
2 dbpool 16 0.00 0.56

Current CPU allocation:

PoolCPUs
prodpool32
dbpool16
pool_default80

The production zone currently owns a dedicated processor set containing 32 CPUs.


Verify Processor Set Configuration

Check the current processor set definition.

poolcfg -c 'info pset prodpset'

Output:

pset prodpset

uint pset.min 32
uint pset.max 32

This indicates that the processor set is statically configured with 32 CPUs.


Backup the Existing Configuration

Before making changes, always save the current configuration.

pooladm -s /var/tmp/poolcfg-backup-$(date +%Y%m%d)

Verification:

ls -lrt /var/tmp/poolcfg-backup-20260427

-rw-r--r-- 1 root root 19.7K Apr 27 14:18
/var/tmp/poolcfg-backup-20260427

Having a backup provides a quick rollback option if necessary.


Increasing CPU Allocation

The requirement was to increase CPU resources allocated to the production cluster zone.

Current allocation:

32 CPUs

Target allocation:

48 CPUs

Modify the processor set:

poolcfg -c 'modify pset prodpset (uint pset.min=48; uint pset.max=48)'

At this point, the change is stored in the configuration repository but has not yet been activated.


Verify Before Activating

Checking immediately after the modification:

poolstat

pset
id pool size used load
1 prodpool 32 0.00 3.92
0 pool_default 80 0.00 1.26
2 dbpool 16 0.00 0.58

The active configuration still shows 32 CPUs because the updated configuration has not yet been committed.


Activate the New Configuration

Apply the modified resource pool configuration.

pooladm -c

This operation dynamically reallocates CPUs without requiring a system reboot.


Verify the Results

Check the allocations again:

poolstat

pset
id pool size used load
1 prodpool 48 0.00 4.42
0 pool_default 64 0.00 1.58
2 dbpool 16 0.00 0.56

Success.

The production processor set increased from:

32 CPUs → 48 CPUs

and the CPUs were automatically taken from the default pool:

pool_default: 80 CPUs → 64 CPUs

All changes were made online with no interruption to the running zone.


Lessons Learned

Always Backup First

Before modifying any resource pool configuration:

pooladm -s <backup_file>

Having a rollback option is a best practice in production environments.

Changes Are Not Active Until Committed

Modifying the processor set changes only the configuration database.

poolcfg ...

To activate the change:

pooladm -c

Online Resource Reallocation

Oracle Solaris allows CPUs to be reassigned between processor sets dynamically, enabling administrators to respond quickly to changing workload demands without requiring downtime.

Resource Pools Simplify Capacity Management

Dedicated pools provide predictable performance and workload isolation for critical applications.


Command Summary

# Review zone configuration
zonecfg -z prodcluster01 info

# Review active pool allocations
poolstat

# Review processor set configuration
poolcfg -c 'info pset prodpset'

# Backup current configuration
pooladm -s /var/tmp/poolcfg-backup-$(date +%Y%m%d)

# Increase allocated CPUs
poolcfg -c 'modify pset prodpset (uint pset.min=48; uint pset.max=48)'

# Apply configuration
pooladm -c

# Verify results
poolstat

Conclusion

Oracle Solaris Resource Pools provide a powerful framework for workload isolation and capacity management. In this example, a production cluster zone's dedicated CPU allocation was increased from 32 CPUs to 48 CPUs using online pool reconfiguration, demonstrating how Solaris can dynamically adapt system resources to meet evolving business demands without impacting application availability.

Wednesday, July 9, 2025

Basic important ODA (Oracle Database Appliance) commands:

Mastering Oracle Database Appliance (ODA): 15 Essential Commands Every Admin Should Know

As working ODA admin, one of the key responsibilities is ensuring the smooth operation of Oracle Database Appliances (ODA). Whether you're managing a Bare Metal (BM) or Virtualized ODA environment, having a solid grasp of the most frequently used commands can significantly enhance your efficiency and troubleshooting capabilities.

In this blog, I’ll walk you through 15 essential ODA commands that every administrator should keep in their toolkit.


🔍 System & Disk Insights

  1. odaadmcli show disk
    Displays the current status of all disks in the system.
    Useful for checking disk health and identifying potential issues.

  2. odaadmcli show server
    Provides server details including serial number and hardware specifications.
    Handy for inventory and support cases.

  3. odaadmcli show env_hw
    Reveals the ODA model and whether it's configured as Bare Metal or Virtualized.
    Crucial for understanding the deployment architecture.


🖥️ Virtual Machine (VM) Management

  1. odacli list-vms
    Lists all running VMs and their associated nodes.
    Helps track VM distribution across nodes.

  2. odacli stop-vm -n <VM_Name>
    Stops the specified VM gracefully.
    Ideal for maintenance or resource reallocation.

  3. odacli start-vm -n <VM_Name>
    Starts the specified VM.
    Quick recovery or scheduled startup.

  4. odacli start-vm -n odavm -nn node0
    Starts the VM odavm on a specific node (e.g., node0).
    Useful for node-specific operations or testing.

  5. virsh list or virsh list --all
    Lists all VMs, including inactive ones.
    A broader view of VM inventory.

  6. virsh domblklist <VM_Name>
    Displays all disk details attached to a VM.
    Essential for snapshot planning and storage audits.

  7. odacli describe-vm -n <VM_Name>
    Provides detailed information about a specific VM.
    Includes configuration, resource allocation, and status.

  8. virsh console <VM_ID>
    Accesses the console of a specific VM.
    Direct interaction for troubleshooting or configuration.


🗃️ Database System & Component Overview

  1. odacli list-dbsystems
    Lists all DB systems configured on the ODA.
    Helps monitor database deployments and usage.

  2. odacli describe-component
    Displays installed and available versions of components like DCS Controller, OS, ILOM, BIOS, etc.
    Vital for patching and version control.


📋 Job Monitoring & Troubleshooting

  1. odacli list-jobs
    Lists all jobs initiated via odacli commands.
    Tracks task execution and history.

  2. odacli describe-job -i <Job_ID>
    Provides detailed status (success/failure) of a specific job.
    Crucial for debugging and audit trails.


✅ Final Thoughts

These commands form the backbone of ODA administration. Whether you're provisioning VMs, monitoring disk health, or managing patch levels, having these commands at your fingertips can streamline operations and reduce downtime.

If you're new to ODA or looking to refine your command-line skills, start by practicing these commands in a test environment. Over time, they’ll become second nature and empower you to manage your Oracle infrastructure with confidence.


Author:
Kiran Jadhav
Mumbai, Maharashtra



 


Wednesday, July 20, 2022

Exadata Patching steps from Beginning

 Exadata Patching steps from start to end: 

* Exadata patching doc number : doc id 888828.1

* Patching sequence: Below patching sequence we have to follow while doing exadata patching:

  1. DB/Grid patching

  2. Cell node patching

  3. Compute node patching

  4. Roce/Ibswitch patching 

Patching steps:

1. Open the doc id 888828.1 in MOS (My Oracle Support) to know the latest (N) or N-1 patch information. Identify the exact patch you want to download

2. Copy the patches to the server ( there will be approx 10 files)

3. After copying patches to the server : 

[root@testJan_2022]# ls -lrth

total 30G

-rw-r--r-- 1 root root 3.1G Apr 22 17:05 p33567288_210000_Linux-x86-64_1of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 17:18 p33567288_210000_Linux-x86-64_3of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 17:27 p33567288_210000_Linux-x86-64_9of10.zip

-rw-r--r-- 1 root root 1.7G Apr 22 17:33 p33567288_210000_Linux-x86-64_10of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 18:18 p33567288_210000_Linux-x86-64_2of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 18:20 p33567288_210000_Linux-x86-64_7of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 18:24 p33567288_210000_Linux-x86-64_5of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 18:58 p33567288_210000_Linux-x86-64_8of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 19:00 p33567288_210000_Linux-x86-64_6of10.zip

-rw-r--r-- 1 root root 3.1G Apr 22 19:06 p33567288_210000_Linux-x86-64_4of10.zip


4. Then unzip all patch files

#unzip p33567288_210000_Linux-x86-64_1of10.zip

.

#unzip p33567288_210000_Linux-x86-64_10of10.zip

or 

#unzip '*.zip'


4. Unzipping files will create tar files something like below:

-rw-r--r-- 1 root root 3.1G Jan 21 15:12 33567288.tar.splitaa

-rw-r--r-- 1 root root 3.1G Jan 21 15:12 33567288.tar.splitab

-rw-r--r-- 1 root root 3.1G Jan 21 15:13 33567288.tar.splitac

-rw-r--r-- 1 root root 3.1G Jan 21 15:13 33567288.tar.splitad


5. Now untar all files to create a common patch repo files:

#cat *.tar.* | tar -xvf -


6. Now unzip patch files from below directory in order to get dbnodeupdate.sh and patchmgr scripts.

#cd /QFSDP/Jan_2022/33567288/Infrastructure/SoftwareMaintenanceTools/DBNodeUpdate/21.211221

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/FabricSwitch

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataStorageServer_InfiniBandSwitch


7. Once DB team confirm that they are done with the DB/GRID patching then we can start cellnode patching.

8. Before doing actual patching we have to raise the prechecks SR in MOS and need to upload necessary logs like sosreport and sundiag from compute nodes and cell nodes, exachk from one of the compute node and the prechecks logs from compute node and cell nodes.

8.a Below is the cell node prechecks commands:

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataStorageServer_InfiniBandSwitch/patch_21.2.8.0.0.220114.1

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -reset_force

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -cleanup

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -patch_check_prereq -rolling -ignore_alerts

Note : If error found in prechecks get it rectified with the help of backend team.

The possible error could be:

https://kiranbjadhav.blogspot.com/2022/05/exadata-cell-node-patching-errorusb.html

8.b If there is no error found in prechecks then we can proceed with the actual patching commands.

Actual patching command :

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataStorageServer_InfiniBandSwitch/patch_21.2.8.0.0.220114.1

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -reset_force

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -cleanup

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -patch -ignore_alerts   

all cells patching will get completed (DB needs to be down here)

If you are doing cellnode patching without DB downtime then 

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group -patch -rolling -ignore_alerts   

all cells will get patched in rolling mode one by one.


If you want to do manual cell node patching taking one cell at a time then

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group_1 -patch -rolling -ignore_alerts  ---> 1 cell at a time assuming cell_group_1 has only one cell entry.

#./patchmgr -cells /opt/oracle.SupportTools/onecommand/cell_group_2 -patch -rolling -ignore_alerts and so on


9. Once cell nodes patching completed successfully then we can start with Compute node patching 

9.a Compute node patching prechecks

#cd /QFSDP/Jan_2022/33567288/Infrastructure/SoftwareMaintenanceTools/DBNodeUpdate/21.211221

#./dbnodeupdate.sh -u -l /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataDatabaseServer_OL7/p33665705_212000_Linux-x86-64.zip -v 

The possible error could be:

https://kiranbjadhav.blogspot.com/2022/05/exadata-compute-node-patching-prechecks.html


9.b Actual patching command: 

Prerequisite:

. Once compute node at a time and 

. DB and CRS must be down on the particular compute node

. NFS mount point should be unmounted

#./dbnodeupdate.sh -u -l /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataDatabaseServer_OL7/p33665705_212000_Linux-x86-64.zip


9.c After successful compute node upgrade run below command to finish post steps.

#./dbnodeupdate.sh -c 


10. After successful compute node patching we can do ROCE switch or IBswitch patching

10.a Roce switch patching prechecks:

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/FabricSwitch/patch_switch_21.2.8.0.0.220114.1

#./patchmgr --roceswitches /roceswitches.lst --upgrade --roceswitch-precheck --log_dir /scratchpad/


10.b Actual patching command:

./patchmgr --roceswitches /roceswitches.lst --upgrade --log_dir /scratchpad/


10.c If ibswitches are there instead of Roce switches

#cd /QFSDP/Jan_2022/33567288/Infrastructure/21.2.8.0.0/ExadataStorageServer_InfiniBandSwitch/patch_21.2.8.0.0.220114.1

  ./patchmgr -ibswitches /opt/oracle.SupportTools/onecommand/ibs_group -upgrade -ibswitch_precheck


10.d Actual patching command:

./patchmgr -ibswitches /opt/oracle.SupportTools/onecommand/ibs_group -upgrade



Regards,

Kiran Jaadhav