High Disk Space Usage on / root partition and How To Clear
 626072
Created On 09/25/18 19:37 PM - Last Modified 05/10/23 20:27 PM
CONTENT RELEASEDEPLOYMENTHARDWAREPAN-OS

Symptom
Some common symptoms of the root partition getting filled up are:
Unable to log in or access Web UI
Certain daemons processes not starting
Incomplete tech support bundles
System log alerts with "Disk usage for / exceeds limit,  X percent in use, cleaning file system"
Root partition is high
> show system disk-space 
Filesystem            Size  Used Avail Use% Mounted on
/dev/md2              3.8G  3.1G  581M  85% /                   <----- root partition
/dev/md5              7.6G  4.2G  3.0G  59% /opt/pancfg .        
/dev/md6              3.8G  3.0G  666M  82% /opt/panrepo
tmpfs                 2.0G  210M  1.8G  11% /dev/shm
cgroup_root           2.0G     0  2.0G   0% /cgroup
/dev/md8               88G  2.4G   81G   3% /opt/panlogs
tmpfs                  12M     0   12M   0% /opt/pancfg/mgmt/lcaas/ssl/private


Environment
PAN-OS
Palo Alto Firewall
Panorama


Cause
Some of the common causes of a filled partition:

An admin troubleshooting certain processes and creates core files. These files are stored on the root and remain there until deleted by the administrator.
Enabling of diagnostic logs for the dataplane (packet diags) can also take up space on the root partition.


Resolution
Enable Aggressive Cleaning (PAN-OS 7.1.14+, 8.0.7+, 8.1.0+)
This will automatically truncate all old log files (entries under all *var/log/pan directories matching *.1, ... *.4, *.log.old) if the 95% occupancy alarm is tripped.

 > debug software disk-usage aggressive-cleaning <enable|disable>
Note: Enabling aggressive clean up may clear up logs, rendering logs unavailable for troubleshooting  purposes.

To verify the changes or if it is already enabled use the command below.

> show system state | match aggressive-cleaning
cfg.debug-sw-du.config: { 'aggressive-cleaning': True, }
 

Run disk-usage cleanup command (PAN-OS 8.1.0+)
disk-usage cleanup can be done manually using the command below:
> debug software disk-usage cleanup ?
+ deep        cleanup with deleting backup logfile
* threshold   percentage threshod of size of system partition to kick off cleanup

> debug software disk-usage cleanup deep threshold 90

Note that this command has to be run manually each time to bring the disk usage to below 90% and is not persistent across reboots.
Note:
1- Enabling aggressive clean up may clear up logs, rendering logs unavailable for analysis.
2- If the following