ubuntu:disk:no_space_left_on_device_-_running_out_of_inodes
Table of Contents
Ubuntu - Disk - No space left on device - Running out of inodes
Most likely you have too many small or 0-sized files on your disk, and while you have enough disk space, you have exhausted all available Inodes.
To Fix:
1. Check available disk space to ensure that you still have some space free
df
returns:
root@server1:~# df Filesystem Type Size Used Avail Use% Mounted on udev devtmpfs 16G 0 16G 0% /dev tmpfs tmpfs 3.2G 316M 2.9G 10% /run /dev/mapper/vg01-root ext4 9.1G 2.0G 6.8G 23% / /dev/mapper/vg01-usr ext4 20G 2.6G 16G 14% /usr tmpfs tmpfs 16G 0 16G 0% /dev/shm tmpfs tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs tmpfs 16G 0 16G 0% /sys/fs/cgroup tmpfs tmpfs 16G 0 16G 0% /run/shm /dev/mapper/vg01-opt ext4 1.8G 2.8M 1.7G 1% /opt /dev/mapper/vg01-var ext4 24G 22G 1.1G 96% /var /dev/mapper/vg01-srv ext4 102G 93G 6.0G 94% /srv /dev/sda3 ext4 1.9G 286M 1.5G 17% /boot /dev/mapper/vg01-backup ext4 3.7G 332M 3.1G 10% /backup /dev/mapper/vg01-home ext4 11G 4.8G 5.0G 49% /home /dev/mapper/vg01-sharewiz ext4 9.5G 936M 8.1G 11% /sharewiz cgmfs tmpfs 100K 0 100K 0% /run/cgmanager/fs tmpfs tmpfs 3.2G 0 3.2G 0% /run/user/1000 total - 250G 126G 118G 52% -
2. Check available Inodes
df -i
returns:
root@server1:~# df -i Filesystem Inodes IUsed IFree IUse% Mounted on udev 4098818 522 4098296 1% /dev tmpfs 4103598 1075 4102523 1% /run /dev/mapper/vg01-root 609600 61155 548445 11% / /dev/mapper/vg01-usr 1276096 244376 1031720 97% /usr tmpfs 4103598 1 4103597 1% /dev/shm tmpfs 4103598 6 4103592 1% /run/lock tmpfs 4103598 18 4103580 1% /sys/fs/cgroup tmpfs 4103598 1 4103597 1% /run/shm /dev/mapper/vg01-opt 121920 11 121909 1% /opt /dev/mapper/vg01-var 1544320 315122 1229198 21% /var /dev/mapper/vg01-srv 6738112 192752 6545360 3% /srv /dev/sda3 122160 329 121831 1% /boot /dev/mapper/vg01-backup 244320 3836 240484 2% /backup /dev/mapper/vg01-home 674624 25855 648769 4% /home /dev/mapper/vg01-sharewiz 625856 10440 615416 2% /sharewiz cgmfs 4103598 14 4103584 1% /run/cgmanager/fs tmpfs 4103598 4 4103594 1% /run/user/1000
If you have IUse% at 100 or near, then huge number of small files is the reason for “No space left on device” errors.
3. Find those little files
for i in /*; do echo $i; find $i |wc -l; done
This command will list directories and number of files in them. Once you see a directory with unusually high number of files (or command just hangs over calculation for a long time), repeat the command for that directory to see where exactly the small files are.
For example:
for i in /home/*; do echo $i; find $i |wc -l; done
4. Once you found the suspect files – just delete the files
sudo rm -rf /home/bad_user/directory_with_lots_of_empty_files
ALERT! DO NOT use an * after the rf part of the rm command.
For example, do not do sudo rm -rf /*
This could remove a huge number of files and destroy your system.
5. Check the results with df -i command again
df -i
returns:
root@server1:~# df -i Filesystem Inodes IUsed IFree IUse% Mounted on udev 4098818 522 4098296 1% /dev tmpfs 4103598 1075 4102523 1% /run /dev/mapper/vg01-root 609600 61155 548445 11% / /dev/mapper/vg01-usr 1276096 244376 1031720 20% /usr tmpfs 4103598 1 4103597 1% /dev/shm tmpfs 4103598 6 4103592 1% /run/lock tmpfs 4103598 18 4103580 1% /sys/fs/cgroup tmpfs 4103598 1 4103597 1% /run/shm /dev/mapper/vg01-opt 121920 11 121909 1% /opt /dev/mapper/vg01-var 1544320 315122 1229198 21% /var /dev/mapper/vg01-srv 6738112 192752 6545360 3% /srv /dev/sda3 122160 329 121831 1% /boot /dev/mapper/vg01-backup 244320 3836 240484 2% /backup /dev/mapper/vg01-home 674624 25855 648769 4% /home /dev/mapper/vg01-sharewiz 625856 10440 615416 2% /sharewiz cgmfs 4103598 14 4103584 1% /run/cgmanager/fs tmpfs 4103598 4 4103594 1% /run/user/1000
ubuntu/disk/no_space_left_on_device_-_running_out_of_inodes.txt · Last modified: 2020/07/15 10:30 by 127.0.0.1