Repository navigation
Empty directories accumulate in local preview storage after preview files are deleted #46425
Description
This should delete empty preview directories as well.
After running a few weeks with automatic ZFS snapshots folder attached as a history (It was a bad idea),
I've got a monstrous directory structure and the cron scan constantly utilized my disk for several days(SELECT-UPDATE on the oc_filecache table).
files:cleanup didn't do anything, 0 orphaned files.
files:scan-app-data took almost 6 hours to complete:
+---------+--------+--------------+
| Folders | Files | Elapsed time |
+---------+--------+--------------+
| 5868385 | 151476 | 05:45:49 |
+---------+--------+--------------+
So I'm now deleting empty folders manually with find . -type d -empty -delete in the preview directory.
Activity
petruchito commentedon Jul 11, 2024
occ files:scan-app-dataafter deleting empty directories:szaimen commentedon Jul 11, 2024
Hi, which nc version?
petruchito commentedon Jul 11, 2024
Nextcloud AIO v9.1.0
(NEXTCLOUD_VERSION | 29.0.3)
petruchito commentedon Jul 11, 2024
Snapshots were ~400Gb each, 24 hourly, 7 daily, 4 weekly and 12 monthly, so every hour the server was introduced with 400Gb of data and 400Gb disappeared. IO utilization issues persisted after I unmounted the
history.Folders structure under the `history/` was like this:
And this might also relate to the problem - I have
'filesystem_check_changes' => 1in theconfig.php.joshtrichards commentedon Jul 19, 2024
I don't quite understand your bug report and you seem to have entirely bypassed our reporting template.
I understand your suggestion about the background job, but what was the originally problem that arose?
There are (or were) some bugs like #45654. Is this merely that?
petruchito commentedon Jul 20, 2024
Constant disk utilization by nextcloud's postgres led me to finding 5M folders in the preview folder.
The cleanup job seems not to remove all empty folders and they were building up there.
(#45654 does not seem to relate.)
petruchito commentedon Jul 20, 2024
I suppose this deletes
preview/b/5/7/7/2/e/7/39548/but leavespreview/b/5/7/7/2/e/7/orphaned even if it's an empty directory.Thus you theoretically can end up with 7^16=3,32×10^13 empty directories.
oTTo-oTTo commentedon Oct 23, 2024
I plead for 16⁷ directories 2.7E08
and yes, the empty ones should be deleted.
NC29.0.4: if have 1300 empty preview dirs in 3 months.
marcan commentedon May 20, 2025
Same problem here, and this is particularly painful on CephFS where folders aren't cheap (they each create a metadata object and scanning them is slow if too many exist to fit in MDS cache).
The cleanup should be walking up the folder tree and trying to delete any empty folders. I'm doing the
findthing in a cron job to clean up empty folders now, but I don't know if this will confuse nextcloud since the empty folders are still cached inoc_filecache. Presumably they should be deleted from there too.Edit: Trying this now, hopefully it's a bit safer:
[-]Empty directories building up in the `preview` folder.[/-][+]Empty directories accumulate in local preview storage after preview files are deleted[/+]