Skip to content

Bucket usage continues even after the bucket is deleted. #13057

Open

Description

@rodrigues8

problem

Good afternoon everyone,

I've identified a potential bug in the CloudStack usage section. After deleting a bucket, I noticed that the cloudstack.listUsageRecords API call was still reporting usage of the deleted buckets for the accounts.

I started exploring the code and how this bucket usage system works, and I found that it all depends on the bucket_statistics table.

From what I understand and replicate in my tests, when a bucket is removed, the bucket_statistics table retains the used size value. In my understanding, the correct approach would be to reset this value or delete the record for that bucket in this table. I corrected this in the accounts by resetting the size value, thus stopping the generation of new records in the cloud_usage table.

If you have any further questions, feel free to ask and I'll provide more details.

Thank you.

versions

Cloudstack 4.22

The steps to reproduce the bug

  1. Create a bucket
  2. Add data to the bucket
  3. Wait a few days for cloud_usage records to be generated
  4. Remove the data from the bucket
  5. Then remove the bucket from Cloudstack
  6. Verify in the listUsageRecords API calls that usage records are still being generated for the bucket.

What to do about it?

When removing a bucket, the bucket_statistics table retains the used size value; in my understanding, the correct approach would be to reset this value or delete the record for that bucket in this table.

Activity

  1. boring-cyborg commented on Apr 22, 2026

    @boring-cyborg

    Thanks for opening your first issue here! Be sure to follow the issue template!

  2. andrijapanicsb commented on Apr 23, 2026

    @andrijapanicsb
    Contributor

    Sounds like a bug...

  3. DaanHoogland commented on Apr 28, 2026

    @DaanHoogland
    Contributor

    @rodrigues8 did you see a bucket delete event in the usage_event table?

  4. added this to the 4.20.4 milestone on Apr 29, 2026
  5. rodrigues8 commented on Apr 30, 2026

    @rodrigues8
    Author

    There are no bucket records in usage_events; it seems the method for checking bucket usage is different from compute offers.

    Only the bucket_statistics table is taken into consideration. @DaanHoogland

  6. DaanHoogland commented on May 1, 2026

    @DaanHoogland
    Contributor

    @abh1sar can you give your opinion on this one, please?

  7. abh1sar commented on May 13, 2026

    @abh1sar
    Contributor

    Yes, this does look like a bug. bucket_statistics entry should be removed when a bucket is deleted

  8. added theissue type on May 13, 2026
  9. github-actions commented on Sep 11, 2026

    @github-actions

    This issue is stale because it has been open for 120 days with no activity. It may be removed by administrators of this project at any time. Remove the stale label or comment to request for removal of it to prevent this.

  10. ditahkk commented on Sep 15, 2026

    @ditahkk

    Related failure observed on CloudStack 4.22.1.1: when a type-29 usage record references a bucket that is absent even from findByIdIncludingRemoved, listUsageRecords returns HTTP 530 instead of returning the usage response.

    The exception is Cannot invoke "com.cloud.storage.BucketVO.getUuid()" because "bucket" is null.

    ApiResponseHelper.java in 4.22.1.1 dereferences the lookup result through getUuid() and getName() without a null check. The same branch remains in current main, checked on 2026-09-15.

    This adds a response-handling failure to the stale-statistics problem described here. Normal soft deletion alone does not explain a lookup returning null. The cause of the missing bucket row in our case remains unknown.

    Suggested regression case: a BUCKET usage record whose bucket lookup returns null, included in an otherwise valid usage response. The API should handle the missing resource without failing the entire response. This is a proposed test, not a new reproduction run.

    Stopping the orphaned statistics source and clearing only the affected usage backlog restored the API in our environment. Subsequent completed aggregation runs did not recreate the orphaned usage. No upstream patch has been applied; the missing-resource guard still needs a fix independently of statistics cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions