Soft delete

Setup Usage

This page describes the soft delete feature, which preserves objects and buckets that get deleted or overwritten for a specified period of time. Soft delete helps protect your data from accidental or malicious deletion by retaining deleted objects or buckets in a soft-deleted state, during which the object or bucket cannot be permanently deleted. Soft delete is enabled by default on all buckets and has a retention duration of seven days unless you or your organization have chosen a different policy.

Overview

When you enable soft delete on a bucket, deleting the bucket or the objects within the bucket results in the resource entering a soft-deleted state instead of being permanently deleted. Soft-deleted objects and buckets behave in the following ways:

To enable soft delete on a bucket, you create a soft delete policy that specifies a retention duration controlling how long the bucket and objects within it are retained before they get permanently deleted.

To learn how to set and manage a soft delete policy to enable or disable soft delete on a bucket, see Use soft delete.

Soft delete policies

By default, soft delete is enabled on all newly created buckets. You can edit a soft delete policy to enable or disable soft delete for a bucket. A soft delete policy can be set, deleted, or edited during a bucket creation or update request.

Soft delete policies behave in the following ways:

  • Updating a bucket's soft delete policy only applies to objects that you delete after the soft delete policy takes effect. Objects that you deleted prior to the update are retained for the duration that was in effect when they were deleted.

    For example, say you enable a soft delete policy in your bucket with the default retention duration of seven days, and then you delete the object cat.png. In this scenario, cat.png is retained as a soft-deleted object for the next seven days, and then it's permanently deleted. This occurs even if you later change or remove the soft delete policy for your bucket.

  • If you add a new soft delete retention duration to an existing bucket, it won't apply to any of the objects or buckets that were deleted before the soft delete policy takes effect.

  • If you delete your project, you cannot use soft delete to restore the buckets or objects in it, even if they had soft delete enabled. To ensure you don't lose your data in case of malicious or accidental deletion, we recommend limiting access to project-level deletion by placing a lien on your projects or backing up business-critical data to buckets in a different project.

  • If a bucket is deleted, the objects within it are not permanently deleted but cannot be listed or restored unless the soft-deleted bucket is restored first. When you restore a soft-deleted bucket, the bucket is restored without live objects in it. You'll need to either perform an object restore operation or add new objects in it. To learn more about using soft-deleted buckets, see Restore a soft-deleted bucket.

Soft delete retention duration

Within a soft delete policy's retention duration, you can restore deleted objects or buckets, but after the duration ends, Cloud Storage permanently deletes the soft-deleted resource.

When you create a bucket, a soft delete policy is added to the bucket with a default retention duration of seven days. You can edit a policy's retention duration or set a new default retention duration. The maximum retention duration you can set is 90 days and the minimum retention duration you can set is 7 days. You can also set the retention duration to 0 to disable soft delete on the bucket.

Restoration of soft-deleted objects

When you restore a soft-deleted object, Cloud Storage creates a copy of the soft-deleted object in the same bucket.

Restore behavior for soft-deleted objects can be described as the following:

  • When you restore a soft-deleted object, Cloud Storage creates a new, live version of the object in the same bucket. The soft-deleted version of the object remains in the bucket until its retention duration expires.

    In some cases, if a live object already exists with the same name as the soft-deleted object at the time of restoration, the restored object replaces the live object, and the live object is then soft-deleted. In such a scenario, your bucket contains the following objects:

    • The replaced live object that is archived or in the soft-deleted state.

    • Two copies of the previously soft-deleted object, one live copy and one still-soft-deleted copy.

  • Soft-deleted objects can be restored using multiple methods. You can restore the soft-deleted objects synchronously, or you can create a long-running operation to bulk restore objects that meet a certain criteria.

Restoration of soft-deleted objects in buckets with hierarchical namespace enabled

Although rare, duplicate soft-deleted objects with identical names and generation values can occur in buckets with hierarchical namespace enabled. Consider this scenario:

  1. Object folderA/my-object.txt is soft-deleted.
  2. Its parent folder, folderA, is then deleted.
  3. A different object, folderB/my-object.txt, is moved to the original path by renaming folderB to folderA. This object now becomes folderA/my-object.txt.
  4. This new folderA/my-object.txt object is also soft-deleted.

While each object version in Cloud Storage has a unique generation value, independently created objects like the original folderA/my-object.txt and folderB/my-object.txt might coincidentally share the same generation value. Combined with the scenario, this can result in two soft-deleted objects with the identical names (folderA/my-object.txt) and generation values.

To make sure you correctly restore, list, or get the right object, include the restoreToken parameter in your request.

When using soft delete in buckets with hierarchical namespace enabled, consider the following information:

  • If you try to restore, list, or get duplicate objects without using the restoreToken parameter, you receive an error asking you to use the restoreToken parameter in your object restore or object list request.

  • If you use the wrong values for the restoreToken, name, or generation parameters as part of your object restore, list, or get request, you receive an error indicating that the specified object does not exist.

Restoration of buckets

When you restore a soft-deleted bucket, it returns to a live state and the objects that were in the bucket at the time of deletion also become recoverable within the soft delete retention duration.

Like soft-deleted objects, Cloud Storage permanently deletes a soft-deleted bucket at the end of the soft delete retention duration if the soft-deleted bucket isn't restored during the retention duration.

Pricing

Soft-deleted objects incur storage charges until the soft-deleted objects are permanently deleted after the retention duration is over. For more information about costs associated with soft-deleted objects, see Cloud Storage pricing.

Soft delete cost optimization

To help lower storage costs on short-lived, temporary objects while maintaining data protection, we recommend the following:

  • Rename objects: use the Objects: move API to rename a short-lived, temporary object rather than copying the object and deleting the original.

  • Separate temporary data: when possible, separate temporary data into a separate bucket with soft delete disabled.

Interactions with other products and features

When soft delete is enabled, it behaves with other products and features in the following ways:

  • Object Lifecycle Management:

    • Object Lifecycle Management rules don't affect soft-deleted objects. It is not possible to use Object Lifecycle Management to change the storage class of soft-deleted objects or to permanently delete then.

    • Objects deleted by Object Lifecycle Management become soft-deleted. If you also have Object Versioning enabled on your bucket, deleted live objects become noncurrent and deleted noncurrent objects become soft-deleted.

  • Object Versioning: When you delete a noncurrent object, it becomes soft-deleted.

  • XML API multipart uploads: XML API multipart upload parts are not protected by soft delete.

  • Autoclass:

    • Autoclass management fees are not assessed for soft-deleted objects.

    • Autoclass does not change the storage class of soft-deleted objects.

    • When you restore a soft-deleted object, the resulting object is set to the Standard storage class.

  • Bucket Lock and Object Retention Lock features: These features will prevent you from deleting objects until they fulfill their retention policies. Soft delete then provides an additional layer of protection by retaining deleted objects for a separate soft delete retention duration.

  • Pub/Sub notifications: Restoring a soft-deleted version of an object triggers an OBJECT_FINALIZE event.

  • Managed folders: Soft delete cannot restore the IAM policies on managed folders. If you soft delete an object and delete a managed folder that grants IAM policies for the object, you might have to recreate those IAM policies before you have the necessary permissions to restore the soft-deleted object.

  • Tags: If you create a bucket without including a soft delete setting yourself, the Cloud Storage default is to create the bucket with a seven day soft delete retention duration. You can change this default by using a tag. The tag can change the default retention duration for new buckets to be any value between 7 and 90 days, or it can change the default to disable soft delete on new buckets. You can also use tags to disable soft delete by default.

  • Recommender: You can enable the Recommender API to receive suggestions and insights on enabling or disabling soft delete based on Cloud Billing costs and usage. To learn more, see soft delete recommender.

What's next