Page Comparison

...

Object locking prevents object versions from being deleted or overwritten – for a fixed amount of time or indefinitely. You can use Use an object lock to help meet regulatory requirements that require requiring WORM storage, or to simply add another layer of protection against object changes and deletion.

...

Object locking does not prevent the creation of new versions of an object while the object is locked – it makes it impossible to delete or otherwise change the version(s) of the object that have locking enabled.

There are two types of object locking that can be used simultaneously and independent of each other:

• Retention – Specifies a fixed period of time ("retention period") during which the object remains locked. During this period, the object is WORM-protected and can not be overwritten or deleted. After the period expires, the lock goes away automatically.

• Legal hold – When applied, the object stays locked indefinitely. A legal hold does not expire; it must be explicitly removed.

Retention

Retention Periods

A retention period is used to set the fixed amount of time that the object needs to remain locked. Until that time expires, the object version cannot be changed or deleted.

There is more than one way to set a retention period on an object version:

• Newly created objects can inherit a default retention period configured on the bucket level.

• You can explicitly set

  Explicitly setting a retention period when creating a new object

  . This

  overrides the default retention period configured for the bucket if present.

• You can explicitly

  Explicitly set a retention period on an existing object version.

A bucket default retention period specifies a duration (in days or years) for which every object version placed in the bucket should be locked. When you place placing an object in the bucket, Gateway calculates a retention period for the object version by adding the specified duration to the object version's creation timestamp.

...

A retention period can always be extended after it has been set. To do this:

1. Submit a new lock request for the object version with a retention period that is longer than the current one.

2. Gateway replaces the existing retention period with the new, longer period.

Any user with permissions to set an object retention period can also extend a retention period.

...

A retention mode must always be specified when you lock locking an object or set setting a bucket default retention period. 

There are two retention modes that impact what you can do impacting actions with objects under retention:

• In governance mode,

  you can still

  grant some users the permission to shorten or remove a retention period or delete object versions under retention if necessary.

• In compliance mode, a locked object version cannot be overwritten or deleted by any user, even the admin user. When an object is locked in compliance mode, its retention mode cannot be changed, and its retention period cannot be shortened.

In a deviation from S3, Gateway will always use uses the maximum of either the bucket default retention duration, or the duration specified in a per-object request.

...

Legal holds do not have an associated retention period . They and are completely independent from retention periods and retention modes. As long as the bucket containing the object has object locking enabled , you can set and remove adding and removing legal holds are available. It does not matter if the specified object version has a retention period set or not.

Prerequisites

While object locking is implemented fully from Gateway 7.6 onwards, in order to use this feature, you need to be running Swarm Storage 12.0 or above because  must be running because it relies on the Swarm lifepoints feature to prevent the deletion of locked objects until a certain date has passed.

Whenever an object gets locked until a certain date, it will get gets a deletable=no lifepoint protecting it from deletion until that date.

In order for object locking to work, versioning needs to be enabled.

Gateway will refuse refuses to enable object locking when versioning is not enabled. Once object locking is enabled, Gateway will refuse refuses to disable versioning. In both cases an error message is displayed.

...

Object locking uses the following headers:

• on buckets:
  x-object-lock-meta-status: ENABLED
  (empty means DISABLED)
  x-object-lock-meta-default: <GOVERNANCE|COMPLIANCE>[:<duration>]
  Bucket default retention period duration is expressed as <integer>y for number of years or

  <integer>d 

  <integer>d for number of days.

• on objects:
  x-object-lock-meta-mode: <GOVERNANCE|COMPLIANCE>
x-object-lock-meta-retain-until-date: <date>
x-object-lock-meta-legal-hold: ON
  (empty means OFF)
  x-object-lock-meta-original-lifepoints:
  <original lifepoints>
  lifepoint: [<date>] deletable=no
  (for retention period)
  lifepoint: [] deletable=no
  (for legal hold)

The above headers are listed using their SCSP names. The corresponding S3 names start with x-amz-*. Only the SCSP headers are effectively stored with the objects. The S3 names are mapped onto their SCSP counterparts and back on-the-fly.

Assumptions and Limitations

• Internal to Gateway, all header values are treated as case-insensitive.

• Dates are in the rfc1123 format, eg. "Wed, 12 Dec 2016 15:59:02 GMT". For S3 these are translated into the ISO8601 format.

• The x-object-lock-meta-retain-until-date header applies only to retention periods and specifies the end date of the retention period. The x-object-lock-meta-legal-hold header only applies to legal hold.

  You can set b

  Both a retention period and a legal hold can both be set on the same object version.

• The x-object-lock-meta-original-lifepoints header stores the complete set of user defined delete/deletable lifepoint headers found on the object at the time the retention period/legal hold was applied. The original delete/deletable lifepoint headers are removed. This means Swarm

  will

  no longer

  consider

  considers these lifepoints.
  Gateway

  will

  only

  manipulate

  manipulates delete/deletable type lifepoints, all other lifepoints

  will be

  are unaffected and continue functioning normally.
  As long as an object lock is in effect, it takes precedence over any user-defined delete/deletable lifepoints, blocking delete. When an object lock expires (or gets removed), the user-defined lifepoints take effect again.

• Object locking only works for tenanted objects.

  You

  Object locking cannot be set

  object locking

  on untenanted and unnamed objects in

  your

  a cluster.

Lifepoint Headers

Gateway now adds a single deletable=no lifepoint header (the lock lifepoint), to go along with the object lock. This lock lifepoint is what actually protects the object against deletion in Swarm, both through user requests and through built-in functionalities like HP or bucket policies.

The lock lifepoint is computed as follows;

1. If the object is locked with a retention period, then the

   The lock lifepoint end date matches the end date of the retention period if the object is locked with a retention period. For legal hold, the lock lifepoint has no end date.

2. Next we go over the list of original lifepoints, and append those whose end date is later than the one from the lock lifepoint. In case of legal hold, there is no end date so none of the original lifepoints get appended.

The purpose of storing the set of original lifepoints is to allow later modifications/removal of the object lock to recompute/reinstate the original lifepoints as they were before the object lock.

...

Enabling Object Locking on a Bucket

Before you can lock locking any objects, object locking must be enabled on the bucket.

...

Object Locking and Versioning Inheritance Rules

...

Inheritance Rules

Versioning can be disabled for another bucket or domain in the cluster that is unrelated .If if a bucket in one domain has object locking (and therefore versioning) is enabled at a bucket level, it cannot enabled.

Cannot be disabled at the bucket, domain or cluster level .If it if object locking (and versioning) is enabled at a domain level, you can still disable it bucket level.

Can be disabled for individual buckets .If it is if enabled at the cluster level, you can disable it a domain level.

Can be disabled for individual domains and/or buckets if enabled at the cluster level.

It does not matter if versioning was enabled on the bucket itself, or whether it was inherited from cluster or domain level. Gateway will refuse refuses to disable versioning at the domain or bucket level if object locking is in effect anywhere within it.

...

The request to enable object locking can fail with the following errors:

• 412 Precondition

  412 Precondition Failed

   will be

   is displayed if the Swarm cluster does not yet support all the features necessary to perform the operation.

  412 Precondition

• 412 Precondition Failed

   will be

   is displayed if the bucket does not have versioning enabled.
  

• 403 Forbidden if the user does not have the PutBucketObjectLocking permission.

Enabling object locking on a bucket comes down to storing the x-object-lock-meta-status and optionally x-object-lock-meta-default headers on the bucket context object. This immediately takes care of caching the bucket’s object locking configuration in memory so that it is readily available during object requests.

Enabling Object Locking using S3

You can enable Enable or inspect the object locking configuration on a bucket using the following calls;

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html

Enabling Object Locking using SCSP

To enable object locking on a bucket:

PUT /<bucket>?objectlock=<defaultmode> [ :<defaultperiod> ]

• defaultmode can be either "governance" or "compliance"

• defaultperiod is optional; it is a number of years (y) or days (d), eg. 1y or 20d.

In this call you can omit either defaultmode, defaultperiod, or both.

If both are omitted, the bucket will allow The bucket allows object locking (it will be is enabled), but no locking will happen happens by default if both are omitted. Objects written to the bucket without any locking directives will remain unlocked.

The defaults can be modified or removed at any time via additional PUT commands. This does not affect the object locking status of the bucket – once it is enabled, it stays enabled.

Object Locking Cannot be Disabled After it has been Enabled

It does not matter if versioning was enabled on the bucket itself, or whether it was inherited from cluster or domain level. Gateway will refuse refuses to disable versioning at the domain or bucket level if object locking is in effect anywhere within it.

To allow writing unlocked objects into a bucket with object locking enabled, you can remove the lock defaults and write your new objects without any of the object locking headers.

To remove the lock defaults:

PUT /<bucket>?objectlock=
(use an empty query argument)
The "=" is optional.

How to Check Object Locking Status

• To query the object locking status of a bucket:
  GET /<bucket>?objectlock
  This returns the following response headers:
  x-object-lock-meta-status: ENABLED
x-object-lock-meta-default: <GOVERNANCE|COMPLIANCE>[:<duration>]
  And the response body says:
  Object locking is enabled on bucket <bucket> with default mode <mode> [ and default duration <duration> ]
  

  If

  No response headers are present and the response body displays as below if the bucket does not have object locking enabled

  , then no response headers are present and the response body says

  :
  Object locking disabled

REST API Changes

Object locking introduces the following new REST calls:

• PUT with ?objectlock query arguments

• GET with ?objectlock query arguments

• DELETE with ?objectlock query arguments (DELETE is an object-only call)

Object locking also introduces changes to existing REST calls:

• PUT can now take object locking headers to create locks as a side effect

• POST can now take object locking headers to create locks as a side effect

• COPY can now take object locking headers to create locks as a side effect

• GET can return object locking headers if the user has the appropriate permissions

• HEAD can return object locking headers if the user has the appropriate permissions

For S3, see the Amazon documentation website.

...

x-object-lock-meta-mode: <GOVERNANCE|COMPLIANCE>
x-object-lock-meta-retain-until-date: <date>
This takes precedence over the default bucket retention mode and duration, if present.

In a deviation from S3, Gateway will always use uses the maximum of either the bucket default retention duration, or the duration specified in a per-object request.

If Gateway looks for corresponding defaults at the bucket level if any one of these two headers is omitted from the request , Gateway will look for corresponding defaults at the bucket level. If found, it will take and if found, takes the corresponding values from there.

If either mode or retain-until-date is then still missing, the The request fails with a 400 Bad Request error because both are needed for a successful retention locklock if either mode or retain-until-date is then still missing.

Creating a New Object with a Legal Hold

...

To use these headers, the user needs to have the PutObjectRetention and, respectively, PutObjectLegalHold permission, or the request fails with a 403 Forbidden error.

If The request fails with a 412 Precondition Failed error if the bucket does not have object locking enabled, then the request fails with a 412 Precondition Failed error.

Gateway forwards these headers when creating the new object on Swarm, and also creates a lock lifepoint instructing Swarm to not delete the object before the retention period expires. For retention periods, the lock lifepoint also includes the subset of the original lifepoints that had with a later end date than the retention period.

lifepoint: [<date>] deletable=no, <later lifepoints>

Or in case of legal hold;

lifepoint: [] deletable=no

The original lifepoint headers are preserved in:

x-object-lock-meta-original-lifepoints: <original lifepoints>

Writing an Object as a Normal Unlocked Object in a Bucket with Object Locking Enabled

If Objects are written as a normal unlocked object, despite being written to a bucket with object locking enabled if both headers are omitted from the request, and there are no defaults at the bucket level, then the object gets written as a normal unlocked object, despite being written to a bucket that has object locking enabled.

Managing Retention on an Existing Object

Enabling and disabling retention on an object requires that the user has have the PutObjectRetention permission.

...

The following headers are added or changed;

x-object-lock-meta-mode: <GOVERNANCE|COMPLIANCE>
x-object-lock-meta-retain-until-date: <date>
x-object-lock-meta-original-lifepoints: <original lifepoints>
lifepoint: [<date>] deletable=no, <later lifepoints>

When changing the retention period, Gateway must also recompute the lock lifepoint, starting from the preserved set of original lifepoints and only appending those with later end dates to the lock lifepoint. This is the main purpose of preserving the original lifepoints.

In a deviation from S3, Gateway will always use uses the maximum of either the bucket default retention duration, or the duration specified in a per-object request.

Introducing or extending a retention period is always possible, but there are restrictions to shortening or removing a retention period on an object that is already under retention:

• In compliance mode this is never permitted

• In governance mode, the user needs to have the special BypassGovernanceRetention permission.
  Also, an S3 request must explicitly include x-amz-bypass-governance-retention:true as a request header with any request

  that requires

  requiring overriding governance mode.

Using S3, you can enable or inspect the retention period on an object using the following calls;

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectRetention.html

Using SCSP, you can enable or inspect the retention period on an object using the following calls:

• To set a governance lock onto an object, specifying both lock mode and duration (this overrides any defaults configured on the bucket):
  PUT /<bucket>/<object>?version=<uuid>&objectlock=governance:<untildate>

• To set a compliance lock onto an object and inherit the default duration from the bucket:
  PUT /<bucket>/<object>?version=<uuid>&objectlock=compliance

• To completely inherit the default object lock mode and duration on the bucket:
  PUT /<bucket>/<object>?version=<uuid>&objectlock

• To remove a (governance) object lock from an object, assuming you have the right to do so:
  DELETE /<bucket>/<object>?version=<uuid>&objectlock=<mode>
  The mode will is always be "governance". A compliance mode object lock cannot be removed. Note that in addition you must have the BypassGovernanceRetention permission The BypassGovernanceRetention permission is required to carry out this action and the request must carry the x-object-lock-meta-bypass-governance:true header.

• To query current object lock status:
  GET /<bucket>/<object>?version=<uuid>&objectlock
  The response carries the following headers:
  x-object-lock-meta-mode: <GOVERNANCE|COMPLIANCE>
x-object-lock-meta-retain-until: <date>
  And the response body says:
  Object is locked in <mode> mode until <date>
  When called on an object that is not under retention, none of the headers are present and the response body says:
  Object is not locked

Both S3 and SCSP also allow retrieving object lock information using regular object HEAD and GET requests.
Assuming the user has the GetObjectRetention permission, the information is returned in the form of the above response headers. The response body is not affected.

...

Enabling/disabling legal hold requires that the user has have the PutObjectLegalHold permission. To The GetObjectLegalHold permission is required to check the current legal hold status you need the GetObjectLegalHold permission.

As with retention periods, Gateway stores this as a metadata header.

x-object-lock-meta-legal-hold: ON
(empty means OFF)
x-object-lock-meta-original-lifepoints: <original lifepoints>
lifepoint: [] deletable=no

Using S3, you can enable or inspect the legal hold using the following calls:

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html

• https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLegalHold.html

Using SCSP, you can enable or inspect the legal hold on an object using the following calls:

• To set a legal hold onto an object:
  PUT /<bucket>/<object>?version=<uuid>&objectlock=legal-hold

• To remove legal hold from an object:
  DELETE /<bucket>/<object>?version=<uuid>&objectlock=legal-hold
  This

  will reinstate

  reinstates any original lifepoints by moving them from the x-object-lock-meta-original-lifepoints header back into the proper lifepoint headers.

• To query an object's legal hold status:
  GET /<bucket>/<object>?version=<uuid>&objectlock
  The response carries the following headers:
  x-object-lock-meta-legal-hold: on
  And the response body says:
  Object is locked in legal hold
  When called on an object that is not under legal hold nor retention, the header is not present and the response body says:
  Object is not locked

Both S3 and SCSP also allow retrieving legal hold information using regular object HEAD and GET requests. Assuming the user has the GetObjectLegalHold permission, the information is returned in the form of the above response headers. The response body is not affected.

...

In the SCSP protocol, querying and deleting such combined locks is handled via a uniform GET and DELETE API (as opposed to S3 which has separate APIs for querying/deleting retention and legal hold).

• To query:
  GET /<bucket>/<object>?version=<uuid>&objectlock[=<locktype>]
  When querying the object lock status without specifying the lock type,

  there will be

  response headers for both retention and the legal hold display, and the response body contains both status texts, separated by a new line.
  The user needs both the GetObjectRetention and the GetObjectLegalHold permissions for this request.
  

  You can also query

  Query the lock status for one specific lock type, either "legal-hold" or "retention". In this case only the corresponding permission is required.

• To delete:
  DELETE /<bucket>/<object>?version=<uuid>&objectlock=<locktype>
  Using SCSP

  you can

  remove either the retention or legal hold using DELETE and specifying the appropriate query argument objectlock=<locktype>, locktype being "legal-hold" or "retention". When the locktype is "retention", this can only be done for locks

  that are

  in governance mode.

Differences Between S3 and Swarm's Implementation of Object Locking

In S3, a DELETE request would result in a delete marker, shadowing the locked object version. Swarm's implementation deviates from that logic – it rejects any DELETE requests  requests for undeletable indelible objects with a an HTTP 403 Forbidden error error.

When Gateway gets a  HTTP 403 Forbidden error  error from Swarm, it checks if the object is locked. If it is, Gateway simulates the S3 behavior creating a creating a new (unlocked) object version, immediately followed by a a DELETE, thus creating a delete marker.

For SCSP, you can use a configuration flag to pick the desired behavior:

• fail deletes of locked objects with a 403 Forbidden error, or

• mimic the S3 behavior

[object_locking]
scspDeleteUsesS3Logic=true

...

The following new policy actions related to object locking have been introduced:

• PutBucketObjectLocking: to enable/disable object locking on a bucket

• GetBucketObjectLocking: to query bucket object locking status

• PutObjectRetention: to set or extend object retention

• GetObjectRetention: to query object retention

• BypassGovernanceRetention: to shorten/remove a retention in governance mode

• PutObjectLegalHold: to set/remove a legal hold

• GetObjectLegalHold: to query legal hold

Interactions with Existing Swarm Functionality

...

Clients can request recursive deletes of entire domains/buckets using DELETE <uri>?recursive=yes requests. Swarm implements this by synchronously deleting the domain/bucket object, and asynchronously deleting the objects in it. This has potential for conflicting with object retention/legal hold. Gateway will first check checks if there are any objects under retention/legal hold and refuse the recursive delete if so.

If any buckets with object locks are found, the The recursive delete request fails with a 412 Precondition Failed error if any buckets with object locks are found.

APPEND

SCSP APPEND does create a new version when versioning is enabled; it does not allow an object's metadata to be modified. So for instance, the "retain-until" header value could not be changed on an APPEND. Gateway will does not impose any restrictions on the use of APPEND in combination with object locking.

...

S3 allows defining a max-retention-duration limit in the policy. Gateway currently approximates this functionality using a new configuration flag:

[object_locking]
retentionMaxYears=100
If unspecified, the The default limit value is 100 years (input type int) if unspecified.

• When doing conversions between numbers of days and years, it is assumed

  that

  a year is 365 days.

• In the SCSP/S3 APIs, any user-specified value exceeding the limit is capped to the limit.

Audit Logging

Object Locking operations are audit logged. Since object locks can also be requested as part of the object PUT/POST/COPY requests, Gateway will tag the request's audit log line with additional object lock information, rather than inserting new log lines.

The tags are appended to the audit log line, enclosed in [] brackets. If there are multiple Multiple tags (for example, both legal hold and retention were requested) then they are comma separated by a comma (,).

Object locking tags are always prefixed with OBJLCK.

• Enabling retention on a bucket and setting defaults if provided:
  <audit log line> [OBJLCK:ENABLE:<mode>:<duration>]

• Setting/removing retention on an object:
  <audit log line> [OBJLCK:RETENTION:<mode>:<retainUntil>]
<audit log line> [OBJLCK:RETENTION:NONE]

• Setting/removing legal hold on an object:
  <audit log line> [OBJLCK:LEGALHOLD:ON]
<audit log line> [OBJLCK:LEGALHOLD:OFF]