How to use the S3 Object Locking feature

This guide presents the steps required for configuring and using the S3 Object Locking feature using curl and the AWS CLI.

Prerequisites

Before proceeding with configuring object locking please perform the following steps

1. Make sure you have a Swarm 14.0+ cluster with a search feed pointing to an Elasticsearch 7+ cluster; also make sure to issue the requests in this guide through a Content Gateway 7.6 or higher.

2. Create a tenant, a domain in that tenant and a bucket, using either the Content UI or the Content Gateway’s Management API, as follows:

   1. Create a tenant.

      curl -XPUT -i -u caringoadmin:caringo "http://192.168.8.109:9090/_admin/manage/tenants/example.tenant.com"

   2. Create a domain within that tenant.

      curl -XPUT -i -u caringoadmin:caringo "http://192.168.8.109:9090/_admin/manage/tenants/example.tenant.com/domains/locking.example.com"

   3. Create a bucket in the domain.

      curl -XPUT -i -u caringoadmin:caringo "http://192.168.8.109:9090/_admin/manage/tenants/example.tenant.com/domains/locking.example.com/buckets/b1"

3. Enable versioning at the cluster-level. You can do that through the swarmctl utility available in our support bundle https://support.cloud.datacore.com/tools/swarm-support-tools.tgz, by running the following command from within the dist/ folder.

4. Enable versioning for the newly created domain.

5. Enable versioning for the newly created bucket.

6. In order to use the S3 API, create an S3 token for the user that’s going to manage object locking on the bucket. In the following examples, that role will be played by someuser. Write down the token UUID outputted by the request below which will be your S3 AWS_ACCESS_KEY_ID and the secret that you’ve configured for this token, which will be your S3 AWS_SECRET_ACCESS_KEY.

7. The user that will manage object locking on the bucket needs to be either the bucket’s owner or a user with admin-level permissions for that bucket. The following requests grant all privileges on the bucket to someuser, which is the user that we’re going to employ in the next steps.

Configuring Object Locking through the SCSP API

A. Enabling object locking on the bucket and all subsequently created objects

In this case, the object locking parameters that you will apply through the following request will be automatically applied to all the objects that you write onwards, unless you explicitly set the object-locking headers when writing an object. In the latter case, Gateway will always use the maximum of either the bucket default retention duration or the duration specified in a per-object request.

Enabling object-locking at the bucket level through SCSP requires the following steps:

1. Do a PUT /<bucket>?objectlock=<defaultmode> [ :<defaultperiod> ] where

   1. defaultmode can be either "governance" or "compliance"

   2. defaultperiod is optional; it is a number of years (y) or days (d), eg. 1y or 20d.

2. (Optional) Check that object-locking was enabled at the bucket level by issuing a GET /<bucket>?objectlock and checking that the following headers exist and that the response body says: Object locking is enabled on bucket

   1. x-object-lock-meta-status: ENABLED

   2. (Optionally) x-object-lock-meta-default: <GOVERNANCE|COMPLIANCE>[:<duration>]

All the objects that you write/modify after this, will have the bucket-level object-locking specification automatically applied to them. To verify this do the following:

1. Create a new object in the bucket.

2. Make sure the object is locked according to the locking configuration defined at the bucket level, namely that it has the following headers:

   1. X-Object-Lock-Meta-Mode: COMPLIANCE

   2. X-Object-Lock-Meta-Retain-until-date: <current date + 50 days>

B. Enabling object locking on the bucket without automatically setting it on the objects written afterwards.

For users that want object locking enabled on the bucket, but not automatically applied to all subsequently created objects/versions, the following request needs to be performed.

In order to verify that object-locking is not automatically set on a new object (i.e. new objects are unlocked unless the user explicitly locks them when creating them) do the following:

1. Create a new object.

2. Check that it does not have the X-Object-Lock-Meta-Mode and X-Object-Lock-Meta-Retain-until-date headers.

At this point you can create another object for which you can define your object-locking parameters of choice by setting the X-Object-Lock-Meta-Mode and X-Object-Lock-Meta-Retain-until-date headers, as follows:

1. Create the object, making sure to set the aforementioned headers.

2. Make sure that the object has the 2 headers, namely that it is locked.

C. Enabling legal hold for a newly written object

Enabling legal hold on a newly created object requires setting the x-object-lock-meta-legal-hold: ON header when writing the object. For this particular use case, you need to perform the following steps:

1. Write the object with the aforementioned header.

2. Check that legal hold was applied on the object by issuing a HEAD request and confirming the presence of the following headers:

   1. X-Object-Lock-Meta-Legal-Hold: ON

   2. Lifepoint: [] deletable=no

   3. X-Object-Lock-Meta-Original-Lifepoints: <if you've added other lifepoints, they should be in this list>

D. Managing the retention settings of an existing object

The workflow for managing the retention settings of an existing object is generally the following (in these examples we will use an object for which no object locking was set, stored in a bucket that has object-locking enabled, but not enforced, namely X-Object-Lock-Meta-Retain-until-date is not set on the bucket):

1. List the versions, noting the hash field returned by the listing requests, which is the versionId/etag of that particular version.

2. Set your desired lock (governance | compliance) and the retention period on one of the versions. In this example, we will set a governance lock with a retention period of 30 days. This is done through the ?objectlock=governance:30d query argument.

3. Try to remove the governance lock, making sure to set the x-object-lock-meta-bypass-governance-retention: true header and the ?objectlock=retention query argument.

4. Set the version’s locking to compliance with a retention period of 40 days.

5. Try to reduce the retention period to 30 days, confirming that this is not allowed.

6. Try to increase the retention period to 50 days, which is allowed.

7. Check the version’s locking status through a GET request that includes the ?objectlock argument.

8. Change the bucket’s locking mode to compliance with a retention period that is less than what we’ve set for the object at step 6 (e.g. 30 days).

9. Try to change the version’s locking mode to compliance, without specifying the retention (i.e. use this query argument ?objectlock=compliance), which means that it should inherit the retention set at the bucket level. Since at the bucket level we have set retention of 30 days, which is less than the 50 days that the version currently has, this request should fail:

10. Change the bucket’s locking mode to compliance with retention higher than what the object currently has (e.g. 60 days).

11. Now change the version’s locking mode to the bucket’s default (use ?objectlock=).

12. Confirm that the object has a retention period that matches the bucket’s (i.e. 60 days).

E. Managing the legal hold settings of an existing object

The typical scenario for managing the legal hold settings of existing versions consists of the following steps:

1. List the object’s versions.

2. Set legal hold for one of the object’s version using the ?objectlock=legal-hold query argument on a PUT request.

3. Check the version’s locking status through a GET request that uses the ?objectlock query argument.

4. Remove the version’s legal hold, through a DELETE request that includes the ?objectlock=legal-hold query argument.

5. Confirm that the legal hold has been removed.

Configuring Object Locking through the S3 API

A. Enabling object locking on the bucket and all subsequently created objects

If you want to enable object locking on the bucket and have it applied automatically to all objects written afterwards, then the typical usage scenario is the following (NOTE: You will still be able to set object-locking on a per-object basis and the Gateway will always use the maximum of either the bucket default retention duration or the duration specified in a per-object request):

1. Use the put-object-lock-configuration S3 API call for enabling locking using the desired mode and retention period.

2. Check the bucket’s locking status using the get-object-lock-configuration S3 API call.

3. Create a new object in that bucket using the put-object API call.

4. Check the object’s retention period using the get-object-retention S3 API call, making sure it matches the locking settings applied to the bucket. NOTE: This will return the retention settings of the object’s current version. If you need to check the retention settings of a different version, specify that using the --version-id CLI option.

B. Enabling object locking on the bucket without automatically setting it on the objects written afterwards

If you want to enable object locking on the bucket without having it applied automatically to all objects written afterwards, then your typical usage scenario will look like this:

1. Use the put-object-lock-configuration S3 API call for enabling locking without setting a mode or retention period.

2. Check the bucket’s locking status using the get-object-lock-configuration S3 API call.

3. Create a new object in that bucket using the put-object API call.

4. Check the object’s retention period using the get-object-retention S3 API call, making sure the object has no retention period set.

5. Set a retention period for the current version of this newly written object using the put-object-retention API call.

6. Using the get-object-retention S3 API call, confirm that the new retention period is now in place.

7. Alternatively, you can set the object lock mode and retention period at write time, using the --object-lock-mode and --object-lock-retain-until-date

C. Enabling legal-hold on newly written objects

Using the AWS CLI, you can set legal-hold while writing an object by setting the --object-lock-legal-hold-status option to ON:

Checking the legal hold status of the newly written object can be done using the get-object-legal-hold API call:

D. Managing the locking settings of existing versions of an object

The typical workflow when managing the locking settings of an object’s version using the AWS CLI is the following:

1. List the object’s versions using the list-object-versions API call, noting the VersionId of the version for which you want to manage the locking settings.

2. Set a GOVERNANCE locking mode on one of the versions (in this example we will use the 2nd version) with a retention period until 2022-01-01T00:00:00. This can be done using the put-object-retention API call.

3. Using the get-object-retention API call confirms that the new settings are in place.

4. Remove the Governance lock using the put-object-retention call, with the --bypass-governance-retention flag set.

5. Confirm that the lock has been removed.

6. Set a Compliance mode object lock with a retention period of 90 days.

E. Managing the legal-hold settings of existing versions of an object

The typical workflow in this case consists of the following steps:

1. List the versions of an object and choose the version for which you want to apply the legal hold (in these examples we will use the 2nd most recent version.

2. Enable legal hold on that version using the put-object-legal-hold API call.

3. Check the version’s legal hold status using the get-object-legal-hold.

4. Remove the legal hold lock.

5. Confirm that the legal hold lock has been removed.

Related articles

https://caringo.atlassian.net/wiki/spaces/public/pages/2859925505

https://caringo.atlassian.net/wiki/spaces/KB/pages/2739666967