How to apply XForm metadata transformations

Owned by Tony Lokko (Unlicensed)
Last updated: Oct 27, 2022 by Tony Lokko

Content Gateway supports "metadata transformation", which is a set of rules that adds custom metadata to every object in a domain or bucket.

The feature is documented here:

The feature is used in conjunction with the Management API and SCSP on the Content Gateway, and has been available since Gateway version 3.

Designing the metadata

Use XForm documents when you need to apply certain metadata to all objects in a domain or a bucket (depending on where you apply them).

The XForm document itself is a JSON document that defines the custom metadata. At its simplest, the document would look like this.

{
	"metadata": {
		"X-Color-Meta": "blue"
	}
}
xform.json (END)

This form would apply the metadata X-Color-Meta : blue to every file uploaded to a bucket or domain.

A more practical use case would be to add a company identifier or tag for situations where multiple tenants share a cluster.

{
	"metadata": {
		"X-Company-Meta": "MetaCorp, Inc."
	}
}
xform.json (END)

While assigning static metadata, you can make use of several variables:

Variable Name

Description

${date:format}

Create/update time stamp where format is defined by Java SimpleDateFormat specification

${user}

Authenticated user ID

${domain}

Domain name

${bucket}

Bucket name

Using these can result in very useful applications. For example:

{
	"metadata": {
		"X-Written-When-Meta": "${date:yyyyMMdd-HHmmss}",
		"X-Contains-Meta": "${domain}/${bucket}",
		"X-Copyright-Meta": "Copyright ${date:yyyy}, MetaCorp, Inc",
		"X-Author-Meta": "${user}"
	}
}

This JSON causes any file uploaded to the domain to have the additional metadata highlighted below: 

Installing the XForm

To use the feature, upload your XForm document with the correct method and headers:

  1. Create the XForm file with the definitions for your custom metadata, described above.
  2. Important: Validate the syntax of the JSON you are applying. Use a JSON validator such as https://jsonlint.com.
  3. Save it as xform.json or similar.
  4. Verify that the user that will upload to the Gateway has the permissions on their policy to be able to PUT a policy or GET a policy.
  5. The XForm is an /etc document, which you upload via the management API using the /_admin/manage path:
    • If your Gateway IP is 10.0.0.10, the starting URL is http://10.0.0.10/_admin/manage
    • If your Gateway resolves from cloud.example.com, the starting URL is http://cloud.example.com/_admin/manage/
  6. To etc documents in a tenant, use this form (see Namespace Structure):
    • http://cloud.example.com/_admin/manage/tenants/{tenantname}/etc/{insertyourjson}

  7. The application of the XForm metadata doc is a HTTP PUT, shown here as cURL:

    curl -X PUT --data-binary @xformcorp.json -u tlokko -H "Content-Type: application/json" https://cloud.example.com/_admin/manage/tenants/cloud/domains/d1.cloud.example.com/etc/metaxform.json

Usage:

curl -X PUTThe HTTP verb to PUT an object
--data-binary @xformcorp.jsonSpecifies reading the body of the xformcorp.json from a local file
-u tlokkoSpecifies the user that will PUT the file (must have that policy permission)
-H "Content-Type: application/json"

Required: This is a custom header for the file, and it is very important: If this content type isn't there, Gateway will fail to read the contents of the JSON correctly and writes to the Gateway may fail.

https://cloud.example.com/_admin/manage/The Management API URI
...tenants/cloud/The base tenant
...domains/d1.cloud.example.com/The domain we're applying it to
...domains/d1.cloud.example.com/etc/metaxform.jsonThe end filename
...domains/d1.cloud.example.com/buckets/b1/etc/metaxform.json

Applies it only at the bucket level

Precedence

Any domain-level XForm will supersede a bucket-level XForm.



© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.