Metadata Field Matching

In addition to specifying query arguments, metadata field matching criteria are specified in the URI. This allows for fine-tuning the result set to return objects in the storage cluster matching one or more matching criteria.

Matching criteria are logically AND expressions by default and can be switched to OR expressions using the or=yes query argument. The context for the search, everything after the domain name, and any value for the prefix argument are always considered logical AND constraints for the match. A bucket name or prefix pattern must match even when or=yes is used if specified in the URI.

Wildcards can be used to match field values:

  • ? - wildcard for a single character

  • * - wildcard for multiple characters

The following examples show the different matching concepts.

Matching Examples

Return JPEG images
GET http://{cluster}/ ?format=json &content-type=image/jpeg
Return JPEG or PNG images
GET http://{cluster}/ ?format=json &or=yes &content-type=image/jpeg &content-type=image/png

Using glob-style pattern matching:

GET http://{cluster}/ ?format=json &content-type=image/*
Return JPEG or PNG images in a bucket

{png OR jpeg} AND "pics" bucket:

Match a single positional wildcard

This example finds values including "grey" or "gray":

Searchable Metadata Fields

Each Search Feed indexes metadata for searching, but which metadata depends on how the feed is defined in the Storage UI (or legacy Admin Console). With the Search full metadata checkbox selected, Swarm indexes all available metadata for the objects in the cluster; with it unselected, Swarm indexes basic metadata fields to support listing operations.

Allow for additional storage and RAM on the search servers to support it if Search full metadata is implemented on the search feed.

Tip

Searching on the basic metadata fields can be performed even without Search full metadata enabled.

Basic Metadata Fields

The following table provides a list of the standard, baseline field names as they are mapped between the name used in the query argument values and the name given in the XML and JSON output formats. Notice that the output name may be different from the name used in the query argument and that the output name can change depending upon the output format.

Query Argument

XML Name

JSON Name

Description

Query Argument

XML Name

JSON Name

Description

tmBorn

LastModified

last_modified

Time of create or last update.

The query argument may use either UTC date-time or Unix timestamp (float) in search requests.

The microseconds and time-of-day portions of a UTC date-time are both optional.

content-length

Size

bytes

Size in bytes

name

Key

name

UUID or name using URL encoding

content-type

content-type

content_type

Content type

etag

ETag

hash

Entity tag

sizewithreps

sizewithreps

sizewithreps

Number of bytes using the maximum reps value

Full Metadata Fields

The following list shows the metadata field names that are indexed for full metadata search. Custom Metadata Headers are included in these patterns.

  • castor-* (except castor-system-*)

  • content-base

  • content-disposition

  • content-encoding

  • content-language 

  • content-location

  • content-md5 

  • lifepoint

  • x-*-meta[-*]

These fields do not show up in listings unless explicitly included using the fields query argument, like this:

See Search Operations.

Case: While the metadata field names are case-insensitive for the purposes of matching, they are stored in the cluster as given during the WRITE operation. The metadata field values are case-sensitive.

Hyphen Conversion

Custom metadata field names that contain hyphens (-) have these characters converted to underscores (_) in the result output. Swarm allows variants with either hyphens or underscores on input, but it favors underscores on output. 

For multipart uploads, both Castor-System-Uploadid and Castor-System-Partnumber allow query argument2 to use either hyphens or underscores in the field name, as is supported for content-type. (v10.2)



© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.