Versions Compared

Old Version 13

changes.mady.by.user Bala Harish A(AC)

Saved on

compared with

New Version Current

changes.mady.by.user Bala Harish A(AC)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Argument

Values

Usage

Example

format

json

xml

Info

Required

Requests a search query and specifies the desired output format. This query argument is required to trigger a domain/bucket listing query.

This format applies to the body portion of the HTTP/1.1 response and does not affect the format of the response headers. All search operations are available with either format.

&format=json

&format=xml

context

string

Filters the search to certain buckets one of these ways:

  • Greater/less than comparison, for buckets in numbered ranges: 0123, 0124, 0125, ... (context>=0124)

  • Wildcard matches, for buckets matching a prefix: us-tx, us-ky, us-tn, us-ga, ... (context=us-*)

    • Wildcards are supported for matching on prefixes, so use an asterisk at the end of the string.

    • Wildcard matching across many buckets may impact performance.

To restrict search to a bucket, include the bucket in the URL path. Always use with use domain={domain-name}.

&context<=example.com/2015

&context=example.com/foo*

contextid

UUID

Specifies a search filtered to the context identified by the UUID given.

&contextid=80a4957…

decorates

UUID

Used in a listing query to find any annotation objects that may exist for a given ETag (or earlier query result “hash”). See Metadata Annotation and Listing Operations.

&decorates=a95780a4…

domain

domain-name

In a normal SCSP storage request, the required Host header is the assumed domain (or context) for the operation. Most HTTP/1.1 clients and browsers use the host portion of the URL as the value for the Host header in the Swarm request. Supply the domain argument to explicitly override it if the value of the Host request header does not match the domain name. A domain argument always has precedence over the Host header in the HTTP/1.1 request.

Set the value equal to nothing (empty string) to find unnamed objects not tenanted in any domain.

&domain=example.com

&domain=

domains

none (ignored)

Used for listing the domains within a cluster. No value is used with this argument.

&domains

du

withoutreps
|yes|true

withreps

Dynamically queries the disk usage for a domain or bucket context. With du, use the size=0 query argument (to shorten the output by preventing the return of the request body) and add any filters needed. The computed result value appears in the Castor-System-Bytes-Used[-With-Reps]   response header.

  • withoutreps sizes the content uploaded: it finds the sum of the bytes of the unique objects being stored.

  • withreps sizes the storage impact: it finds the underlying disk space impact of the objects stored in a domain or bucket context, a value weighted by the expected number of replicas in the cluster, which is an approximation of the data footprint of the results.

See Storage in Use for how to query the storage in use for domains and buckets.

&du=withreps&size=0

field‑name

any indexed metadata field, such as: 

content-length

Filters results to those matching the operations on one or more specified fields. Multiple filters are joined by a logical AND; to specify OR, add the &or argument.

Supports standard comparison operators on specific fields to filter the results returned:

  • a = b, equal to  

  • a <= b, less than or equal to  

  • a >= b, greater than or equal to

This is an overloading of normal query argument processing.

&x-width-meta<=800 &x-height-meta<=600

&content-type=image/bmp&or &content-type=image/png

&content-length>=1073741824

fields

Comma-separated list of field names, or all

Replaces the default fields and specifies, as a comma-separated list, which fields to display in the output (see See Metadata Field Matching).

Use fields=all on bucket and domain listing requests to return all available fields on each record in the response. (v9.2)

&fields=name,content-length

&fields=all

index

yes/true to activate, no/false to inhibit

On a write (POST, PUT, COPY, APPEND, DELETE) request, forces immediate search indexing of the newly written/deleted object.

Appears on Gateway requests when enabling the Gateway Configuration option  EnhancedListingConsistency. (v9.3)

&index=yes

sync

now or wait to activate, no/false to inhibit

On a read (listing query GET) request, forces consistency of results returned on the listing.

  • now performs a refresh on the index in Elasticsearch immediately before preforming the listing query.

  • wait delays execution of the listing query to provide Elasticsearch time to refresh the index.

Appears on Gateway requests when enabling the Gateway Configuration option EnhancedListingConsistency. (v9.3)

&sync=now

marker

empty value or a comma-separated list of sorted column key values

Provides a mechanism to use multiple requests to receive a complete result set. Defaults to &sort=name.

Used with the size argument to paginate large result sets. Use an empty key to begin a new search. Use the last sort key value of the results on the next request to continue pagination from that place on the next request.

&marker=&sort=tmBorn&size=50

or

yes/true
to activate

Joins them with a logical OR (default is AND) when used with multiple field comparisons. Defaults to false.

&or

prefix

string

Matches the string to the start (prefix) of the object's name or UUID. The names of the objects in the output list share the same prefix string.

  • Named objects are always contained within a bucket, and the first "/" following the bucket name is the delimiter. Any additional slashes following the first slash are part of the object's name. That is, for /photo/Q1/apple.jpg, where photo is the bucket, retrieve the object using prefix=Q1.

  • Unnamed objects require the additional argument stype=unnamed to match on UUIDs.

To find untenanted unnamed objects, include "domain=" (empty string).

&prefix=Q1

&stype=unnamed&prefix=93f

&stype=unnamed&prefix=93f&domain= 

size

integer

Constrains the number of results to be returned by the query. Defaults to 1000. This argument is equivalent in purpose to the "limit" or "max-keys" arguments in other cloud protocols.

The maximum is 50000 but we do not recommend using more than 10000.

Info

Important

This is the size of the result set, not the object size. To work with object size, use the field content-length.

Tipinfo

Tip

Set size to 0 when no actual listing is needed, such as when using the "du" query argument.

&size=5000

sort

Comma-separated list of field names, with optional modifier :asc|:desc

Sets the sort order of one or more fields, with the first field sorting first. Sorting direction defaults to ascending (:asc); to specify descending (:desc), add the modifier to one or more of the fields.

Info

Important

Sorting exacts a computational penalty, so sort the output only when necessary.

&sort=size:desc,name

stype

Comma-separated list of Swarm types

Specifies the type of objects being requested in a search. Defaults to all.

Valid values: domain, bucket, named, alias, immutable, unnamed, all

&stype=unnamed

versioned

true|false

Allows filtering updateable (named and alias) objects by versioning status, checking whether versioning was enabled in the context at the time they are written. Checks the value of the Castor‑System‑IsVersioned header. (v10.0)

&versioned=true

versions

versions&name

versions&prefix

true|(no value)
| previous

name of object

first part of  name

True (or no value) requests a listing of all versions of all objects, including the current.
Previous requests past versions (v10.2).

Requests a listing of all versions of the named object.

Requests a listing of all versions of all objects matching the prefix.

See Object Versioning.

&versions

&versions&name=logo.jpg

&versions&prefix=2019Q1/

...

Marker Argument

The markerargument provides a mechanism to retrieve a single result set using multiple HTTP requests. This is useful when a large result set is impractical to receive using one HTTP request. All marker operations work using a sorted result set. The system implicitly usessort=nameif a sort argument is not provided. The size argument is used to control how many items are returned per HTTP request.

On the first request,marker= (without a value) is used to indicate a request at the beginning of the result set. For subsequent requests, the last value(s) from the sort field(s) is used to indicate the last record received from the previous request.

Tipinfo

Tip

Use a UTC date-time or Unix timestamp (float) when usingtmbornas the marker. The microseconds and time-of-day portions of a UTC date-time are both optional.

Marker Example

Consider the following set of object names in the bucket dictionary.

...

Code Block
GET /dictionary?format=json&domain=example.com&size=4&fields=name&marker=defender [ ]

Sort vs Marker

...

Relationship

The marker value is either the last value or values from the sort fields. The marker also contains the last field values in it if using:
sort={field1},{field2},...,{fieldN}

...

DataCore Swarm cannot return more results than specified by ?size and can only return objects after ?marker so the sort needs to be a total ordering.

Determining

...

Completion

The set of objects in the cluster is continually changing so list/search requests are never complete. A criterion exists for considering marker requests complete. The Castor-System-Object-Count response header returns the number of objects remaining to enumerate including the records retrieved by the current request.

A client may consider the iterating request complete when the CastorObject-Count value equals the number of records retrieved by the current request.

Markers

...

Across the

...

Domain

A strategy for using markers to retrieve a result set of a search across the entire domain is to sort based on a unique tuple of metadata fields all objects have. For example,

Code Block
languagexml
sort=context,name

See Baseline Metadata FieldsField Matching.

The name field is a UUID or an application-assigned string depending upon the object type. Anchor_stype_stype

...

Stype Argument

The stypeargument allows filtering search results to certain types of objects. The following table lists the stype argument values:

...

Use the unnamed, immutable, or alias value of the stype to select the desired set to list the unnamed objects in the domain. Include domain= set to nothing (empty string) to find untenanted unnamed objects. Untenanted unnamed objects return an empty string for the domain field in search and listing results.

Info

Important

Unnamed objects are guaranteed to be tenanted in a domain when the cluster.enforceTenancy configuration option is enabled. Include domain= (empty string) to find untenanted objects.