Listing Operations

Listing operations are a specialized class of searching that usually have a context constraint of a domain or a bucket, except in the case of listing untenanted unnamed objects or finding annotation objects. When performing listing operations, the user is typically interested in the hierarchy or membership within a context. Since listing operations are in fact searches, you can use other searching options and metadata constraints in combination with them.

Encoding

When non-ASCII characters are included, list query response bodies are UTF-8 encoded. When reading and writing non-ASCII characters, applications must decode the response body from UTF-8 prior to interpreting the list body.

Listing domain contents

Domains are a context that contains bucket objects and unnamed objects. Buckets are identified by name. Unnamed objects are identified by UUID and are either mutable (alias) or immutable.

Basic form of listing a domain

GET /?format=json&domain=myDomain

Filter buckets that start with a string

GET /?format=json&domain=myDomain&prefix=Southwest_

Listing unnamed objects within a domain

GET /?format=json&domain=myDomain&stype=unnamed

Listing everything within a domain

GET  /?format=json&domain=myDomain&stype=all

Note

Swarm assumes stype=bucket when listing a domain, so it returns a list of bucket names unless you request a specific stype.

Listing a bucket context

Buckets are a specific context type that belongs to a domain and can contain only named objects. This operation is similar to listing a domain context with additional URI element of the bucket name added.

Listing bucket contents

GET /myBucket?format=json&domain=myDomain

Note

Swarm assumes stype=named when listing a bucket because no other stype is valid.

Listing untenanted unnamed objects

If the storage cluster includes unnamed objects that are not contained in a domain (untenanted), you may still list those objects by specifying an empty domain context.

Listing untenanted unnamed objects

GET /?format=json&domain=&stype=unnamed

Storage in use

You can dynamically query the storage in use for a domain or bucket context using the du query argument. This is valid for domains and buckets.

du=withreps requests the total storage impact of objects.
size=0 prevents the return of the request body (the calculated value is returned in the header).

Argument

Header Result

Description

du=withoutreps

Castor−Bytes−Used: 2189243

For du=withoutreps, the HTTP/1.1 response header returns a Castor-System-Bytes-Used field that indicates a summary of the storage used by the objects within that context.

Swarm calculates the space from the sum of the body bytes of all relevant and distinct objects. This calculation does not consider the number of replicas for each distinct object. .

du=withreps

Castor−Bytes−Used−With−Reps: 109416625

For du=withreps, the HTTP/1.1 response header returns a Castor-System-Bytes-Used-With-Reps field that indicates a summary of the storage used by the objects within that context.

Swarm calculates the space from the sum of the body bytes of all relevant objects using each object's maximum reps value from the object lifepoint headers and the assigned value from the cluster multiplied by the object's size. A 100 MB object with reps=2 will consume 200 MB of space. The same object with reps=3 will consume 300 MB of space. Stored with an erasure coding value of reps=4:2, the object will consume 150 MB of storage.

If the object has three lifepoints that include all of these previous example values, the maximum of reps=3 will be chosen for the calculation and the storage impact will be recorded as 300 MB. Every object has the metadata field sizewithreps that records its space impact.

Important

The Castor-System-Bytes-Used and Castor-System-Bytes-Used-With-Reps fields are computed on-demand. Depending upon the object count within the context, they can consume computational resources on the search servers. Applications should only request du operations when the space calculations are required.

Domain storage in use

When querying the storage in use for a domain, you have the option of selecting the types of objects to consider using the stype argument.

Argument summary:

du=withreps requests the full storage impact of objects.
size=0 prevents the return of the request body (the calculated value is returned in the header).
stype=named|unnamed|all selects the types of objects included in the calculation.

Space used by all content in domain

GET /?format=json&domain=myDomain&du=withreps&size=0&stype=all
 
HTTP/1.1 200 OK 
Gateway-Request-Id: 26F809F67D883E6D 
Content-Type: application/json; charset=utf-8 
Castor-System-Object-Count: 17 
Castor-System-Bytes-Used-With-Reps: 121590 
[snip]

Space used by named objects

GET /?format=json&domain=myDomain&du=withreps&size=0&stype=named 
 
HTTP/1.1 200 OK 
Gateway-Request-Id: C6A8D293950C1FD5 
Content-Type: application/json; charset=utf-8 
Castor-System-Object-Count: 12 
Castor-System-Bytes-Used-With-Reps: 121422 
[snip]

Space used by unnamed objects

GET /?format=json&domain=myDomain&du=withreps&size=0&stype=unnamed 
 
HTTP/1.1 200 OK 
Gateway-Request-Id: 3D79D93B73A07E35 
Content-Type: application/json; charset=utf-8 
Castor-System-Object-Count: 2 
Castor-System-Bytes-Used-With-Reps: 168 
[snip]

Bucket storage in use

When querying the storage in use for a bucket, there will only be named objects within the bucket context. The stype argument is not required.

Argument summary:

du=withreps requests the total storage impact of objects.
size=0 prevents the return of the request body (the calculated value is returned in the header).

Calculating space used by named objects in a bucket

GET /mybucket?format=json&domain=myDomain&du=withreps&size=0 
 
HTTP/1.1 200 OK 
Gateway-Request-Id: 100601F9E31D5ECC 
Content-Type: application/json; charset=utf-8 
Castor-System-Object-Count: 1000 
Castor-System-Bytes-Used-With-Reps: 22016 
[snip]

Annotations in existence

To retrieve any annotation objects that may exist in the cluster for a given object, submit a listing query that sets the argument "decorates" equal to the ETag of the target object in question:

Listing annotation objects for given ETag

GET /?format=json&domain=&decorates=8c2c582c216a1f088c3652bced5a5f91

See Metadata Annotation.