SwarmNFS 2.x - Listings

Each SwarmNFS export you specify presents the named objects that are stored within a Swarm bucket. For manageability, the exact objects that a client sees when connected to SwarmNFS is filtered, and virtual hierarchies are presented as much as possible.

Listing behaviors

Listing delay — Objects created in Swarm natively (not via SwarmNFS) can take up to 5 minutes plus the Search Feed's Batch Timeout to appear in SwarmNFS listings, because they must be indexed by Elasticsearch. For best listing performance, lower the search feed's Batch Timeout to 1 or 0 (recommended).

Listing limit — How many entries are returned in a folder listing is limited to the value configured in Elasticsearch for "index.max_result_window". SwarmNFS will allow new files to be created even if the number would exceed "index.max_result_window", but, over time, the number of entries listed will fall back down to that configured maximum. See Configuring Elasticsearch.

Exclusive opens — SwarmNFS supports exclusive opens of a file (O_EXCL and O_CREATE) but does not support exclusive reopens (EXCLUSIVE4).

Named Object Listings

To present named object listings as if they were in a traditional file system, SwarmNFS creates a simulation: it translates each forward slash (/) in any object name into a traditional directory delimiter. SwarmNFS then presents a view of objects at the simulated directory level. For example, suppose these named objects exist in the bucket “AcmeBucket”:

Sales/Leads/campaigns/2016/Jan/list.xls
Sales/Leads/campaigns/2016/Feb/list.xls
Sales/Leads/campaigns/addword.xls
Sales/Leads/campaigns/partners.xls

Slash characters

Objects in Swarm Storage use forward slashes in their names for directory delimiting; SwarmNFS converts these to backslashes inline during listing transfers only if a client expects backslashes to be the directory delimiter. SwarmNFS always converts backslashes to forward slashes when communicating with Swarm Storage (including through Gateway).

To be valid, object and bucket names must not start with a slash; if you have existing ones so named, they will not appear in SwarmNFS listings. If this occurs, rename and upload them with valid names.

By default, for a client request for a bucket-level listing, SwarmNFS provides a simulated view of the first level of contents within the bucket:

Each client request to open a specific directory level within the bucket (such as /Sales/Leads/campaigns) returns a new listing for that context:

If the user requests a listing for a file (such as Sales/Leads/campaigns/addword.xls) then only that single file would be returned.

If an end user creates a new directory (such as AcmeBucket/Sales/Leads/campaigns/2016/Mar), SwarmNFS creates a new empty object with Directory attribute metadata, which marks it as a simulated directory.

Important

To delete a directory, end users must first remove any files within it.

To rename a directory, end users must create a directory with the new name, move the files into it, and then delete the directory with the old name. (NFS-607)

Keep in mind that simulated directory objects only exist to support the expectations of end users and applications: objects can be written to a new virtual directory just by including it in the pathname.