...

Listing Cache (LC) is a performance optimization feature designed to improve the speed of listing large datasets within Swarm storage. It works by caching pseudo-folder listings, reducing the time and resource consumption required to fetch and display object listings repeatedly.

The listing cache Listing Cache solves a scalability problem with the gateway's delimited folder listing functionality. To determine if a folder has subfolders, an Elasticsearch query has to enumerate all objects with the folder name as a prefix to their object names. This can run into the millions of objects for large buckets. When such queries are issued repeatedly and at high frequencies, the resulting CPU use brings an entire Elasticsearch cluster to a halt.

Limitations

• Client-Specific Bindingspecific binding: Bound to a dedicated client, with no cross-gateway sharing allowed. Once you decide to serve 1 or more domains on a listing-cache enabled gateway it must serve all requests to those domain(s) exclusively. This is achieved by configuring your load-balancer with dedicated host based traffic redirection rules.

• Non-Persistent Cachepersistent cache: The disk/memory cache is discarded by default on restart.

• Limited Lifecycle and RecursiveDeletion Supportlifecycle and recursivedeletion support: No support for bucket lifecycle policies, delete lifepoints, or recursive deletes. All writes and deletes must originate from the gateway.

• Memory Constraintsconstraints: Caching large volumes of data can quickly consume system memory. Misconfiguring cache sizes can lead to memory exhaustion or excessive eviction, reducing cache effectiveness.

• Delimiters Supportsupport: Custom delimiters are not yet supported, only forward slash "/".

• Replication Supportsupport: <pending engineering feedback>, right now do not setup replication when LC is enabled.

• Not supported functionalities: Custom delimiters, S3 lifecycles, and recursive deletes.

...