SwarmFS is a lightweight protocol converter that seamlessly integrates Swarm scale-out object storage with NFS v4. It combines Swarm's universal, multi-protocol namespace with the power of enhanced and custom metadata, offering new ways to manage, view, and analyze data. SwarmFS provides a traditional file interface to Swarm object storage for content generators (enterprises, researchers, web-based applications, and developers) who use native NFS-based applications to create, access, and manage that content while allowing for the same content to be created, accessed, and managed through modern cloud and object APIs such as S3 and SCSP.

SwarmFS avoids the common problems of traditional gateway and connector file-based storage solutions: protocol and storage silos, bottlenecks, and single points of failure. SwarmFS with Swarm object storage provides, as standard, high availability (HA), data management (from creation to expiration), powerful metadata management, and ad hoc search to the content.

Key Features

Universal namespace - eliminates protocol silos by enabling every object to be written/accessed via NFSv4, S3, or SCSP/HTTP, all without restriction
Lightweight for rapid deployment (VM or physical)
Built-in active/active HA, no clustering required or limitations on the number of active access points
Direct access to read and update an object's metadata over NFS
Stateless - reducing data loss of uncommitted resources
Data is read from and written directly to Swarm. There is no latency and risk of data loss introduced through staging data, as with traditional gateways and connectors
Leverages Swarm’s built-in distributed features for resilience and HA

Benefits

Productivity	Store, access, and manage files	Data portability - multi-protocol in/out through NFSv4, S3, HDFS, or HTTP; read data written through FileFly Streams data directly to and from Swarm (no local gateway storage staging or spooling) Brings rich custom object metadata to file through NFS Mount domains, buckets, or views that are filtered by custom object metadata
Less Risk	Security and scale with no single point of failure	Limitless scale Rapid scale through physical servers, VMs or appliance No storage or protocol silos No read performance performance latency through data staging Multi SwarmFS instance managed through a single pane of glass Security settings in Swarm propagate through all protocols Builtin active/active HA that requires no local disk and no clustering Auto client resume — if there is a communication issue between client and SwarmFS, the client restarts up where it left off
Lower TCO	Leverage Swarm scale-out storage	High availability and data protection are standard automated features Continuous protection with seamless movement between replication and erasure coding Eliminates the need for backups Leverage Swarm’s automated, policy-based data management Automatically replicate content to a remote site for distribution Manage files from creation to expiration WORM, Legal Hold, and Integrity seals are standard

SwarmFS provides distributed file management, with the self-healing modularity of clusters that can have resources come online and go in and out of service with no disruption to user experience:

Simplified Architecture

SwarmFS is a lightweight nfs-ganesha plugin. A traditional gateway or connector has many more moving parts, each adding restrictions, overhead, and complexity:

High Availability

SwarmFS is stateless, so as many SwarmFS instances as needed can be spun up and all NFS instances are active, with no failover or clustering required. High Availability is part of the standard product and no special configuration, clustering or management is required.

Traditional gateways and connectors must stage objects as files locally on the gateway. Disk space on the gateway server is a limitation. SwarmFS does not cache or stage files/objects on local disk space; rather, it streams data directly to and from Swarm. Escape the performance overhead of writing complete objects to a local gateway staging disk, so there is no risk of losing data if the Gateway crashes before data is spooled off the gateway to the object store.

Note

Given the stateless nature of Swarm and SwarmFS, file locking exists only within a single SwarmFS server.

Simplified Security

POSIX: SwarmFS adds basic POSIX security to the modern object security inherent in Swarm. Uniform object security is achieved through and across NFS, SCSP, and S3, with additional POSIX file security when objects are accessed through SwarmFS.

Access Control: SwarmFS supports basic POSIX UNIX-style ACLs (user, group, other); object access control is managed via Swarm Content Gateway. SwarmFS validates the login/password, and it then leaves Gateway to control object access. Native Swarm access means Anonymous NFS access to objects only.

Swarm Documentation

SwarmFS Overview