Use Cases and Architectures

Most use cases for Swarm involve ingesting petabytes of unstructured data, such as image, video, and document files, which must be secured, preserved, searched, and retrieved on demand.

  • Active Archive: Video evidence, medical imaging, and cultural media

  • Cloud: Cloud services, hosting (multi-tenant), and backup to the cloud

  • Content Delivery: Social media (millions of photos per day), streaming video (millions of videos), and content publishing (millions of images)

  • Big Data: Evidence analysis, medical insurance records and analysis, IoT/M2M and analytics

  • Compliance: Legal documents, court materials, and digital evidence

Swarm supports many usage scenarios based on four fundamental access methods:

Direct Access

Native

(SCSP API)

  • Native client/application integration using a vendor API (RESTful HTTP 1.1 compliant)

  • The application works directly with the object store

Web Access

Content Gateway

(S3 API)

  • Data in the object store is presented via web browser (Content UI)

  • S3 endpoint is provided

  • Support for actions such as upload, download, and browse

  • Back-end access to the object store are native API calls

File Protocol
Gateway

SwarmFS

(to Native SCSP)

  • Provides translation of traditional file protocols (such as NFS and SMB) to object storage protocols

  • Usually translates to object store native API

  • Used as a “drop box” target for clients or applications coded to work with traditional filers

  • Advanced protocol gateway support for manipulation of metadata, in addition to data via traditional utilities (such as shell sessions)

  • Placement into object store supports alternate access methods (SCSP or S3) and metadata queries, listings, and collections

Automated
Tiering

FileFly

  • Application/agent integration (native API integration)

  • Agents exist on data sources (such as filers)

  • The relationship between local file reference and data stored at object tier is maintained by agent software

  • Data is moved from local to object tier based on policy (scheduled or ad hoc)

  • Retrieval of data when client requests access is automatic and transparent

These are common architectures for object storage:

Archiving

  • Medium to long term storage

  • “Write once, read rarely”

  • Library of unstructured data (documents, graphics, pictures, and videos)

  • Query and list, based on metadata tags

  • Conduct “Data Lake” analysis (by pooling a vast amount of raw data in a native format)

Data Tiering

  • Relocation of data from traditional filers to object storage

  • Scheduled tiering based on policy

  • Automated recall when an access request is made

  • Transparent access to the end user

  • “Cheap and deep” object-store tier to reduce filer expansion costs

FileFly and Virtualization

Remote Replication and Disaster Recovery

  • Automated replication from a local object store to a remote object store

  • Data is usually populated in a local store, then replicated to remote or DR

  • “Hot” sites can also act as replication targets or DR for each other

  • Can be whole-site replication or policy-based (per domain)

  • Varying complexity in replication topologies supported (site-to-site, M to N, single or bi-directional)

Dual Site with Single Interface

Dual Site with Dual Interface

Managed Service (“Storage as a Service”)

  • Storage protocol endpoints made available to service subscribers

  • Support for multiple RESTful access protocols

  • SSL/TLS

  • Provides authentication and authorization

  • Allows for metering and billing

  • Supports quota control

  • Multi-tenancy (individuals, business organizations, and business units)

Hybrid Cloud (Local Storage with Cloud)

  • A local object store integrated with a cloud service endpoint (such as Azure)

  • “Copy to Cloud” for backup and/or publication of data

  • “Retrieve from Cloud” for data recovery

  • Lower CapEx when meeting backup, replication, or DR requirements



© DataCore Software Corporation. · https://www.datacore.com · All rights reserved.