About This Documentation

The Hammerspace Global File System (GFS) enables a single logical file system to span multiple locations regardless of geographic distance. This allows for visibility and accessibility of Enterprise data to remote sites, regardless of where the data is physically stored, while adhering to access controls and permissions. The goal is to make data a global resource for your Enterprise, instead of files being confined to a single location.

This is an important distinction, since traditional storage limits data to being a local resource and the only way these systems can provide data accessibility to remote locations is by making copies of files. This leads to copy sprawl, significantly increases management complexity, and requires the manual staging and destaging of data.

Metadata and Data Disaggregation

Hammerspace has disaggregated metadata and data within our file system. In doing so, we are able to replicate metadata, the presentation layer of the file system, across multiple locations, making data visible and accessible regardless of where it is physically stored. Logically, it is a single system that is unbounded by distance.

The Hammerspace Anvil metadata servers communicate with one another, providing updates on changes that occur within the GFS every five seconds. Data is moved and/or replicated by policy or on-demand leveraging Hammerspace Data Orchestration.

Data Orchestration

Hammerspace provides Data Orchestration, a powerful capability that efficiently makes data available to remote locations. It leverages Objectives, a policy-driven, file-granular methodology.

The Hammerspace Anvil metadata server is our control plane. We orchestrate data through metadata. You can set policies for data locality based on any file metadata attribute including file name, file type, file size, creation date, modified date, last access, etc. You can also customize metadata and can organize and group files together based on your business requirements for specific projects, use cases and workflows.

Hammerspace replicates instances, not copies. When you copy a file, it is now an entirely different file with the exact same content. An instance is a file that can physically be in two or more locations but to the file system, it is the same file. This is the essential distinction.

Hammerspace uses an eventually consistent mechanism to ensure that all of the instances are aligned. We purposefully did not use global file locking because by its nature, this method over long distances can slow down performance to the point of being unusable. Additionally, if any of the sites participating in the global file lock become unavailable, then all of the data goes into read-only mode, which fundamentally can greatly impact productivity. The Hammerspace eventual consistent methodology does not impede performance, nor is it affected if a site becomes unavailable.

With Hammerspace, file attributes (name, size, permissions, modification times) and custom metadata alike are continuously updated between all sites ensuring that every user across the GFS sees the same file system structure. The data path includes the file content that is replicated asynchronously via a shared Object Storage system. File data moves only when ordered to by a Place-On Objective (proactively) or when a remote user attempts to access the file (on-demand).

Workflow Considerations

The Hammerpace GFS is unique and powerful. As such, some of the concepts and requirements are vital to understand in order to optimally leverage it. It is important to establish workflows that take into consideration its eventually consistent nature.

The use cases include distributed workflows and burst-to-compute across Enterprise data centers, remote sites, edge and Cloud.

This guide discusses this two-path architecture and outlines the best practices to ensure data is placed in alignment with the needs of the workflow including data durability, maximizing performance, and/or enabling workload locality.