Search the docs

Hammerspace Metadata Fundamentals

In this section, we will review some key fundamental concepts regarding Hammerspace metadata.

Hammerspace Metadata

Hammerspace manages three classes of metadata:

  • POSIX - includes the directory structure, file names, and basic file properties.

  • Internal - includes additional statistics, file instance locations, etc.

    • More detailed file usage data (beyond atime and mtime)

    • Extensive inode information - e.g. volume(s) and path(s) to instances

  • Custom - rich metadata that can be defined by the administrator or user.

    • Can be manually created or extracted and applied by script

    • Customer or third-party tools and applications can directly access and manipulate custom metadata

Custom Hammerspace Metadata

There are four types of custom metadata: tags, attributes, labels, and keywords. The acronym TALK can be used to remember them. The distinction between the types is two dimensional:

  • Whether there is a predefined schema or set of metadata that the user can apply

  • Whether the type uses a simple string or key-value pair

Schema Non-Schema

Simple Strings

Label

Keyword

Key-Value Pairs

Attribute

Tag

Attributes are defined by Hammerspace and are generally not configurable by the administrator, although they can be applied and values set by the user.

Interacting with Metadata

It is important to note that Hammerspace does not change the rules with regard to interacting with filesystem metadata. The following rules apply to Hammerspace just as they would with any other storage system:

  • Users can only see metadata of files to which they have read access.

  • Users can only manipulate metadata of files to which they have write access.

Shadow Namespace

Hammerspace metadata can be accessed and manipulated by Linux clients through the shadow namespace that exists in any mounted Hammerspace share. Hammerspace implements a proprietary file naming extension that invokes calls into the shadow namespace. If a user reads a file adding ‘?.' (question mark and dot) and some valid Hammerspace metadata reference to the filename, the filesystem will return metadata (or take action on metadata) based on the metadata reference rather than the file contents.

For example, if a file called telemetry.txt exists in the current directory, cat telemetry.txt would return the contents of the file. However, cat telemetry.txt?.attribute=inode_info would return extensive information about the inode (fileheader) and other internal Hammerspace metadata about the file.