Search the docs

Tier 0 Superpowers

These superpowers are listed from the most basic, to the most sophisticated. Each builds on the previous capabilities.

Make Use of Stranded Capacity

The most basic level of utility for Tier 0 is putting data on any drive on any node in the cluster without regards to affinity to nodes, whether data is active or idle, or where else the data might be needed. Any orchestration is limited to having multiple instances of each file on different nodes.

Data Orchestration Across Multiple Tiers

The next level of use case adds orchestration to keep active data instances on Tier 0 and move idle data to less expensive tiers that are separate from the GPU nodes. The orchestration is managed by a storage service, with no regard for integration with the GPU/AI job management.

Job scheduler-Integrated Orchestration

The AI job scheduler knows which nodes will be assigned to a job. Integrating storage and data management APIs into the job setup workflow allows the scheduler to control data placement into the nodes that will run the job.

Advantages:

  • Higher probability data needed by the job is on a node running the job.

  • Remove vulnerability of instance or data loss due to reimage of nodes used for other jobs.

Data to Node Affinity

This level of Tier 0 consists of three elements:

  1. Orchestration to place data on job-specific nodes.

  2. When data is placed on multiple nodes, Anvil provides layout with the requesting client as the first entry in the layout.

  3. The ability for I/O to bypass the NFS and network stack when accessing a local instance of a file. This is referred to as localio in the Linux NFS client.

HCI and CPU-based Servers

While this document mostly discusses GPU-based workloads that are most urgently needed for AI use cases, the fact is that CPU-based servers often ship with the same ultra-fast local NVMe drives that can be activated as Tier 0. This is also true for low-power ARM CPUs.

These server types are often confused with Hyperconverged Infrastructure (HCI), which typically combine compute and storage in a single unit. HCI systems are predominantly used for structured data, VMs, and other block-oriented I/O patterns that are not suitable for CPU-intensive unstructured data workloads. And while HCI systems can use distributed storage architectures and other tiering solutions, their focus on block-oriented use cases make them unsuitable for processing unstructured data.

There are numerous CPU-intensive use cases that need large volumes of unstructured data, and that can take advantage of the same performance and cost improvements of Tier 0 as the GPU-computing workloads outlined above. Among them are other AI use cases, which include CPU-based AI inferencing, predictive analytics, or machine-learning pipelines dealing directly with unstructured datasets.

However, beyond AI use cases there are numerous other CPU-based application workflows that demand high performance and that can directly benefit from the ultra-low latency and enhanced throughput provided by activating Tier 0 on local NVMe drives within CPU servers.

Some of these use cases include:

  • Real-time analytics and data streaming: Rapid processing of log data, sensor inputs, or clickstream data to generate insights instantly.

  • High-performance search and indexing: Fast indexing and search across large-scale document repositories or datasets (e.g., Elasticsearch, Splunk).

  • Video and media processing: Low-latency editing, transcoding, rendering, and content delivery requiring quick, frequent data access.

  • Financial trading and modeling: Real-time risk analysis, algorithmic trading, and financial modeling needing immediate access to large datasets.

  • Genomics and bioinformatics: Rapid processing of genome sequencing data, imaging data, or bioinformatics workloads demanding near-instantaneous data retrieval.

The benefits for CPU-based Tier 0 include the same advantages highlighted in the GPU-based examples above:

  • The lower cost of activating ultra-high performance local NVMe drives within the servers vs. going over expensive networks to external arrays.

  • The significant boost in performance and lower latency that Tier 0 makes possible compared with external storage, which directly benefits these applications.