Example 1: Two Tenants, Shared Resources

This example will configure a single Hammerspace instance for two tenants, in this case two engineering departments at a university. For simplicity and ease of administration by the university’s central IT organization, all resources are shared. Servers are bare metal.

Hammerspace Architecture

A pair of Anvil nodes are deployed for redundancy, with a lightweight quorum service running elsewhere in the customer infrastructure. The quorum service provides extra protection against 'split brain' if a network partition occurs and is a best practice when using two Anvils.

Three DSX nodes are used, each with the S3 service enabled. This is the minimum quantity of DSX nodes recommended as it enables one to be taken out of service for maintenance while still retaining protection against a DSX failure. In this installation, the DSXs host S3 services and also act as data movers for data orchestration. If these three are not sufficient to handle the load, it is a simple matter to deploy additional DSXs.

s3mt example 1 hammerspace architecture image1

Figure 1. Example 1 architecture: two tenants sharing all resources

Back-end storage consists of HDD-based JBODs attached to each DSX. The university is also considering repurposing some older NAS systems to provide additional capacity. These may be easily added later. For offsite DR protection, a copy of each object is maintained in inexpensive public cloud storage.

Configuration Steps

The process to set up this example configuration consists of ten steps.

Create Floating IPs: On the Administration page, Network tab, click Add Floating IP to add a floating IP.

Figure 2. The Add Floating IP button on the Network tab

Fill in the IP and subnet mask, and select Portal for the Type, and click Add.

Figure 3. Adding a floating IP with Portal as the type

Follow the same process to add two more floating IPs. Note that one is assigned to each DSX. If a DSX goes down for any reason, its floating IPs will be reassigned to a surviving DSX node.

Figure 4. One floating IP assigned to each DSX node
Enable S3 Services on all DSXs: Still on the Administration page, switch to the Services tab and enable the S3 service on all three DSXs.

Figure 5. The S3 service enabled on all three DSX nodes
Create Users: If using Active Directory, go to the Active Directory tab to configure Hammerspace to connect to AD. Full instructions are contained in the Hammerspace Administration Guide. The instructions below are for creating local users.

Still on the Administration page, switch to the Users tab and use the Create Users button to add at least one user for each tenant. Select the S3 Data Access Role.

Figure 6. Creating a local user with the S3 Data Access role
Create a Volume Group: The local DSX disk volumes will have already been added to Hammerspace as storage volumes as part of the DSX installation process. Since all storage volumes will be shared among all tenants, configuring a volume group containing all of these storage volumes will simplify other configuration.

On the Infrastructure page, Volume Groups tab, click Create Volume Group. Fill in the details in Step 1, then skip to Step 4.

Figure 7. Volume group details in Step 1 of the wizard

In Step 4, select all of the DSX storage volumes and click Next. On Step 5, click the button to confirm and create the Volume Group.

Figure 8. Selecting all DSX storage volumes in Step 4

Back on the Infrastructure page, you will see the new Volume Group in the list. If more DSX storage volumes for S3 use are added in the future, adding them to this Volume Group is all that needs to be done.

Figure 9. The new volume group in the Volume Groups list
Create a Cloud Volume: This example uses an AWS cloud bucket to store DR copies of objects. Before that can happen, the AWS bucket needs to be configured into Hammerspace as a storage volume.

On the Infrastructure page, Storage Systems tab, click Add Storage System.

Figure 10. The Add Storage System button on the Storage Systems tab

Fill in the details, selecting Object Storage and Amazon S3 for the storage type, using the correct Access Key / Secret Key credential pair and selecting the correct region.

Figure 11. Adding an Amazon S3 storage system

Click Add Storage System to continue.

The following dialog will appear once the storage system is successfully configured. Note at this point we have simply connected to the AWS account with the supplied credentials. No bucket has been selected yet. Click Add Volumes to continue.

Figure 12. The storage system is configured; no bucket selected yet

Click the checkbox next to the appropriate bucket to select it. Bucket names appear in the 'Name' column following the storage system name. In this example, the bucket name is "ivydr."

Figure 13. Selecting the ivydr bucket

In the next step, un-check the Shared Volume checkbox, as this bucket will not be used for sharing data between multiple sites. Then click Next Step.

Figure 14. Un-checking the Shared Volume checkbox

In step 3, leave the Capacity limit at 'Auto' to ensure all objects are protected, regardless of how large the capacity grows, or place a limit on the amount of data Hammerspace can store in the bucket. Enable Compression to minimize cloud storage charges. Objects stored in cloud buckets are automatically deduplicated. Click Next Step to proceed.

Figure 15. Capacity and compression settings for the cloud volume

On steps 4 and 5, simply click Next Step to proceed, leaving all values at their defaults. On step 6, click Add Volume to add the bucket as a storage volume.

Figure 16. Confirming the new cloud volume

The new volume will appear in the list of volumes on the Volume tab of the Infrastructure page.

Figure 17. The cloud volume in the volumes list
Create a Share: Hammerspace S3 servers require a target share, so before creating any S3 servers, one or more shares must be configured. On the Data page, Shares tab, click Create Share to begin the process.

Give the share a name and description, and if desired set a limit on the maximum size of the share (a share quota).

Figure 18. Naming the share and setting an optional share quota

If desired, set up a snapshot schedule for data protection using the Snapshot Schedules tab. The settings on the other tabs may be left at their defaults. Objectives will be customized after the share is created. Click Create to create the share. After a few moments it will appear in the shares list.

Figure 19. The S3_Root share in the shares list
Configure Share Objectives: Shares, like S3 servers, are logical entities. Objectives are used to control the storage volumes on which data written to the share will be stored. In this case, S3 servers are writing into the share, so the objectives determine which storage volumes will house incoming S3 objects from each S3 server.

To edit the objectives for the S3 share created in the previous step, click the pencil icon in the Applied Objectives column.

Figure 20. The pencil icon in the Applied Objectives column

On the Applicability: TRUE row of the table, click the pencil icon to edit those objectives.

Figure 21. The pencil icon on the Applicability: TRUE row

In the Select Objectives section, un-check optimize-for-capacity to remove that objective. In this configuration, all objects will be written to the AWS cloud as a backup vs. using the cloud as a capacity tier.

Figure 22. Removing the optimize-for-capacity objective

Scroll down and check the boxes next to the place-on-DSX volumes and place-on-object-volumes objectives. Note that place-on-object-volumes is not the same as place-on-shared-object-volumes! The two selected objectives ensure that one instance of each object will be placed on local DSX storage, and one will be placed in the cloud for DR.

Figure 23. Selecting the place-on objectives

Click Apply to apply the changes. The Applicability: TRUE row in the table should now look like this:

Figure 24. The Applicability: TRUE row after applying the objectives

Click Close to finalize the changes.
Configure S3 Servers: Now that the groundwork has been laid, the two S3 servers can be created. On the Administration page, S3 Servers tab, click Create S3 Server.

Figure 25. The Create S3 Server button on the S3 Servers tab

Give the S3 server an administrative name of MechE Dept S3. Un-check the Default Server checkbox, and ignore the warning. If a client attempts to connect to this server without specifying an endpoint, or using a non-existent endpoint, we want the attempt to be rejected. Specify an endpoint name of s3.meche.ivy.edu. This is how S3 clients will connect in our example. For the Identity Provider, select Local.

Figure 26. Creating the MechE Dept S3 server

Scroll down and set the remaining options as desired. The tooltips that appear when hovering over the blue information icons explain what they do. The most important is the Umask.

Figure 27. The remaining S3 server options

The Umask determines the access permissions for objects when Local users are used, not AD. The default setting (0002) is equivalent to "Group." For a complete explanation of how Umask works, see the Appendix. For this use case the default is appropriate.

Figure 28. The Umask setting

Click Next to continue to the Bucket Container and Buckets step.
Create the Bucket Container: In this example we need S3 clients to be able to create buckets using S3 API commands. This requires a bucket container to be configured, which is the location where created buckets will live. Click Add Bucket Container to begin.

On the next screen, fill in the Bucket container name Mech E Dept Buckets, choose the share previously created (S3_Root), and specify the bucket container path as /MechE. The directory /S3_Root/MechE will be created and represent the bucket container in the filesystem. When a bucket is created using S3 API commands, a corresponding subdirectory will be created in /S3_Root/MechE. For example, creation of a bucket named "Bucket1" will result in the directory /S3_Root/MechE/Bucket1 being created. Objects stored in Bucket1 will become files in the Bucket1 directory.

The rest of the options can remain at their defaults. Note however that the S3 server-level default S3 access permissions could be overridden here, and the change would apply to all buckets created in this bucket container. The ability for clients to create and delete S3 buckets may also be enabled or disabled here.

Figure 29. Adding the bucket container

Click Add to add the bucket container, which will be displayed in the table.

Figure 30. The bucket container displayed in the table

Click Next to continue to the next step, adding access keys.
Assign S3 Credentials: Users were created previously, in step 3. These users need access keys and secret keys to be able to authenticate to their S3 server. The assignment of these credentials happens in this step. Begin by clicking Add Access Key.

Figure 31. The Add Access Key button

On the next screen, choose the user 'meche' from the dropdown, and either leave the Access Key and Secret Key fields blank to have them automatically generated, or manually enter values.

Click Add to add the credentials, which will return you to the Access Keys step.

Figure 32. Adding an access key for the meche user

To finish creating the S3 server, including the bucket container and access keys, click Create. You can always edit the server later to add more buckets, bucket containers, or access keys, or to change the access permissions or other options.

Repeat steps 8, 9, and 10 to create another S3 server for the Chemical Engineering Dept.

Figure 33. The two S3 servers, one per tenant

That completes the configuration for Example 1. Users 'meche' and 'cheme' can now use their S3 access key / secret key pairs and any S3 client to create buckets and store and retrieve objects from the Hammerspace S3 servers. Hammerspace will automatically create a secondary instance of every object in AWS for DR protection.