Search the docs

Troubleshooting

A user receives a permission-denied error on an NFSv4.2 mount.

The user’s identity may not be resolving. Look up the user with User/Group Lookup or --resolve-user and check that the UID and GID are what you expect. Also check that the name service’s status is UP in the Directory Services table. A status of UP shows only that the last connection test passed; the lookup is the real check.

User/Group Lookup returns "not found" for a known user.

Possible causes:

  • The user is outside the configured search base. Check the Search Base.

  • The user entry does not have uidNumber and gidNumber, or the user name is not in the uid attribute.

  • The schema setting (RFC 2307 or RFC 2307bis) does not match the directory.

  • The name service uses LDAPS or StartTLS and the server certificate is not trusted by the Anvils. The connection test and the UP status do not detect this, and users that were looked up before the problem started may still resolve from the SSSD cache. See Securing LDAP Connections with LDAPS or StartTLS.

  • The LDAP server is unreachable from the Anvils. Test the connection; see Testing the Connection to a Name Service.

  • The change was made in the directory recently and SSSD still has the old entry cached. Wait for the cached entry to expire. The Clear Cache button on the Directory Services page clears the Active Directory cache only, not the LDAP cache.

A qualified lookup fails with "Domain '<domain>' specified by user name '<name>' could not be found".

The domain in the qualified name (user@domain) does not match the domain name of any configured name service. Check the domain names in the Directory Services table. Domain names are matched without regard to case.

Adding a name service fails with "Failed to connect to LDAP server".

Every combination of address and transport mode that was tried failed. The message has one line per combination, naming the domain, the address, and the transport mode (or "any transport mode" and "and with all common LDAP ports" when those were probed); it does not give the reason. Check the following causes, testing from an NFS client or another host on the Anvils' network with standard LDAP tools where you can:

  • The server is unreachable on the port, or the transport mode is not enabled on the server.

  • No naming context on the server matches the domain name. The domain corp.example.com requires the naming context dc=corp,dc=example,dc=com in the server’s root DSE, even if you set a narrower search base. The GUI’s Domain Name tooltip example (ldap.test.com) does not change this rule.

  • The search base does not exist, or the Bind DN cannot read it.

  • The Bind DN or Bind Secret is wrong.

  • For LDAPS or StartTLS, the certificate does not list the configured address in its Subject Alternative Name. See Securing LDAP Connections with LDAPS or StartTLS.

When some attempts fail but one succeeds, the name service is added, and the failed attempts are listed with their reasons (in the GUI, click Show Details; in the Admin CLI, under Connection attempts:).

Adding or updating a name service with more than one address fails.

A name service with more than one address can be saved only when the connection test is skipped. In the Admin CLI the failure is reported only as name-service-config: unexpected error. See Limitations.

Updating a name service fails with "If one of 'bindDn' or 'bindSecret' is specified, both must be specified".

The name service uses a Bind DN, and the update did not include the Bind Secret. In 5.3, include the Bind Secret with every update of such a name service (in the GUI, enter it again in Bind Secret; in the Admin CLI, add --bind-secret), unless the update removes the credentials with --bind-clear. Giving --bind-dn without --bind-secret fails with the same message.

Test Connection shows the port with a comma, for example 10.0.0.15:1,389.

The GUI formats port numbers of 1000 and above with a thousands separator. The port in use is 1389; no action is needed.

A name service shows status DOWN.

The last connection test to the name service failed. Check:

  • Network connectivity from the Anvils to the LDAP server.

  • That the LDAP service is running on the server.

  • That the saved address, port, and transport mode are still correct.

Hammerspace retests a DOWN name service every 10 minutes and changes it back to UP when the test passes. It tests UP name services only once every 24 hours, so a failure can take up to a day to show as DOWN. To test immediately, see Testing the Connection to a Name Service.

A NAME_SERVICE_UNHEALTHY event appears.

The event reads Name Service '<name>' (domain '<domain>') is unhealthy and corresponds to a name service with status DOWN; see the previous entry. It clears automatically when the name service passes a connection test.

An SSSD_CONFIG_SYNC_FAILED event appears.

The event reads Failed to synchronize SSSD configuration to node '<Anvil>'. Hammerspace could not apply the SSSD configuration on that Anvil, or could not start SSSD with it. Check that the Anvil is online and reachable from the other Anvil. Hammerspace retries every 2 minutes and clears the event when it succeeds. While the event is active, LDAP lookups can fail; see Limitations.

/etc/sssd/sssd.conf is missing on an Anvil.

This is normal when no LDAP name service is configured and Active Directory is not joined with an ID-mapping schema. See How Hammerspace Manages SSSD on the Anvils.

Joining Active Directory fails with ad-join: Cannot join Active Directory while an LDAP Name Service is configured.

LDAP name services and Active Directory are mutually exclusive in Hammerspace 5.3. Remove all LDAP name services before joining Active Directory. In the GUI and the API, the same message appears without the ad-join: prefix.

Adding a name service fails with "Cannot configure an LDAP Name Service when the system is joined to an Active Directory domain".

The cluster is joined to Active Directory. In 5.3, leave Active Directory before adding an LDAP name service.

Error messages appear when adding a name service with the Admin CLI.

When the connection test tries transport modes and ports to find one that works, it reports each combination that fails. This is expected as long as the output ends with a successful attempt. If no attempt succeeds, the name service is not added; check the address, port, and transport mode.