Summary
This article discusses an authentication issue on Kerberos-enabled clusters running on CentOS 7 (and related Linux distributions).
Applies to
- CentOS 7.x
- RedHat Enterprise Linux (RHEL) 7.x
- Oracle Enterprise Linux (OEL) 7.x
Symptom
Attempts to connect to a cluster with Kerberos authentication enabled running on CentOS 7.x fail with an authentication exception. For example, a failed cqlsh connection generates the following error in the system.log:
ERROR [IOThread-0] 2020-03-04 05:42:32,069 DseAuthenticator.java:778 - Error obtaining subject for server identity
javax.security.auth.login.LoginException: Message stream modified (41)
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:808)
at com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:618)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at com.datastax.bdp.transport.server.KerberosServerUtils.loginServer(KerberosServerUtils.java:36)
at com.datastax.bdp.cassandra.auth.DseAuthenticator$GSSAPISaslNegotiator.(DseAuthenticator.java:774)
at com.datastax.bdp.cassandra.auth.DseAuthenticator.getSaslNegotiatorForScheme(DseAuthenticator.java:682)
at com.datastax.bdp.cassandra.auth.DseAuthenticator.access$400(DseAuthenticator.java:100)
at com.datastax.bdp.cassandra.auth.DseAuthenticator$UnifiedSaslNegotiator.evaluateResponse(DseAuthenticator.java:513)
...
Caused by: sun.security.krb5.internal.KrbApErrException: Message stream modified (41)
at sun.security.krb5.KrbKdcRep.check(KrbKdcRep.java:101)
at sun.security.krb5.KrbAsRep.decrypt(KrbAsRep.java:159)
at sun.security.krb5.KrbAsRep.decryptUsingKeyTab(KrbAsRep.java:121)
at sun.security.krb5.KrbAsReqBuilder.resolve(KrbAsReqBuilder.java:308)
at sun.security.krb5.KrbAsReqBuilder.action(KrbAsReqBuilder.java:447)
at com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:780)
... 31 common frames omitted
Cause
The issue occurs due to an unknown problem with Kerberos on CentOS 7.x and related Linux distributions including RHEL 7.x and OEL 7.x. The problem was reported by other users and is being tracked as a CentOS bug #17000. Neither the root cause nor a fix in CentOS or RHEL was available at the time of writing.
Workaround
Users have reported that removing the renew_lifetime parameter from the Kerberos configuration file /etc/krb5.conf allows applications to authenticate successfully. For example, remove the line:
renew_lifetime = 7d
from the [libdefaults] section:
[libdefaults] default_realm = TEST.COM dns_lookup_realm = false dns_lookup_kdc = false ticket_lifetime = 24h renew_lifetime = 7d forwardable = true
Solution
This is not a DataStax Enterprise issue. Please contact your Linux operating system vendor to obtain a fixed version of your Linux distribution.
See also
Dev Community - https://community.datastax.com/questions/2846/