Problem:
Cassandra v4.0.6 in non-production environments experienced an issue where the “cassandra” superuser password, which had been changed two months prior, reset to its default password (“cassandra”) after applying OS patches and rebooting the server. No manual password changes were evident in the audit logs.
Solution:
- Initial Steps:
- The superuser “cassandra” initially had the default password.
- The password was changed using the “ALTER USER” statement.
- OS patches, including kernel-related packages, were applied, necessitating a VM reboot.
- Post-reboot, the password reset to its default value.
- Nodetool_status.log, cassandra.log, system.log, debug.log were collected.
- Troubleshooting:
- A detailed review of the system logs and upgrade scripts was suggested to identify any post-upgrade scripts that might reset the password.
- The expert recommended changing the password again using the command
ALTER USER cassandra WITH PASSWORD 'NEW_PASSWORD';
and then restarting the server to observe if the issue persisted.
- Client Actions and Results:
- The client followed the expert’s instructions by changing the password again and restarting the server.
- After the reboot, the password did not reset to the default value, indicating the issue did not occur this time.
- The client was recommended to monitor the situation and planned to patch the Cassandra VMs in the following month to see if the issue resurfaced.
Conclusion:
By reapplying the password change and restarting the server, the issue of the Cassandra superuser password resetting to the default was temporarily resolved. The client did not experience the same issue during subsequent tests.