16.04.2024 Problem:
The client sought consultation on the best way establishing the standard Cassandra topology for a production-ready environment.
Process:
Cassandra Configurations:
- 2 Data Centers (PROD and DR).
- Each DC has 3 nodes of Cassandra.
- 2 DC are part of one cluster, thereby forming a ring architecture of 6 nodes (3 PROD + 3 DR).
- RF = 3
- ConsistencyLevel = ONE
- Volume expected for next year is 2-3 million data
Solution:
Upon review, the architecture was generally sound, with one notable exception: the replication factor (RF) set to 3. Clarification revealed that RF=3 implied one main copy and three replicas, totaling four copies of live data across four nodes. The suggestion was to reduce RF to 1 for smaller clusters, ensuring one main copy and one replica, optimizing space, and improving write and read performance. The client agreed to adjust the RF from 3 to 1, ensuring two copies of data per data center. The VMs were appropriately configured to align with the recommended specifications for optimal performance.
Additional recommendations included:
- Memory & CPU: Optimal configuration suggested not exceeding 64GB RAM per node, dedicating a maximum of 30GB for heap, which aligned with their VM specifications.
- Data: Advised keeping data per node under 1.5TB and suggested running nodetool repair periodically for smooth operation.
Conclusion:
In conclusion, the adjustments made to the Cassandra topology, including setting the replication factor (RF) to 1, effectively resolved potential issues identified in the initial setup. This modification not only ensured data consistency but also optimized performance, aligning with recommended best practices for smaller clusters in Cassandra environments. Additionally, the client has committed to conducting periodic nodetool repairs to uphold system integrity over time.
With these modifications and commitments in place, the client’s Apache Cassandra environment is now well-prepared for a reliable and efficient deployment in their production environment.