Resolving Elastic Crashes due to StackOverflowError Involving “GraphTokenStreamFiniteStrings”

Problem:

The client’s Elastic server is experiencing sporadic crashes, evident from StackOverflowError logs linked to “GraphTokenStreamFiniteStrings.” While a potential fix exists in Lucene 9.7, the product’s certification with Elastic 7.17 using Lucene 8.11.1 complicates the implementation of the fix. The client seeks assistance to evaluate the possibility of backporting Lucene 9.7 changes to Lucene 8.11.1.

Process:

After thorough review and consideration, it’s acknowledged that modifying ElasticSearch core functionality, particularly Lucene, poses significant challenges and risks. Preparing a backport fix for Lucene 8.11.1 would necessitate the involvement of senior Java developers with expertise in ElasticSearch and Lucene. The complexity of this process would entail a considerable estimated time of arrival (ETA) and cost calculation. Additionally, Lucene’s pivotal role in ElasticSearch and its history of breaking changes across major releases necessitate careful analysis.

Solution:

• Option A: Focus on backporting fixes from Lucene 9.7 into Lucene 8.x and building a custom release of Elasticsearch 7.x.
• Option B: Evaluate building a custom release of Elasticsearch 7.x with Lucene 9.7.
• Option C: Disregard analyzing problematic queries in the Core Product due to irrelevance to the use-case.

Steps:

1. Analyze the disparities between Lucene 8.x & 9.x relevant for Elasticsearch.
2. Backport changes from Lucene 9.7 to Lucene 8.x and develop a proof of concept (PoC).
3. Conduct thorough quality assurance testing to ensure stability and compatibility.
4. Prepare technical and user documentation for deployment.
5. Plan and execute production deployment while considering system specifications and requirements.

Required Info:

• Host operating system details.
• Potential impediments related to Java version upgrades.
• Details about the current Elasticsearch upgrade process for automation considerations.

Conclusion:

While Options A and B offer potential solutions, both entail significant complexities and risks. Option A is prioritized due to its focus on backporting fixes, aligning with the client’s request. Collaborative efforts between IBM and the client are imperative to identify the most suitable approach and ensure a seamless deployment aligned with specific needs and constraints.