Upcoming change to Confluence search (AKA “Hello, OpenSearch!”)

[WendyR]

Hi everyone,

We have started researching and working on replacing our current Confluence search engine, Lucene, with OpenSearch. Although the change is some time away, we’re providing this early announcement to help you prepare for any likely breaking changes.

If you’re using the Lucene API independently, consider moving to the Confluence v2 search API to minimize disruption. We will maintain compatibility with our v2 search API as much as possible, so most usages should not cause any issues.

When our research is complete and we are aware of all breaking changes, we will provide an upgrade guide with these details.

[david]

Just to say I can’t wait for a time when “I am so excited about this” is no longer interpreted as “excited about this”.

Don’t have stop words removed from exact matches in your opensearch implementation.

CONFSERVER-14910/CONFCLOUD-69222

[rlander]

I’m assuming this is only really being considered for Confluence, and not Jira?

Bitbucket already supports OpenSearch, to my knowledge.

[jens]

Could you please provide a bit of extra context what this means for apps that are already compatible with Confluence 8?
As documented here, Atlassian has already removed app access to Lucene APIs in Confluence 8.0, so I was wondering if the announced changes entail any additional limitations.

[WendyR]

Thanks Jens. I’ve shared your question with the team and they’ll come back to you shortly.

[WendyR]

Hi rlander. That’s my understanding, but I have shared your message with our dev team so they can clarify.

[HendryLuk]

Thank you for your inquiry Jens. If your apps are exclusively utilising the Confluence 8.0 API for index access, it is highly likely that they will be compatible with OpenSearch. We aim to maintain API compatibility between our Lucene and OpenSearch implementations.

It is essential to be aware, however, that there are subtle differences between Lucene 4.4 and OpenSearch 2.x, which could potentially affect specific behaviors and performance aspects of your apps. Additionally, there may be minor components of the API that require some breaking-changes in order to work nicely with OpenSearch. We will provide more details in the upcoming months as we get closer to delivering this feature.

Furthermore, we are planning to offer early-access opportunities before the general release, allowing you to test your apps and provide us with valuable feedback.

[HendryLuk]

Hi rlander, OpenSearch is also currently being considered for Jira, but it’s not part of our roadmap at this stage.

[HendryLuk]

Hi David. Search behaviors will largely remain unchanged between Lucene and OpenSearch, since those behaviors are written on top of a set of API that’s agnostic to the underlying platform (i.e. Lucene vs OpenSearch).

Regarding CONFSERVER-14910, the fix is currently available as a dark feature and will eventually be released to the broader public at a later point. This will remain true independently of the transition from Lucene to OpenSearch.

[tied]

replacing our current Confluence search engine, Lucene, with OpenSearch

It’s funny in a way, that OpenSearch is based on Lucene as well

Looking from an operational / deployment / immutable infrastructure / cost point-of-view…

• The Lucene index stored in Confluence home is a local state, which should be avoided in regards to immutable infrastructure
• Resource wise, it just cost storage
• OpenSearch / ElasticSearch needs more resources (CPU / RAM) so it will increase costs
• It would required separate instances (looking at Bitbucket DC it mentions remote ElasticSearch / OpenSearch instance)
• Creates additional operational dependencies (network, security, backup & restore, etc.)

So to say, the research should include those topics for consideration as well

[Sake]

Please make it possible to use one Opensearch cluster for all the applications (Bitbucket, Confluence and maybe Jira). This reduces overhead to maintain a cluster for each application!