Introducing OpenSearch for Jira

What’s changing

We’re making preparations to introduce OpenSearch as an opt-in feature for our customers in a future Jira Data Center release. With OpenSearch, we’re aiming to externalize indexing processes to support large customers in improving performance for their search- and index-related functions in Jira DC. We’ve already successfully rolled out OpenSearch to Confluence.

As a first milestone, we’ll be introducing the Jira search API. This API will encapsulate Lucene’s usage so that we can swap the search platform to OpenSearch in a later release. Partners, you’ll need to move to the Jira search API to minimize disruption to customers who take advantage of the OpenSearch engine capabilities.

Starting from Jira 11, we will stop providing Lucene as a dependency, and we’ll remove all APIs exposing Lucene. We’ll announce any deprecations soon after the release of Jira 10.3.

Timeline

This project has only just begun, so we don’t yet have a timeline. We wanted to give you as much advance notice about this feature as possible, and we’ll share a guide to help you migrate and test your apps as soon as it’s ready.

We’ll keep you updated on progress.

5 Likes

I assume that this will affect the Lucene collector we use in our plugin. Will there a replacement for it?

1 Like

Please ensure that we have sufficient time to test out the new API and have some chanel opened to discuss limitations of the new API. Our app extensively use Lucene to index the data so that we offer powerful and diverse search functions. It’s not only about searching for specific text.

1 Like

Yes, we will provide a replacement for the Lucene Collector. We will share more details as they become available.

Thank you for the advance notice, much appreciated. ScriptRunner is an app that offers a number of custom JQL functions, some of which depend on custom Lucene indexing.

This change is potentially very disruptive, the search API changes in Confluence DC effectively required a rewrite of some of our features, I am expecting the same here.

ScriptRunner also allows customers to write custom JQL functions, which presents additional challenges: https://docs.adaptavist.com/sr4js/latest/features/jql-functions/custom-jql-functions

Anecdotally I know of some large DC customers who have hundreds of custom JQL functions, Atlassian should consider that these changes may not only impact app vendors, but also DC customers.

It is a little unclear how broad the deprecation scope is, does this merely include org.apache.lucene classes, or could we expect current Atlassian classes such as those in the com.atlassian.query package to be deprecated?

We would be happy to work directly with Atlassian teams implementing this change at the earliest opportunity, we can provide a list of all of our usage of the existing APIs. Hopefully we can ensure any replacement has minimal disruption to customers.

Cheers!

Reece

Should Jira 11 be expected 24 months from ~now? As is the typical timeline for major releases?

1 Like

No Elasticsearch or provide the customer the ability to choose between both?
But more important for now; please allow for setting pre-fixes so we can use one cluster to provide multiple instances of Jira. Or even better, make it possible to use one Opensearch cluster for Bitbucket, Confluence and Jira!

Please handle case sensitivity better that the current REST API …

We initially chose OpenSearch as ElasticSearch was no longer opensource. That has very recently changed but we’re still focusing on OpenSearch as the first deliverable.
Regarding the prefixes, we’ll follow the same approach as we’ve done for Confluence, which allows for multiple instances to use the same OpenSearch cluster.

@npn could you possibly share more details or an example of this?

Thank you for the quick response!

That’s why I ask about Elasticsearch, the licenses are changed :slight_smile: But they stayed in https://jira.atlassian.com/browse/CONFSERVER-96112 there won’t be any support :frowning:
Will look into the documenation for Confluence about the prefixes.

1 Like

Hi @rlander,

Thanks for the feedback and we’ll get in touch to discuss this further.
While the change is very disruptive, it is necessary in order for Jira DC to scale in line with customers growth.

Hi @yvesriel thanks for sharing your concern and suggestions regarding the changes with the introduction of the Jira Search API. We will ensure regular updates are provided on this and other relevant community channels. We are working with our Marketplace program team to make this change successful for everyone. Do visit this channel for further updates.

Are these changes expected to also impact/improve search via the REST API, or only via the SDK API?

Could you please provide a little more detail about the new Jira search API and what exactly it aims to replace.

Thank you

Hi @danut.manda the changes will mainly impact the SDK API. We expect the REST API to remain as it is for search.

We’ll provide more details as we get closer to Jira 10.4

Hi @mabo with the Jira Search API we aim to decouple all search related operations from the underlying search platform. The Jira Search API aims to replace all SDK APIs that expose Lucene, instead the the Jira Search API will use search platform agnostic types.

This will allow us to add support for OpenSearch as the underlying search platform.

As we get closer to Jira 10.4, we’ll publish documentation on how to migrate to the Search API.

Thank you @GabrielWeyer.