Introducing OpenSearch for Jira

I see 10.4 is out but the search upgrade guide is 404 at the moment

1 Like

Hi Reece. Apologies — it looks like the page didn’t get unlocked correctly during go-live. I’ve now made the guide available for all to view: Search API upgrade guide | Administering Jira applications Data Center 10.4 | Atlassian Documentation

Thanks for the documentation, the part about the org.apache.lucene.search.Collector is a bit concerning.
The main reason to use the Collector was the high performance when you need to collect only a small part of the indexed documents (e.g. only issue ids or keys). Would that be possible using the suggested aggregation?
I don’t think that there are more than a few plugin vendors out there that had anyting to do with OpenSearch and it would be helpfull if you could at least provide a few examples for the migration

1 Like

Thanks for reviewing the docs @m.herrmann.
The purpose of the Aggregation will only be for usages where you’re using the Collector to aggregate field values.
If you only need specific fields, we suggest using the com.atlassian.jira.search.index.IndexSearcher#scan(SearchRequest request, Function<Document, Boolean> callback) function.
We’ll update the docs to include examples as requested.

3 Likes

We tested our Jira app with Jira 10.4’s new Search API and found a discrepancy in how FieldIndexers are used in com.atlassian.jira.issue.customfields.searchers.information.CustomFieldSearcherInformation:

Situation

  • FieldIndexer
    • old and deprecated: com.atlassian.jira.issue.index.indexers.FieldIndexer
    • new: com.atlassian.jira.search.issue.index.indexers.FieldIndexer
  • CustomFieldSearcherInformation:
    • it now uses a Builder to construct it → is has 2 methods for FieldIndexers
      • relatedIndexers(...) that expects a list of old FieldIndexers
      • indexers(...) that expects a list of new FieldIndexers

Problem
When constructing a CustomFieldSearcherInformation with its Builders build() method, it requires both indexers(...) and relatedIndexers(...) having been called with non-empty Lists. Or in other words: It requires both old/deprecated FieldIndexers and new FieldIndexers. Otherwise the searcher cannot be set to customfields and the following error is logged:

com.atlassian.jira.util.dbc.Assertions$EmptyArgumentException: relatedIndexers should not be empty!
	at com.atlassian.jira.util.dbc.Assertions.notEmpty(Assertions.java:50)
	at com.atlassian.jira.issue.customfields.searchers.information.CustomFieldSearcherInformation.<init>(CustomFieldSearcherInformation.java:40)
	at com.atlassian.jira.issue.customfields.searchers.information.CustomFieldSearcherInformation$Builder.build(CustomFieldSearcherInformation.java:124)
	at de.communardo.jira.plugins.userprofile.customfield.searcher.MultipleSelectSearcher.init(MultipleSelectSearcher.java:74)
	at de.communardo.jira.plugins.userprofile.customfield.searcher.MultipleSelectSearcher.init(MultipleSelectSearcher.java:36)
	at com.atlassian.jira.issue.fields.ImmutableCustomField.loadCustomFieldSearcher(ImmutableCustomField.java:1888)
    ...

Questions
Is that intended? Do we as an app really have to maintain both types of FieldIndexers, one of which is marked as deprecated via this very Search API change? That seems like an oversight.
If we really have to provide both FieldIndexers, then how does Jira decide which one to use? Is it actually using the new ones, or is it still hard-wired to use the old/deprecated ones?

Hey @AndreasEbert this was an oversight. We detected it too late to be able to fix it in 10.4 There will be a fix for it in 10.4.1.

With 10.4.1, you’ll be able to provide only the search API indexer. We don’t expect you to implement both.

1 Like

@GabrielWeyer, ah, perfect. Good to hear. We’ll skip 10.4.0 then, and have another look with 10.4.1 :+1:

Another question, concerning backwards compatibility:
Seeing that the new Search API classes don’t exist prior to Jira 10.4, we can’t use them in app-versions that also support older Jira-versions (at least not without using Java Reflection). For some similar cases in the past, Atlassian provided a compatibility-helper library (or whatever such things are called), that apps could use to achieve backwards compatibility. Any plans for providing such a library here in this case?
Otherwise, we’ll probably wait till Jira 11+ and release a non-backwards compatible app-version.

Hi @AndreasEbert, we don’t have any plans to provide a helper library. If maintaining two versions is costly, I’d suggest the “wait till Jira 11” approach.
The primary purpose of the Search API is to decouple Search and Indexing from Lucene, allowing Jira to switch over to OpenSearch in the future. We understand there will be a significant amount of rework for many vendors and customers which is why we’re releasing it incrementally before Jira 11.
OpenSearch will only be generally available in Jira 11+, so I think waiting until Jira 11 is fine.

1 Like

The IssueIndexManager previously provided methods to hold and release indexing operations. However, in Jira Data Center 10.4+, neither com.atlassian.jira.search.index.IndexAccessor nor com.atlassian.jira.issue.index.IssueIndexer offer equivalent functionality to pause or resume indexing. As a result, there is currently no direct API support for programmatically holding or releasing indexing in the latest Jira version.

Would you like to include possible alternatives or workarounds?

Hey @denispasha, could you please describe your use case for calling hold and release? We’re currently looking at how we’ll replace internal calls to hold and release and would benefit from additional context.

1 Like

Hi @GabrielWeyer,
We are importing a large number of issues, and during this process, we want to temporarily hold reindexing to avoid unnecessary performance overhead and potential indexing inconsistencies. Once the import is complete, we would then release indexing to ensure all imported issues are indexed correctly in one go. This helps prevent partial or inconsistent search results during the import and improves efficiency by reducing the number of reindexing operations.

Previously, we achieved this using hold() and release(), but with the deprecation of these methods, we are looking for an alternative approach to maintain indexing control during such operations.

2 Likes

Thanks for explaining your use case @denispasha. We agree that this is a nice feature for optimising performance regardless of whether indexing is done by Lucene or OpenSearch.

From Jira 10.5, hold() and release() functionality will be available on the com.atlassian.jira.issue.index.IssueIndexingService, which is part of the public API and will be supported through Jira 11+.