How to migrate from Lucene Document indexing to the new Search API indexer in Jira 11?

Jira Development Jira Data Center
, , ,
1

Hi everyone,

Previously, our Jira app used the Lucene Document API to index custom field values. With this approach, we could add multiple values for a single custom field by calling doc.add(new StringField(...)) several times per issue. This allowed us to support flexible JQL queries, since each value was indexed separately.

However, after upgrading to Jira 11, we are required to migrate to the new indexing system, which only allows indexing a single value for each custom field (especially for fields based on CalculatedCFType). This breaks our previous logic, as we can no longer index multiple values for a single field.

Additionally, we have noticed that the ~ (contains) operator does not work with custom fields based on CalculatedCFType, even when the field is indexed as text.

My questions are:

  • Is there any way to index multiple values for a custom field (like before) using the new Jira 11 indexing system?

  • Is it possible to enable the ~ operator for CalculatedCFType fields, or is this a limitation of this field type?

  • What are the recommended approaches or workarounds to replicate the flexible search and multi-valued indexing we had with the old Lucene Document method?

Thank you for your help!

2

Hi,

While I am not familiar enough to comment on the CalculatedCFType field type and its implication with indexing, I would strongly suggest you start by going through the migration documentation and do one field at a time in your indexing logic.

In general, instead of handling a document, you now have to populate the FieldValueCollector with one (or many) different fields(AnalyzedTextField for ~ and tokenized searches, KeywordField for = and text related searches, LongField for numeric searches, etc.)¹

I could get through most of the migration myself by going through the documentation and inspecting Atlassian’s indexing code for their system fields.

When all else failed, I had tremendous help from their dev team when I reached out to them in an ECOHELP ticket.

To your specific questions:

Is there any way to index multiple values for a custom field (like before) using the new Jira 11 indexing system?

You now need to declare your different fields in an override of your indexer’s getFields method so Jira knows what fields can or cannot be indexed (the schema is now forced).

You then need to populate your fields via the FieldValueCollector. Something like collector.add(this.dueDateLongField, dueDate);

Is it possible to enable the ~ operator for CalculatedCFType fields, or is this a limitation of this field type?

I personally do not know for the CalculatedCFType. I had to create a separate AnalyzedTextField in my case to enable ~ searches.

What are the recommended approaches or workarounds to replicate the flexible search and multi-valued indexing we had with the old Lucene Document method?

We have multi-valued indexing, and you can specify multiValued() when creating your different fields.

I hope this helps.

¹ You can use different field types for custom applications that I have not listed either, but I just wanted to give you a general idea.

1 Like
3

@PascalPerreault Thank you so much for your help — I really appreciate it! :folded_hands:

I’ve managed to resolve the issue on my end.
Turns out the solution was actually quite simple: I just needed to enable the multiValued option for the KeywordField. After turning that on, I was able to add multiple entries into the FieldValueCollector without having to change the custom field type.

Thanks again for pointing me in the right direction and for taking the time to help out!

1 Like