Migrating apps to Jira 8: be careful when using JQL from background threads

migration
index
jira-80
jira-server

#1

Hi everyone!

I would like to share ideas about a problem we had faced when we were making our app, Structure, compatible with Jira 8. I think this can be helpful to other app developers.

As you probably know, Jira 8 comes with the new Lucene engine. The binary Lucene files are not compatible between Lucene used in Jira 7 and Jira 8. As a result, Jira 8 will automatically remove the old index files and start full reindexing on Jira start. (See this article: Preparing for Jira 8.0)

That means Lucene index is going to be absent/empty for some time after Jira 8 (and all apps) have started. Which, in turn, means that any JQL will return an empty result.

This doesn’t affect most of Jira and app functionality where code is executed in response to a user’s request, because Jira is unavailable for interaction during complete reindexing. But, in our product there are also background jobs (synchronizers) that start with the application and that potentially can make changes in Jira database based on the results of a user-configured JQL query.

Executing such background tasks with an invalid index is extremely undesirable and may be harmful to Jira state, stored in the database.

To resolve this problem, we needed a mechanism for detecting absent or corrupt index state. We would then postpone executing the background jobs until the index is back to normal.

Thanks to the Atlassian team, we learned about method IndexLifecycleManager.isIndexConsistent() in the Jira API. Under the hood, this method counts issues in the database and compares it with count of issues in the index. So we can check this method and delay background work while index is still not valid.

Calling this method is not cheap, but it’s not super-expensive either. We used a guideline to cache a negative result (index inconsistent) for 5 seconds and a positive result (index consistent) for a minute or more.

If your app has background tasks that rely on JQL and make any kind of mutations based on the JQL result, we strongly advise you to review your code and implement a similar solution.

Here is example of a small component that we’ve implemented. Feel free to grab this code if you need it.

Click to expand a component code

public class IndexConsistencyChecker {
  private static final long SUCCESS_EXPIRATION = TimeUnit.MINUTES.toNanos(2);
  private static final long FAILED_EXPIRATION = TimeUnit.SECONDS.toNanos(5);

  // implementation of IndexLifecycleManager
  private final IssueIndexManager indexManager; 
  // last result of consistency check
  private volatile ConsistencyState currentState; 

  public IndexConsistencyChecker(IssueIndexManager indexManager) {
    this.indexManager = indexManager;
  }

  public boolean isConsistent() {
    if (!indexManager.isIndexAvailable()) {
      return false;
    }
    long now = System.nanoTime();
    ConsistencyState state = currentState;
    if (state != null && !state.isExpired(now)) {
      return state.consistency;
    }
    synchronized (this) {
      ConsistencyState state1 = currentState;
      if (state1 == state) {
        boolean isConsistent = indexManager.isIndexConsistent();
        currentState = state1 = new ConsistencyState(isConsistent, now);
      }
      return state1.consistency;
    }
  }

  private static class ConsistencyState {
    final boolean consistency;
    final long timestamp;

    ConsistencyState(boolean consistency, long timestamp) {
      this.consistency = consistency;
      this.timestamp = timestamp;
    }

    boolean isExpired(long now) {
      long expiration = consistency ? SUCCESS_EXPIRATION : FAILED_EXPIRATION;
      return now - timestamp > expiration;
    }
  }
}

Thanks again to Jira and Premier Support teams, Andriy Yakovlev, Łukasz Wlodarczyk, Syed Masood, Adam Jakubowski and others for helping with this!

Best regards,
Egor Lyashenko
ALM Works


#2

Thanks for sharing !


#3

Thank you @egor.lyashenko!


#4

Thank you, @egor.lyashenko!


#5

Very useful I may well need this in my app at some point