Jira search API throws internal server error for searching entity property

We have an app that sets and utilises salesforceAssociatedIds entity property in Jira issues. Using search issue API throws a 500 Internal Server Error when Jira has more than 700k issues and salesforceAssociatedIds entity property is being used in the query. For example using this JQL:

salesforceAssociatedIds is not empty

I noticed Jira takes much longer to perform the search when third-party entity property is being used in the JQL. For example, text ~ something results in 4-5 seconds, while salesforceAssociatedIds is not empty takes at least 30 seconds in 700k issues.

Is there any limit on searching using entity property? or any suggestion to make this work?

1 Like

Hi. Thank you for bringing the question! We are aware and have it on long term roadmap. I will bump up the priority for this one. There is a 30s time limit so I can suggest trying narrowing the resultset by adding the project name for example.

4 Likes

Thanks @BartoszCzerwonogrodz for quick reply!
It makes sense to have a 30s time limit, but having an entity property in the JQL makes the search very slow and easy to exceed the time limit. As I mentioned in the description, a search that normally takes 4-5s, takes 30s when the JQL has entity property. Is that an expected behaviour? Is there any way or any plan to improve it?

Unfortunately, we are aware of the scalability problems with the entity property queries. On the instances with a high volume of entity properties stored we may observe slow performance for JQL queries using them. Regrettably, the fix is far from trivial, we have plans to improve in the future, this item is high on our priority list.

Have you tried narrowing the JQL query to a more specific result set (e.g. with a project in () clause)? It may bring gains in some use-cases.

Consider also contacting us via Support Case, so we can investigate closer your particular case, perhaps we would be able to offer a better, more suitable solution.

1 Like

Hi @BartoszCzerwonogrodz,

I ran into this problem after releasing a new version of our app. During the troubleshooting session, we were told that there are approximately 3 to 4,000,000 issues in the client’s Jira.

I must admit that the query we are building is complex, however I didn’t expect it may don’t work at all. On that particular instance we are getting 100% rate of failed responses: status 500, message: “Internal Server Error”, (we saw 30s timeouts during the zoom session)

I reached Atlassian support (DEVHELP-6883) thinking that there must be some kind of problem with the client instance, however I am also observing failed requests on other instances, luckily the failure rate is much lower.
Example of an issue property:

{
  "templateName": "Task in A",
  "scopeId": null,
  "copySubtasks": true,
  "isTemplateSelectable": true,
  "isTemplateAvailableInAllProjects": false,
  "projectsIdsThatTemplateAvailableIn": [
    11122
  ],
  "enabled": true
}

Configuration in atlassian-connect.json for extractions:

"keyConfigurations": [
          {
            "extractions": [
              {
                "type": "string",
                "objectName": "enabled",
                "alias": "issueTemplateEnabled"
              },
              {
                "type": "string",
                "objectName": "scopeId",
                "alias": "issueTemplateScopeId"
              },
              {
                "type": "text",
                "objectName": "templateName",
                "alias": "issueTemplateName"
              },
              {
                "type": "string",
                "objectName": "isTemplateSelectable",
                "alias": "issueTemplatePrimary"
              },
              {
                "type": "string",
                "objectName": "isTemplateAvailableInAllProjects"
              },
              {
                "type": "string",
                "objectName": "projectsIdsThatTemplateAvailableIn"
              }
            ],
            "propertyKey": "com.deviniti.issue-templates__template"
          }
        ],

Example of a complex query:

/rest/api/2/search?jql=issue.property%5Bcom.deviniti.issue-templates__template%5D.enabled+%3D+%22true%22+AND+issue.property%5Bcom.deviniti.issue-templates__template%5D.isTemplateSelectable+%3D+%22true%22+AND+(issue.property%5Bcom.deviniti.issue-templates__template%5D.projectsIdsThatTemplateAvailableIn+is+empty+OR+issue.property%5Bcom.deviniti.issue-templates__template%5D.projectsIdsThatTemplateAvailableIn+%3D+11124)+AND+issuetype%3D%22Task%22&properties=com.deviniti.issue-templates__template&fields=issuetype&startAt=0&maxResults=100

JQL before encoding:

issue.property[com.deviniti.issue-templates__template].enabled = "true" AND issue.property[com.deviniti.issue-templates__template].isTemplateSelectable = "true" AND (issue.property[com.deviniti.issue-templates__template].projectsIdsThatTemplateAvailableIn is empty OR issue.property[com.deviniti.issue-templates__template].projectsIdsThatTemplateAvailableIn = 11124) AND issuetype="Task"

I am sharing in hope that you will gain better understanding how we are using the API.

Now my journey to find out more performant search begins

TBC

Do you have any tips how to make a query more performant? Some time ago, I learnt that adding project to the query makes it looking only in that project and, as a result, narrows down the search. However, that doesn’t help much as the templates can be kept in all projects.

Is there a performant way to get issues with a particular property?
How does the JQL works behind the scenes? Does it work like a pipe, i.e. computes the first section, then computes the next section after AND/OR with narrowed number of issues?

Hi @maciej.dudziak! Sorry for the late response. I have checked that your request type is being handled :wink:

I am sure that you already got answers from our support, but just in case:

Is there a performant way to get issues with a particular property?

It depends on tenant data shape. The tenant with a huge number of properties may have a performance issue.

How does the JQL works behind the scenes? Does it work like a pipe, i.e. computes the first section, then computes the next section after AND/OR with narrowed number of issues?

No, JQL does not work like a pipe. It is being executed as a query. Although, limiting JQL results with additional AND clauses can improve query performance. It is also good to narrow the scope, ex. adding project name, just as you mentioned.

1 Like