Hi, dear colleagues.
We are creating a Connect app and configuring it via descriptor. In particular we’ve specified “baseUrl” property at which all the webhooks are delivered. Please see the documentation for Connect apps: https://developer.atlassian.com/cloud/jira/platform/connect-app-descriptor/.
All the Jira Cloud webhooks arrive at this url with certain postfixes depending on the webhook type. As an example issue webhooks for our app arrive at htts://our-connect-app.com/jira/wh/issue/…
We also use REST API v2 endpoint to get failed webhooks (https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-webhooks/#api-rest-api-2-webhook-failed-get). The response for this REST API request contains a URL field for each failed webhook (this URL is called “The original webhook destination.” in the doc). The doc itself is a bit unclear and I can suggest that this URL corresponds to the one specified in the app descriptor.
However we noticed a strange thing: sometimes this URL in the response has a base of the customer’s Atlassian site URL. This is NOT our app’s “baseUrl” specified in the descriptor. Here’s an example URL from the response https://some-customer.atlassian.net/jira/wh/issue/… (again, we’re expecting the address to be htts://our-connect-app.com/jira/wh/issue/…)
We observe this problem for at least 2 months and 99% of such strange failed webhooks come from a single customer. So we need a help to understand whether this is an expected behaviour or it’s a our misconfiguration or something. The general question for us is: “Can we rely on the assumption that the URL in the response for each failed webhook is the app’s URL from the descriptor?”
Thanks for your report. For sure the endpoint should return only webhooks that should be delivered to your app, but failed from various reasons.
The documentation says:
Returns webhooks that have recently failed to be delivered to the requesting app after the maximum number of retries.
If you observe some other URL than yours in the response, it sounds suspicious. As the case is related to a particular instance, the best idea is to create a support case and we may proceed with further investigation then. It will be beneficial if you include some additional data about your app in the ticket (at least app key).
Thank you, Beata. Created a ticket: Jira Service Management
Thanks for the collaboration on the ticket. Sharing the findings from the investigation here so we have that information publicly available as well.
The problem reported is expected due to a fallback mechanism that exists when constructing the destination URL of the Webhook. This can be triggered if there is a problem retrieving the app base URL after an event occurs, which can happen due to database communication issues. In those situations, the tenant URL is used instead, and the webhook fails to be delivered as it does not match the app URL. This is then registered on the failed webhooks API endpoint as observed by Vitaliy.
We also identified a second possibility to trigger this issue, which is through registering a webhook dynamically. This API does not validate the URL passed on the request body, so it can lead to failures in delivering the webhook if an incorrect URL is used. We have created an internal bug for that second scenario after discussing it with the engineering team. The dynamic webhook registration API should recognize when a Connect app is using it and only allow the app’s base URL to be passed on the request body.
Thank you all once again for the help in this investigation!