Multiple webhook migration listener callbacks in every migration, why?

Cloud Migrations
1

When running my migration testing from from my local server to cloud, I get three webhook callbacks to my listener every time when running a test migration.

Here is snapshot from ngrok inspector showing the three calls.
image

For example just on my test now I got following calls:
14:36:28 → First callback
… transferId: ‘8e6be318-27d3-3e5f-a4d8-251a645d22ea’
… webhookEventType: ‘LISTENER_TRIGGERED’,.
… messageId: ‘d8738c8e-19ac-4aee-a2bb-13266ba1580c’
14:37:49 → My endpoint returns 200 OK
14:39:48 → Second callback
… transferId: ‘8e6be318-27d3-3e5f-a4d8-251a645d22ea’
… webhookEventType: 'LISTENER_TRIGGERED
… messageId: ‘d8738c8e-19ac-4aee-a2bb-13266ba1580c’
14:39:43 → My endpoint returns 200 OK
14:43:59 → Third callback
…transferId: ‘8e6be318-27d3-3e5f-a4d8-251a645d22ea’
…webhookEventType: ‘LISTENER_TRIGGERED’
…messageId: 'd8738c8e-19ac-4aee-a2bb-13266ba1580c
14:44:10 → My endpoint returns 200 OK

So basically my processing inside the migration listener takes around 10-11sec and every request ends up returning 200 OK after that time. Is it ok to spend 10-11sec on that endpoint? Or should I kick the processing into async method and return 200 OK immediately?

Why is this happening? Have I missed something in the instructions about handling the webhook call, I thought that just returning 200 OK should be enough for Confluence to be satisfied about the callback being finished ok.

EDIT: Ok, it’s not stopping…
image

Confluence still making callbacks after I finished this posting… But I did notice that it’s doing them in exponential intervals, so the intervals between the calls is about minutes: 1,2,4,8 → It’s apparently doing retries, but why?!?

Hmm… should I maybe return something in the 200 OK response body to satisfy Confluence? I didn’t see anything mentioned about that in the documentation?

2

Well, answering to myself if somebody else is wondering the same. I did the two following things:

  • Now I return 200 OK in about 250ms from the start of the request and continue the migration post processing asynchronously.
  • I include the messageId in the 200 OK response, just a guess if it might be needed for Confluence to correlate the event call with the response.

Not sure which one of the above was the issue, but now I get just one migration listener event call per migration as desired.

Maybe someone has an educated guess / knowledge about what I did there that fixed it.

2 Likes
3

Hi @PetriJuhaniRiipinen,

In answer to your question, yes we have a 10 second timeout. From the Webhooks and events page in the documentation

image
image1458×176 32.8 KB

And we do re-tries on the webhook if we don’t receive a response. It’s expected the receiver is idempotent as well, because in very rare cases we might send a second webhook call.

James.

4

Hi @jrichards

Ok, my execution time was the reason then, barely just passing that 10s timeout in each call, feeding the callback timeout retries each time. This was just my local test migration, the execution time would definitely be way over 10s in a realistic production environment so returning immediately and letting the async method take care of migration changes sounds like the appropriate solution.

The callback is indeed idempotent, it will fix the macros on the first run and on every call after that it does iterate through the macros on each migrated page, but it will see that the pageid-references inside macros are fixed and won’t do anything to them anymore.

Thanks,

  • Petri
1 Like