Testing your cloud app availability with AWS Synthetic Canaries

We are trying to use AWS Cloudwatch Synthetic Canaries to monitor the end-to-end availability of our Jira Cloud Connect app.

We set up a Jira Cloud instance that uses the production version of our service; then we recorded a puppeteer script (that goes to the login page of that Jira, logs in, navigates to the main page of our app, and confirms that it is working); then we set up an AWS Synthetic Canary that runs this script every 5 minutes. Sweet…

The problem is, we can’t control where this canary runs (AWS seems to change location from time to time) and when the canary runs from an unexpected location, Atlassian Cloud registers this as a suspicious login and blocks access. All subsequent script runs fail.

How can we improve this?

Update: We couldn’t get the canary running again. Canary logins are failing even after when click the “this was me” button in the warning mail and then reset the password.