How to disable extra login steps on an Confluence Cloud instance used only for end-to-end testing?

Confluence Development Confluence Cloud
,
21

Do you login same user in a parallel way @Shu ?

22

Our e2e tests could be running in parallel (2x, one for Jira and one for Confluence, but both would trigger logging in using the same user), and we might also re-authorize between tests (something around 3-5/min max when thing are normal). But “Invalid code” error could trigger retries at 10-15 seconds interval.

Come to think of it, things got worse after I implemented the retries. I always thought it might be clock drifts that caused the invalid code error. But if this error is shown for rate limit, then maybe the retries should have a longer intervals. I might have to give this a try.

23

I think what you are experiencing, is that every OTP code can be used once. If same user will try to use same code during 30 seconds, if will fail.

Anyway, we ditched this approach, and we login once per 24 hours with all users and save cookies/session and then reuse this.

24

Ah, that makes total sense. I can see this clearly when there are 5 parallel workers then 4 of them fails right away. Thanks a lot!

Looking back I should have realized it’s One-Time-Passcode after all!

25

I made exact same mistake my friend :slight_smile: And was getting crazy for some time

26

Good point! I occasionally saw OTP-related errors as well and parallel test runs might explain that, as I’ve been optimizing some tests lately.

Are you storing all cookies or do you have a list of specific cookie names that you store (and don’t mind sharing)? I also noticed the JSESSIONID cookie getting renewed a lot in the cloud (even on each page load?), although this didn’t cause any troubles so far. Wondering if that approach keeps working.

27

I use Playwright, which I believe stores all session data when following this guide: Authentication | Playwright. I ended up creating a job in our Github workflow to carry out log in with Playwright and save the session data, then cache the data and restore in the (multiple) test jobs. Worked nicely so far.

[Edit] There are some security risk for doing this kind of caching, but given it’s a private repo and the test user only have access to dedicated test sites, I think the risk is acceptable.

28
  1. We store all cookies, not to think too much
  2. Those credentials are valid at least 24h or more

We encrypt session data in a separate repo using git-crypt

29

Hi @Shu, our Confluence automated integration tests are consistently failing due to the purple forced “check out this new feature” in Confluence. Can you share how you bypass these popups?

Thank you
Chris

image
image311×304 42.2 KB

30

I added extra steps in my UI automation to find those popups (via CSS selector, or XPath) and click whatever close button can be found.

Unfortunately, many of the UI elements are not easily identifyable by ID, data-automation-id, or class name (or whatever). They are just generic controls and thus hard to catch. Tests tend to break and there is the risk of clicking the wrong buttons.

So I’d be fine with the UI behaving like there was a real end user - but please, Atlassian, annotate actionable UI elements with data-automation-id attributes (or similar) to allow reliable automation.

31

Thanks @heinrich-wikitraccs for your thoughts.
Atlassian must have a way to do their own end to end testing and bypass the pop-ups. Wouldn’t they?

Chris

32

Well……

33

@Chris_at_DigitalRose I used to be able to dismiss the promotional modals by logging in as the test user on my local machine and dismissing the modal(the dismissal seemed to be stored on the server-side). But recently I’ve seen cases where the dismissal is only stored on Local Storage on the client side, which requires a different approach.

My current workaround is to figure out the flag from the local storage and then set it before exporting the auth file, for example:

// Setup overrides for spotlight features to unblock tests
// 123456:12345678-f496-4831-b617-1234567 is the user account id for the test user
await page.evaluate(() => {
  // Spotlight "Automation is now available on Free and Standard plans"
  localStorage.setItem(
    'localStorage/atlassian.123456:12345678-f496-4831-b617-1234567.frontend.automation.onboarding',
    '"true"',
  );
});

// Save signed-in state
await context.storageState({ path: authFile });

Still PITA to maintain though.

34

Thanks @Shu for the how to!