Forge storage conversion issues

Hey,

I’ve discovered this really weird behavior when trying to use this endpoint to convert something from storage format to any other format (see possible targets in linked doc):

No matter what target format i use I get a response “401 unauthorized” most of the times. But i don’t always get that response, which is really weird. I can do the exact same thing 10 times and i will get a 200 with the right content once or twice and the other times I’ll get a 401.
According to the docs the oAuth scope read:confluence-content.all is required for this endpoint, the same scope is required for the inline tasks endpoint, which works 100% of the times, so the scope is definitely present.

If you want to try this out yourself I set up a sample project using both endpoints and logging the results for the conversion api. See GitHub - ItzBubschki/forgeConversionIssue: A little repo to demo the issues with the forge conversion api.
Check the readme for more information on how to use this repo to reproduce this issue.

Am I doing something wrong or is this an issue in Atlassian’s backend?

Cheers,
Laurin

5 Likes

Hi @LaurinNiemeyer,

I cloned your repo and registered, deployed and installed your app, pressed “convert something”, authorised my app, and then pressed “convert something” a few more times, and received 10 calls returned status code 200. each time.

Are you still experiencing this issue? Are you able to run npm install and then try again to see if there are any issues with the node_modules possibly? If you are still experiencing the issue, I can pass this onto the team that owns the API to see if they can find anything in the logs.

1 Like

Hi @LaurinNiemeyer,

Is it possible that you are hitting the problem described in [FRGE-212] - Ecosystem Jira ?

If the content you are trying to access has space or page-level visibility restrictions, your app might not be able to access it via the API.

1 Like

Hey @HeyJoe,

thanks for your reply.
I already tried both “asApp” and “asUser” but neither of them worked. I also tried giving my app all possible scopes but it still behaved as described. The Content I’m trying to convert is for test purposes hardcoded: <span>this will likely fail</span> so I don’t think I’m accessing restricted content.
From what I can read in the Issue which you linked it seems like the users always get a “401 unauthorized” which also isn’t the case for me, as I only get a 401 in 80% of the tries.

Cheers,
Laurin

1 Like

Hey @kchan,

thanks for your reply.

I don’t understand how it is possible that it works for you but not for me.
I’m on a newly created instance with only this app installed. I deleted the app from the apps overview and created a new one. I gave the app all oAuth scopes that were listed on the scope page. I tried the forge api version 0.7.0 and 1.0.0, but nothing worked. I’m still getting a few 200 and even more 401.

As it seems like this is working for you, I can try to record a demo and upload it if that helps.

Cheers,
Laurin

1 Like

Hi @LaurinNiemeyer, sorry for the delay. I have been very busy over the last few days. I will get to this soon!

1 Like

Hi @LaurinNiemeyer, we believe we have found the issue and have made a temporary fix, and are doing further investigation for a longer term fix at the moment. Would you be able to try again?

2 Likes

Hi @kchan,

I Updated to the most recent version of forge and @forge/api but even after re-installing the app it still doesn’t work and responds with a 401 in 8 out of 10 cases.

Is @forge/api 1.0.1 the correct version or is there a different version that I’m supposed to use?

1 Like

Hi @LaurinNiemeyer,

That version of @forge/api is correct, and shouldn’t really cause the issue you’re experiencing anyway. I suspect there is something strange going on in the Atlassian backend that is causing your issue, since it gives me 10 200’s every time but fails 80% of the time for you.

Are you able to add a console.log(await response.text()) after making the request, and let me know what appears in the logs? I hope to see some more useful error message, that might lead me somewhere in my investigation, or that I can bring to the team that owns the API to investigate.

i.e.

        const response = await api.asUser().requestConfluence(CONVERT_URL, {
            method: 'POST',
            headers: {
                'Accept': 'application/json',
                'Content-Type': 'application/json',
            },
            body: JSON.stringify(bodyData),
        });
        console.log(await response.text())

EDIT: I’ve also had the team that owns the proxy that Forge sends the request through sync their scopes configuration to latest, so if it works for you now, that might have something to do with it. Let us know!

1 Like

Hey @kchan,

unfortunately it still doesn’t work.

Using console.log(await response.text()) i get the following output: {"code":401,"message":"Unauthorized; scope does not match"}.

1 Like

Sorry for the delay in reply @LaurinNiemeyer, we’ve been investigating the issue but no one has been able to reproduce the issue you’re experiencing yet, even using the same code.

Is there someone you work with who you could ask to try reproduce the same issue? If theirs works, perhaps you may be able to determine the difference in environment or setup that caused the issue, so we can determine the root cause. Or if theirs also fails, it may help us to determine the common denominator so we can understand the root cause as well.

2 Likes

Hi @kchan,

I tried reproducing this on my machine and got the same issue. Here are the full logs:

The app Forge Conversion Issue in the development environment is now installed in Confluence on sschatter.atlassian.net
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
~/git/forgeConversionIssue(main*) » forge tunnel                                                                                                                                                 sschatter@flagship
Running your app locally with Docker. The tunnel displays your usage from everywhere the app in the development environment is installed.
Press Ctrl+C to cancel.

Checking Docker image... 100%
Your Docker image is up to date.

Reloading code...

=== Running forge lint...
No issues found.

=== Bundling code...
App code bundled.

=== Snapshotting functions...
No log output.

App code reloaded.

Listening for requests...


invocation: be7450042bfd0def index.run

invocation: 07bd7937951890a4 index.run
INFO    09:09:36.135  07bd7937951890a4  Hello world
ERROR   09:09:36.145  07bd7937951890a4  [NEEDS_AUTHENTICATION_ERR: Authentication required] {
  serviceKey: 'atlassian-token-service-key'
}

invocation: dfee715e62ba0e8d index.run

invocation: cd439d5ccae29ae9 index.run
INFO    09:09:56.002  cd439d5ccae29ae9  1 calls returned status code 200.
INFO    09:09:56.003  cd439d5ccae29ae9  9 calls returned status code 401.

Cheers,
Sven

4 Likes

Thank you for that info!

Someone working on the APIs team was able to reproduce the error this morning, and they have some ideas for why the issue is occurring, so we’re deploying some additional logs to determine the issue in more detail. Will keep you updated!

2 Likes

@LaurinNiemeyer @sven.schatter We believe we have determined and fixed the issue (in a nutshell, our API proxy was sometimes getting scopes from an out-of-date duplicate copy of the scopes schema).

Please try again and let us know if it works or not!

2 Likes

Thank you so much @kchan ! It’s working for me now! :slight_smile:

invocation: 999f99d4e9414ac4 index.run
INFO    08:27:26.810  999f99d4e9414ac4  10 calls returned status code 200.
INFO    08:27:26.811  999f99d4e9414ac4  0 calls returned status code 401.
2 Likes

It’s working for me as well! thank you for the help @kchan

2 Likes

I am having this same issue. It says it was resolved a year ago. I have ran forge lint --fix just to make sure, but I am getting the 401 errors described here.