View-restricted pages cannot be found through AP.request('/rest/api/search') anymore

A critical issue has emerged that is crashing our app (Scroll Documents) for some of our customers.

I have a page with the ID 123. I can access it through /rest/api/content/123 and /rest/api/search?cql=id=123.

If this page is restricted so that only I can view it, I can still access it using the browser through both URLs. However, using AP.request, I can only access it using /rest/api/content/123, but not through /rest/api/search?cql=id=123.

I have narrowed down the problem to a HTTP header that AP.request is adding: Ap-Client-Key: k15t-scroll-document-versions-for-confluence. When this header is present, access through AP.request('/rest/api/search') does not work. When it is not present, it works.

I have also narrowed down and found out that the problem only exists for pages that the app user does not have access to. If I add the app user specifically to the page restrictions of a restricted page, it works again. The problem also exists for unrestricted pages in spaces whose permissions are not allowing the app user.

So far we have not been using the app user at all, so we have been telling our customers that they do not need to give it access. From a security perspective, this new behaviour also doesn’t make any sense, since it is possible to access the content, just not through the search API.

From the reports of our customers, it seems that this behaviour started a couple of days ago. It seems that it started before the broken new search API went live.

6 Likes

In our case, we have an app that invokes the search API from the UI via AJAX.

A customer recently reported that broke for them. The search was returning no results because our app did not have permission (all of a sudden?). We determined that the Ap-Client-Key request header was involved in the problem. The value of that header was the plugin key for our app. Without the header the problem did not exist.

In our case, the user had revoked all permissions for the ‘confluence-users’ group for the problem space.
When they either add All --> View for confluence-users OR they add All --> View for the “app user” that represents the app in Confluence the problem goes away.
This implies that our app is probably inheriting permissions from ‘confluence-users’ and also the timing of this implies that invocations of the search API by client-side code that is in an app are now being permission checked and maybe were not checked previously?)

This appears to be Confluence plugging a security hole. That is great but it changed behavior in a way that is resulting in support cases for partners and some difficult troubleshooting.

If this is a breaking change made for security I can see that would be tricky to communicate before it was released but the details of the change in behavior are so complex w.r.t. “app user” permissions that I think some explanation from Atlassian to partners about changes in app user permission checks would have been in order. This one was really hard for us to figure out.

5 Likes

Hi @candid and @TureHoefner,

The AP-Client-Key HTTP header has always been added to requests made through AP.request. It informs Confluence that the request is being made on behalf of the app identified by the value of the header. This is used for checking the scopes of the app.

I suspect this change may be due to recent changes in the Confluence search implementation. The Confluence team will be back online shortly and I will bring their attention to this.

Regards,
Dugald

3 Likes

Thank you @dmorrow:
Thanks for the info.
It would make sense that this is due to a change in the search API. We use plenty of APIs and this search API issue is the only behavior change w.r.t. permissions that we have noticed recently.

2 Likes

Hi @candid & @TureHoefner,

Thanks for raising this. So we are in the process of rolling out a new search implementation which could have impacted some permissions restrictions which were previously not checked incorrectly.

As @dmorrow mentioned, the connect addon uses it’s own user account and if you restrict content in which that bot addon is not in, then the content will be restricted.

If you access search successfully via the url then this is because it’s either using a current authorized session and that user has access to the content OR the content is anonymous.

Could either of you please clarify the above scenario and the permission structure so we can identify if there’s an actual issue here other than the Connect Bot not being part of a group/permission setttings to restricted content.

I will also work with @dmorrow to investigate and debug some more.

thanks for your patience,

Glenn.

1 Like

@GlennMartin I assume by “Connect Bot” you mean the app user? AP.request() always uses the context of the current user, never that of the app user. That is the only thing that makes sense from a security perspective, and it is what all other REST endpoints are doing.

1 Like

Hello @GlennMartin:
For our use-case it seems that the search API is working correctly. It is a change in behavior but maybe previous behavior was a security hole.

Our app is invoking the search API via AP.request and in the developer console we can see the request is made with the ap-client-key request header set to our app’s key.

Our customer claims that, until recently, they were able to successfully use that part of our app and then the search started returning no results all of a sudden.

They have a space that had NO permissions granted to ‘confluence-users’ group. The space did NOT specify our app user in the Individual Users section of the space permissions so the app user had no permissions either. So, if it is true that this used to work for them and the ap-client-key header was present when it was working, then the search API was not checking permissions using the ap-client-key header until recently. I suspect that our app was being allowed to query a space for which neither ‘confluence-users’ nor the app user had view permissions.

Our customer says they did not make any changes to permissions in Confluence so that is why we are fairly confident that these new Atlassian changes are involved. Our customer is fine now, they added our app user to Individual Users and gave that app user permission for the space and now the search API is working for them in our app.

Let me know if you need more information.

Thanks!

2 Likes

@candid thanks for clarifying, we’re looking into it now and will provide an update.

What we hear from our clients (affects different our apps) is that the standard search for the page in the default macro editor fails for them to find the page.

Just to clarify - the parameter we use in the app descriptor is

"type": "confluence-content"

And we have a standard dialog for the macro editor (not overriding it ourselves)

Interesting that the page they are looking for is unrestricted.

Another issue is with

/rest/prototype/1/search/name.json

Old endpoint we have used to do a quick search for pages (worked just fine for like 2+ years), but not anymore… At least not for all the clients and we are unsure why.

Cannot reproduce it ourselves at all. We call it through Spring boot in user credentials

atlassianHostRestClients.authenticatedAs(hostUser)

works just fine for us, but not for one specific client

Not sure if both are the same issue, but they started to cause troubles at the same time

I should have mentioned that, during testing to reproduce the new permission check for app users, we noticed that the the filtering of spaces at /wiki/spacedirectory/view.action is not completely working. We have not noticed until now the odd behavior we are seeing.

To see the worst possible behavior try it for a space with a 2 word name like “Test Space”. “Test” finds it, “Test S” does not.

/wiki/rest/spacedirectory/1/search?query=Test+S+OR+Test+S*&status=current&limit=24&start=0&_=…

Sometimes “Test Space” finds it and sometimes it does not. The problem w/ a search filter like “Test Space” is in the JavaScript event handler. If you type really fast then the search is for “Test Spac” not “Test Space” and searches with a partial 2nd word do not work.

1 Like

@sash011 I’m not sure, but maybe your issue is related to this: "Current user not permitted to use Confluence" with OAuth user impersonation on some instances

What I have discovered is that there are some unofficial REST endpoints that are not supported to be used through AP.request(). On most instances, those can still be used through OAuth authentication, but on some instances (running the Free Plan?) this is not possible. I haven’t found any solution for this.

@candid and others, the investigation is still ongoing. Some news: the issue has been escalated, the cause appears to have been isolated, and they’re actively working on a fix. We’ll ask the team to provide more details once they’re available.

Any updates on the ETA for the fix?

We have customers who are unable to use our app due to this issue.

Thanks!

Hello. Thank you for being patient with our investigation. We have just put in a fix for add-ons which impersonate users. ETA is Monday/Tuesday next week. I will update this thread once the fix is out.

3 Likes

Hello all. Our fix was rolled out about an hour ago. Can you please verify that things are working now?

2 Likes

Hi @SwatiKatta,

Thanks for the checkin.

I think this is not fixed for a user of ours serving info via a KM portal with content hosted on Confluence to Jira Service Desk users

Does this issue have a public associated Jira ticket to track? If not should I create one?

The error is:

disallowed to impersonate the user because ‘no valid active user exists’

But our user says this was working correctly a few weeks ago.

The problem she has experienced tracks with the timing of this “issue post”.

On a side note I’m finding it challenging across multiple dimensions not using Jira to track an issue like this (created, assigned, less dense text, status, status changes, etc), is that an oversight or intensional or am I missing the associated Jira issue?

Cheers,
Brendan

1 Like

@brendan This seems to be a different issue from what was mentioned in previous messages in this thread. The fix we rolled out last week was related to missing results in case of impersonation and also a security issue and both of these were fixed. Can you please start a different thread or open a public facing ticket so that the customer can track this issue? Please assign the public ticket to me. Thank you.

Thanks @SwatiKatta , customer opened an Atlassian support ticket so will follow up there first in conjunction with Atlassian. Seeing if we can create a work around on our end as well.