RFC-29: App Access Rule - Revised followup to New Data Access APIs

[AngelinaIgnatova]

Hi @AndreasEbert,

I am unsure how my response relates to locked custom fields, but happy to clarify anyway.

Note that in the context of the App Access Rule both locked and not locked fields will behave in the same way.

How an app can create locked custom fields

Looking at Forge fields, fields created by the custom field module are indeed locked (as described in the Field lifecycle section of the documentation). Conversely, fields that admins create from custom field types aren’t.

How an app can an app update an existing custom field as locked by our app

It’s not possible to change this on a whim, all custom fields are locked by default, and all fields created from custom field types aren’t. I am afraid those are not configurable.

Hope this helps,

Angelina

[AndreasEbert]

@AngelinaIgnatova, oh, maybe I misunderstood your post on RFC-25 where your screenshots show text about “fields associated with an app”. That is what I mean with “locked custom fields”. Please correct me if I have the terminology wrong here.

Anyway, if I understand you correctly then these fields associated with an app are only available for Forge apps, not Connect apps, right? Your links point to the Forge docu, but I don’t see similar pages about such “locked” fields in the Connect documentation Since our app is a Connect app, we would need it for Connect.

Thanks for any clarification.

[AngelinaIgnatova]

HI @BorisBerenberg,

Hi Boris,

Note that earlier this year 37% of server/DC customers surveyed (n=266) indicated that they need to be able to identify which cloud apps give them more control of where the data is stored, and what sensitive data (PII, UGC) apps have access to. We see this as a strong signal that customers will feel more comfortable moving to cloud if they can exercise more control over the content apps can access.

Hope this helps.

Cheers,

Angelina

[AngelinaIgnatova]

Hi @danielwester, hope you have been well.

We hear your concerns about the potential confusion that the App Access Rule may cause to customers. We are also mindful of the end user impact and have implemented (and continue to implement) custom block section messages that we believe will positively affect the end user experience.

We will be updating the documentation to clearly explain how the feature works and how it will impact the end users of apps. Through the policy creation flow we are encouraging org admins to inform their end users how the App Access Rule will affect their experience:

We are informing users that their organisation admins are in control of the app blocking and encouraging them to reach out to their org admins in effort to reduce the likelihood of end users reaching out to you.

Here are a few examples of what end users will see when interacting with apps:

Confluence

All Confluence macros that would be displayed within a restricted space will instead display an error message to users indicating that the app has been blocked by a policy in place of the macro.

App Pages & Space settings

All UI modules that would be displayed within the context of a space will not appear if the app has been restricted from accessing that space, as custom block messages at those interaction points would have been too overwhelming for end users. However, when end users reach out to their site admin to enquire about a change in the app behaviour, site admins will be able to easily identify which apps belong to an active policy as those apps will appear with a tag in Connected apps in Admin hub and Manage apps in the Universal plugin manager.

Jira

Custom fields would be hidden for any blocked apps, however product admins will be able to use the “Find your field” tool to troubleshoot

Layout

If either of: Issue Action, Issue Activity, Issue Context, Issue Glance, Issue Panel, quick add button are hidden as a result of an active App Access Rule, a message will display for admins initially. We can expand this message to all users if we hear that you are receiving an increased amount of support requests specifically for hidden items:

Project settings page and project page

Note that we will not be introducing any changes to the workspace modules, it will be up to the app owners to decide whether/how you want to handle the changes to the end user experience.

Hope the above brings more clarity about how we are addressing the end user experience. As aways, we welcome any feedback.

Cheers,

Angelina

[AngelinaIgnatova]

Hi @julia,

Thanks for raising your concerns with the App Access Rule.

Will this will be available to app vendors before the feature reaches our customers?

The short answer is yes, we plan to give partners a chance to test starting by the end of this calendar year, well in advance of the GA release for customers.

Note that this quarter we would be looking to expand our Customer EAP by inviting Confluence cloud customers who have already created policies using other data security rules. The EAP will still be focused on Confluence only for now, but it will include apps built on Connect and it will be open to any Confluence Cloud customer who wishes to test the feature.

In the same time we want to give you a chance to test out how the App Access Rule affects your apps’ end user experience and how you would take advantage of the APIs to stay informed about polices that affect your app access to data. So keep an eye out for a chance to sign up in the change log in the coming weeks. Additionally, we are also considering rolling out the feature to the Canary developer instances.

Early next quarter we are looking to add the Jira family to the App Access Rule.

It would be nice if Atlassian could provide tooling instead of each vendor doing their own implementation/support effort

I have provided more details about what Atlassian is doing to inform end users, product and site admins about how the App Access Rule works and why they are seeing changes in the apps’ behaviour. Keen to hear if there are any major gaps that you can notice?

If we would go with the option of removing this data when the app no longer has access, then there wouldn’t be a way to restore it if the customer unblocks our app.

In the documentation we are advising customers that data stored by apps may get deleted by partners and may not be restorable should customers decide to disable/delete any app access rule polices.

Another option would be to keep the data forever, with the drawback of having data stored that is never used and that it might not be what the customer expects when blocking our app.

We can not make any specific recommendations about retention of customer data (or data derived from customer data) after additional data controls are applied. App developers must make their own determination based on their app use-cases, the needs of their users, and any related agreements they have established with their customers. Due to the dynamic nature of legislation and regulations relating to how data is handled in different jurisdictions, Atlassian cannot share specific guidance on this matter. However, we would be looking to obtain more feedback from customers regarding their data retention expectations and will make sure we share our findings with you.

Cheers,

Angelina

[BorisBerenberg]

@AngelinaIgnatova

Note that earlier this year 37% of server/DC customers surveyed (n=266) indicated that they need to be able to identify which cloud apps give them more control of where the data is stored, and what sensitive data (PII, UGC) apps have access to. We see this as a strong signal that customers will feel more comfortable moving to cloud if they can exercise more control over the content apps can access.

This feels like an almost willful misinterpretation of the customer feedback to justify an extreme feature design. If this is really the best data to justify this feature, I would request an executive escalation and review of this.

[danielwester]

@AngelinaIgnatova ,
That’s great (and for those apps that are directly blocked - awesome!)

But in our case and anyone else that makes heavy usage of search (both in confluence and in jira) - the big headache will be troubleshooting (and explaining) why an issue/board/etc isn’t showing up. This is the area where I feel a lot of us vendors would be looking to Atlassian to also help out.

Scenario - customer puts in a ticket of “Your app isn’t showing anything from project Y in your general page - please fix”. How would we go about troubleshooting this? (assuming that it’s blocked by the app access rules but the customer putting in the request doesn’t tell us and isn’t a site admin - we don’t have a project based link etc).

Thanks,

/Daniel

[remie]

@AngelinaIgnatova I second the concerns raised by @danielwester.

To give an example: our app Version & Component Sync allows project administrators to synchronise releases and components between projects.

They do this by selecting the target project from a list:

In the current version of this RFC, we will still be able to populate this list because we will continue to have access to the project level. This means that the user will not be limited when creating a link for release / component propagation. However, once the link is created, the actual synchronisation will not work because we will not be able to access the project releases/components.

So we will either have to check app access for each project in that list to see if it is eligible for synchronisation (slow, cumbersome, lots of API requests), or allow the user to finish setting up their project only to alert them of our failure to synchronise.

And that is just during project link creation. What happens if app access is blocked whilst there are already projects actively synchronising?

We have Enterprise customers who are synchronising 200+ projects, often spanning multiple PM’s. It will take a long time before they will notice that one project is no longer getting version/component updates. In the few cases in which we have had this in the past, this led to serious data integrity issues.

Due to the nature of Atlassian being very protective of end-user data it is extremely difficult for us to notify the appropriate person that app blockage will impact synchronisation.

The issue isn’t in the easy parts, but in the 1% of use cases in which partners are doing complex operations. This is where our support requests and angry customers will come from.

[remie]

Hi @tpettersen,

To be honest, I actually took a lot of consideration in that post. The personal nature of the remarks was deliberate, not to disqualify @JustinThirkell, but to also highlight that Atlassian can’t have her cake and eat it.

Either Atlassian is operating as an entity and engineering decisions are made with consideration of a bigger picture, or Atlassian teams operate holistically and autonomous.

If it is the former, I would be able to appeal to a larger body that can evaluate these decisions and override the team. However, as per your comment, this is not the case

we do not yet have a consistent approach to technical API considerations across the board.

That means that we appeal to individual teams, and we apply a thin layer of veneer of decency by pretending that authors are just representatives of a larger organisation, avoiding ad hominem remarks by always referring to “Atlassian” instead.

But when our feedback is dead on arrival, with no possible escalation policy, there comes a point in which I feel that a heartfelt appeal to the individual becomes inevitable. Because whether we like it or not, Atlassian does have a “not invented here” problem, and it does have a huge employee turnover problem, and it does have an OKR problem. That is not something individual Atlassian are to blame, but sometimes they need a reminder of this, because Atlassian also has a lack of internal partner advocacy problem.

So whilst I never meant to offend @JustinThirkell, I do think there are times in which it is important for individual Atlassian engineers to understand the impact of their decisions upon an ecosystem that was there long before they started and will be there long after they are gone.

[JustinThirkell]

@remie

we will have to check app access for each project in that list to see if it is eligible for synchronisation (slow, cumbersome, lots of API requests)

Hi @remie , I’m unclear why the API we propose to (get the status of provided containers won’t work for the use case you describe.
We were expecting that when you call the API that you would pass in an array of project identifiers that you want to check access for, not call the API separately for each project. Does that help with your use case?

Also, it’s worth noting that while this API is described in this post as a graphQL resource and as a resource in the OpenAPI contract, I have just discovered that the short example of the RESTful resource was being parsed as part of the OpenAPI contract - I have fixed this now and it is rendering properly here. Sorry about that!

[lexek-92]

Hi @JustinThirkell

Can this information be also embedded somehow into project objects within the standard REST API? This would greatly reduce amount of API round trips required to display clear end-user messaging.

Also, it seems like there is no API changes to cover arbitrary JQL search, which is going to be a big pain point as @danielwester highlighted.

[JustinThirkell]

@danielwester

Scenario - customer puts in a ticket of “Your app isn’t showing anything from project Y in your general page - please fix”. How would we go about troubleshooting this?

Hi @danielwester , would the API endpoint to get the status of the project in question help here?

Also, it’s worth noting that while this API is described in this post as a graphQL resource and as a resource in the OpenAPI contract, I have just discovered that the short example of the RESTful resource was being parsed as part of the OpenAPI contract - I have fixed this now and it is rendering properly here. Sorry about that!

[JustinThirkell]

@lexek-92

Can this information be also embedded somehow into project objects within the standard REST API?

Hi @lexek-92 , the short answer is we’re sorry but no this is not possible, for the same reason that http response codes could not be changed. Modifying either API status code semantics or response bodies would obviously be a breaking change, and it is not possible for us to manage that (across the API surfaces of Confluence and Jira) within the constraints we have.

[lexek-92]

Hi @JustinThirkell

@JustinThirkell

Hi @lexek-92 , the short answer is we’re sorry but no this is not possible, for the same reason that http response codes could not be changed. Modifying either API status code semantics or response bodies would obviously be a breaking change, and it is not possible for us to manage that (across the API surfaces of Confluence and Jira) within the constraints we have.

To be honest, it doesn’t make sense even with your reasoning why it would be impossible to add something like boolean restricted field into “Project” object, which would cover a really big chunk of use cases for end-user messaging and business logic without introduction of additional API and extra API call overhead.

@JustinThirkell

Hi @danielwester , would the API endpoint to get the status of the project in question help here?

It doesn’t cover arbitrary JQL search in case of Jira. And with the current rest API it’s not realistically possible to reliably extract all project ids that would be involved in execution of arbitrary JQL query.

[remie]

@JustinThirkell

Does that help with your use case?

The issue is not that there isn’t a way to get this information, the issue is that it’s yet another API call.

We are already stacking API calls to A) get the list of projects and B) check if the user has appropriate access rights and C) check if the project isn’t already in use for synchronisation. Adding another API call makes the logic more complex and will increase waiting time.

It would make a lot more sense if this is just integrated into the PageBeanProject or Project responses from existing Project REST endpoints

[remie]

@JustinThirkell

Hi @remie , I’m unclear why the API we propose to (get the status of provided containers won’t work for the use case you describe.

Can you also comment on the second part of the use case, in which app access is blocked after the user configured the project synchronisation, in which synchronisation will fail without any warning and we do not have any means to communicate this to the end-user?

[marc]

Hi @JustinThirkell ,

@JustinThirkell

Hi @lexek-92 , the short answer is we’re sorry but no this is not possible, for the same reason that http response codes could not be changed. Modifying either API status code semantics or response bodies would obviously be a breaking change, and it is not possible for us to manage that (across the API surfaces of Confluence and Jira) within the constraints we have.

As an alternative, would it be possible to add a header to the API response, something like:
Access: restricted
That should be backwards compatible.

[danielwester]

Not really… That assumes that we know the project id ahead of time. For us - the project won’t be existing at that point so it won’t be returned back in searches. If there were ui elements on the project/issue etc denoting it was “locked” down that either the project admin or the user can see to explain things then we can point them towards that.

Otherwise - this will be a potentially expensive feature that we end having to check with all of the support cases (expensive and really annoying for all users).

[julia]

Hi @AngelinaIgnatova,

Thanks for your response!

I have provided more details about what Atlassian is doing to inform end users, product and site admins about how the App Access Rule works and why they are seeing changes in the apps’ behaviour. Keen to hear if there are any major gaps that you can notice?

It would be nice to have the possibility to communicate the impact of blocking our app to the administrator who is blocking our app before they actually block it. In case we decide to remove the data when we’re blocked we would like to be able to inform the administrator blocking us what impact it would have and that their data won’t be possible to restore. I understand that this wouldn’t be part of the first implementation, but could serve as input to future iterations based on how much problems we see in this space.

However, we would be looking to obtain more feedback from customers regarding their data retention expectations and will make sure we share our findings with you.

That would be much appreciated, thank you.

[remie]

Nudge @JustinThirkell / @tpettersen