RFC-106: Future of Forge versioning - Permissions

[Chris_at_DigitalRose]

@billjamison agree that the apps should just work. The idea that a Jira administrator wants to take a course in understanding what the 50+ forge permissions mean and decide individually (for each app) which ones to enable is not a reasonable expectation. (Disclaimer: I actually was the lead Jira administrator for a medical device company for several years). Agree a feature checkbox “Send Email Notifications” could be useful, but a checkbox beside a permission list containing “write:custom-field-contextual-configuration:jira” is not useful.

The change management policy is a good question. I’m thinking that these customers have already found a way to deal with the high-impact, forced, non-optional “Atlassian App” changes. If they can handle those, auto update and notification of marketplace apps is minor.

In my opinion, many of the marketplace app changes are driven by atlassian api and platform changes anyways. Atlassian can deprecate any API within 6 months, which can break any app more than 6 months old. Allowing a customer to stay on a 1 year old version of a marketplace app when the Jira/Confluence API’s that the app needs have been killed makes no sense to me.

Our design team had a saying: Equally smart people disagree on design when they are trying to solving different problems. From the outside all I see is unnecessary complexity, customer pain and broken apps. Maybe the view is different from inside.

[MichaelCooper]

Thanks for your feedback @scott.dudley

@scott.dudley

If admins do enable auto-upgrade, let them decide if they’re OK with automatically upgrading permissions within a limited set of scopes only (for example, if the app has the equivalent of Connect READ permission only, then auto-allow the app to inherit permissions to READ any other object type, but don’t allow auto upgrades to WRITE-type permissions). Or perhaps the admins can also decide that they want to automatically accept all permission upgrades. Maybe there are also options to consider here related to accepting Forge egress permissions and other sensitive features.

I think that is along the lines of what we are thinking for the flexibility part (specifically pre-approval of certain features):

@MichaelCooper

Ultimate Flexibility (Release Tracks): Customers will be able to set their own update policies, pre-approve features, and schedule updates to fit their business (similar to Release Tracks)—while still benefiting from protection against urgent threats.

This feedback will help us design the feature, thanks!

[MichaelCooper]

Can you provide more details on why you might be stuck on a connect to forge migration.

Is there examples you have where an app is not able to use the forge version bulk-upgrade command?

[MichaelCooper]

@AnastasiyaGolendukhi

What would help in this case is a built-in mapping that would assume that if the user had permission ABCD in Connect, then permissions A and C in Forge would go over as a minor update without requesting approval.

Thanks for your feedback @AnastasiyaGolendukhi, I just wanted to see if the change announced yesterday covers your concern: CHANGE-2751: Connect to Forge migrations will be minor version updates by default (for eligible apps )

For example, if a Connect app has the WRITE scope, and after adopting Forge, it specifies the write:comment:jira Forge scope, the Forge version of the app is considered to have inherited that permission from the original Connect WRITE scope. If the Forge version of the app adds fetch permissions, that is considered a minor update because it is not adding permissions: Connect apps did not require permissions to be set, to fetch resources.

[AnastasiyaGolendukhi]

We did make use of this in the preview stage. However, the problem with it is that it only works for the initial Connect to Forge release. For subsequent releases, even if we inherited Connect scopes 1-to-1 it would still require a manual update. Expanding that principle to further releases would solve most of our current versioning problems.

It would have worked fine if we released Forge in a single push, but between phased obsolescence, partner requirements, and this being a tremendous amount of work, it’s just not feasible to do in one go. And that is on top of this taking away resources that could be spent building things our customers actually need.

[marc]

In the optional scopes discussion ( Feedback requested: Optional Scopes in Forge ) I’ve already shared my concerns about the overlap of these proposals. If these proposals are developed in isolation, I think this leads to too much complexity for users and developers.

@marc

Optional scopes have so much overlap with:

• App editions - Partner Preview: App editions are coming to Marketplace
• Forge versioning and permissions - RFC-106: Future of Forge versioning - Permissions

I hope that Atlassian can coordinate these efforts, as there seems so much complexity. E.g. what would happen to an optional scope in an older app edition which becomes required or vice versa? Especially RFC-106 has kind of optional scopes already.

[m.herrmann]

I think this RFC is a huge step forward of how app versions are currently handled. Currently we try to not work on features that would result in a major update because a big majority of customers don’t update their apps when a major update happens.

We switched from Connect to Connect On Forge 5 month ago and 3 month ago to Forge only (Runs On Atlassian). Within these 5 month, only 20% of our cloud customers updated their app and 5% still sit on a version that is older than 3 month.

The reason why we could not use the forge minor upgrade path is a design difference between the Write Scope. In Connect there is a write and a delete scope, while the delete scope is included in the classic write scope in Forge. So we needed to decide between a minor update with 10 - 50 granular write scopes where any missing scope would result in a major update anyway (+ new ones required for new features) and a major update with a more maintainable classic scope.

Feedback for this RFC:

• In my opinion there are granular scopes that should not require consent from the customer or should be included in the classic read/write scope. Just to name a few ones
  • storage:app
  • read:app-data:jira
  • write:app-data:jira
  • read:jql:jira
• I think this feature will further decrease the admin interaction with updating apps, which is a good thing. Without further improvement this will also result in a lot of missing scopes for a very long time as we can’t trigger the admin that a user interaction failed due to a missing permission

@MichaelCooper

Can you provide more details on why you might be stuck on a connect to forge migration.

Is there examples you have where an app is not able to use the forge version bulk-upgrade command?

[RonnyWinkler]

Thanks for sharing the RFC. However, I think it is too complicated to be useful and is from my perspective setting the wrong focus in regard to the underlying problem. It also overlaps with the optional scopes topic ( Feedback requested: Optional Scopes in Forge ), as pointed out by @marc. Actually, the described idea is a technical solution for optional scopes. But scope changes which currently lead to major new versions are not necessarily optional. So for example if another endpoint of the Confluence V1 API is replaced with one of the V2 API, another scope has to be added since the V2 API is using the granular scopes. Such a change is definitely not optional because the app will just stop working after the deprecation period of the endpoint. The RFC is missing this aspect.

Regarding the Asks

What use-cases for your apps that this offering would enable?

As described above, only for changes that are really optional. But I would be really hesitant to use it because the maintenance overhead would be too high if there are several scope related switches in the code (testing different code branches with certain scopes enabled/disabled). Handling support cases would also become much more complicated as already mentioned by others, especially if you compare it with the current situation where you just need to know the app’s version the customer is on.
Of course there is the possibility to use this feature in a way that disables the app completely until the scopes are approved so that you don’t have to hassle with multiple switches. But from customer perspective that’s a terrible approach as it leads to a disruptive experience which can’t be solved by the actual users of the app (imagine the admin being on a 3 week vacation).
With all that in mind I prefer the current state over the suggested solution. There is no need to support an old version forever. Developers can for example add a banner or whatever in an older major version which informs the customer that this version is deprecated and will stop working in X days, if they don’t update. After X days, the frontend features of the app could be reduced to a message that the version is not supported anymore. Well, I have to admit that I never had this use case as some blockers prevent the migration to Forge, so I can’t tell if it’s easy to implement or not. But if it is too complicated maybe it would make more sense to simplify this instead of investing in the optional scopes approach.

Regarding the other Asks: not applicable as the suggested idea is not really useful.

Alternatives and different focus:

• Automatic updates as described in other comments. There are also some interesting suggestions to give admins more control (RFC-106: Future of Forge versioning - Permissions - #39 by scott.dudley).
• If automatic updates are not possible, keep the current solution and add some tools to simplify marking old major versions deprecated/unsupported if needed (e.g. some Forge CLI or Marketplace API to mark versions as deprecated or unsupported with a custom message and some Forge functionality to check for it similar to a license verification).
• Improve how admins can be made aware that they have to approve a new app version. Also consider that a “please inform your admin” message in an app is often not that helpful, as normal app users typically don’t know who’s an admin.

[scott.dudley]

As one additional data point, the new freeze window program implies that Atlassian’s maximum window for customers running outdated versions of its own products is no greater than 14 days.

I grant that there are likely fewer permissions-related concerns there, but in terms of the SWE issues that drove Atlassian to set this limit, the underlying principle is probably very similar for vendors.

I hope that this can included in the analysis of what Atlassian chooses to target for subsequent implementation phases.

[ChristoMastoroudesAd]

I discussed these changes among devs I work with.

Generally the reception is positive. Our app has been around for many years and has undergone several changes. This means we have clients on older versions. We are migrating to Forge, but suspect we will have to continue supporting connect customers that do not click the update button, we hope this will assist in moving customers along to the latest version.

Positives:

• App updates are mandatory.

• Admin has fine grained control over egress and scope.

• Admin notifications of updates.

• It increase flexibility and control from the admins point of view, perhaps making them more comfortable with trusting apps and considering app purchases, hopefully leading to increased revenue.

• Detailed patch notes reinforces the admins trust in our product, making his life easier.

• Would be great if this shortens the timeframe during which we need to continue supporting customers on connect that don’t hit upgrade.

Negatives:

• The burden on app developers continually increases, adding complexity and causing slower development, more bugs and maintenance.

• Partners don’t have the large developer base Atlassian has. We struggle to keep up with AGC, IC, FORGE migration and countless RFCs while trying to deliver a useful app to clients.

Dev Suggestions:

• The Rest API should return an appropriate and clear error message if a scope has not been approved, so we don’t have to guess why a call isn’t working.
• Provide a hook we can use instead of the wrapper component.

[rachelminer]

Chiming in to add some notes from our team:

The problems listed in the RFC are the right ones, however the proposed solution imposes extreme amount of effort onto the developers. It essentially means that the apps should replicate the permission checks for all the APIs, introduce some kind of process for static verification to guard against missing the n-th scope check in the m-th usage across the codebase. This could explode infinitely for more complex apps.
Same goes for the thrown idea of giving the customer the ability to disable app features on demand. This increases the complexity and cost of end-to-end development and testing drastically to ensure the app behaves correctly for all the possible permutations.

Level of control on enterprise customers seems reverse correlated to apps release costs (mainly tests of all/most possible cases).

More feedback:

• The code example demonstrates an anti-pattern (IMO). The required permissions are duplicated from the manifest, and the permission check that Atlassian already does is duplicated by our app through an extra API call that will add latency. The same logic could be applied much more easily by developers if the API contract included returning information sufficient for the app to know that they error is due to missing app scopes. In that case, apps could simply make the API call and show the appropriate error to customers if necessary. (The endpoint for checking scopes could still be valuable in some cases though)

• An alternative to checking scopes manually could through a “dry-run” parameter that could be included in any API call.

• The RFC is missing key design decision, like information:

  1. if Admins can only accept new scopes,

  2. or they can enable/disable on feature level (were feature would group scopes and egresses)

  3. or they can enable/disable on granular level

     options 2) and especially 3) (if selected) will make handling our apps much more difficult and will greatly increase cost of testing before each release.

[MichaelCooper]

@ChristoMastoroudesAd

Dev Suggestions:

• The Rest API should return an appropriate and clear error message if a scope has not been approved, so we don’t have to guess why a call isn’t working.
• Provide a hook we can use instead of the wrapper component.

Thanks for your detailed feedback @ChristoMastoroudesAd!

We really do hope that these changes help all marketplace partners reduce the burden on maintaining older versions of their app.

Regarding the Rest APIs, They will definitely return an unauthorised error as it already does now, indicating the permissions were not granted, but as someone earlier talked about, there could be several other reasons for the same error message. Even if we exposed the scope that was missing in the response of the API, it would be ambiguous if the scope was not granted vs never declared. We hope that this permission checking API can help you avoid making the calls in the first place, or avoid showing features to users that do not work.

In regards to a hook, yes we are looking at adding a hook, it might look something like:

const { hasPermission, isLoading, missingPermissions } = usePermissions({
    scopes: ['write:confluence-content'],
    external: {
      fetch: {
        frontend: ['https://api.example.com']
      }
    }
  });

Would that meet your expectations? I will update the original post with this new design.

[MichaelCooper]

@rachelminer

More feedback:

• The code example demonstrates an anti-pattern (IMO). The required permissions are duplicated from the manifest, and the permission check that Atlassian already does is duplicated by our app through an extra API call that will add latency. The same logic could be applied much more easily by developers if the API contract included returning information sufficient for the app to know that they error is due to missing app scopes. In that case, apps could simply make the API call and show the appropriate error to customers if necessary. (The endpoint for checking scopes could still be valuable in some cases though)

Yes, this first stepping stone has the scopes + egress re-defined in the app code.
We envisage that when we get to the optional features milestone, we will define groups of permissions against a key in the manifest, and you can reference that group in the code, instead of having to re-define it again.

@rachelminer

• An alternative to checking scopes manually could through a “dry-run” parameter that could be included in any API call.

That is a good suggestion, we will look into the feasability of this to help testing.

@rachelminer

• The RFC is missing key design decision, like information:
  1. if Admins can only accept new scopes,
  2. or they can enable/disable on feature level (were feature would group scopes and egresses)
  3. or they can enable/disable on granular leveloptions 2) and especially 3) (if selected) will make handling our apps much more difficult and will greatly increase cost of testing before each release.

The vision is that optional features can be enabled and disabled as groups of scopes and egress, but for permissions defined outside of features (as they are now) will be an all-or-nothing approval. App installation will still require all permissions (as currently), and when an admin is reviewing the additional permissions the app is requesting, it would also be an all-or-nothing proposal, until we get optional features in.
I see I was not clear enough in the original RFC, I will update it to clarify.

[MichaelCooper]

@RomanStoffel

‘Force/Encourage’ update customers in some way. Aka have a time limit for old versions. Some good suggestions where made in this post. Basically, if instances are updated by default, in some form.

Thanks for this feedback, We do have some initial ideas on deprecating old versions that will come in future milestones, sitting along with the Security goal we outlined in the vision statement.

[MichaelCooper]

Just wanted to share that we have now shipped the ability to install previous major versions of an app:

• https://developer.atlassian.com/platform/forge/changelog/#CHANGE-2748
• https://developer.atlassian.com/platform/forge/versions/#installing-previous-major-versions

We hope this helps developers debug issues with older major versions, and allow testing of migrations.

[ChristoMastoroudesAd]

Looks good to me.

[SeanBourke]

@AnastasiyaGolendukhi

We did make use of this in the preview stage. However, the problem with it is that it only works for the initial Connect to Forge release. For subsequent releases, even if we inherited Connect scopes 1-to-1 it would still require a manual update. Expanding that principle to further releases would solve most of our current versioning problems.

It would have worked fine if we released Forge in a single push, but between phased obsolescence, partner requirements, and this being a tremendous amount of work, it’s just not feasible to do in one go. And that is on top of this taking away resources that could be spent building things our customers actually need.

Hey @AnastasiyaGolendukhi,

Thanks for sharing this. We recently also released the forge bulk-upgrade CLI command which can be utilised to deploy updates which are major but do-not require admin approval (i.e. the addition of SQL, removal of a scope or addition of a Connect inherited scope) to all customers. Have you had an opportunity to try this?

[AngelinaIgnatova]

Thank you for your candid feedback. We acknowledge your concerns and initially we will offer an optional approach, allowing you to decide whether to be an early adopter of decoupling permissions from versions.

What did we hear?

Some of you have raised concerns about the burden on app developers due to the complexity of checking and handling every permission scope within each app.

We want to clarify that our vision is to have optional features, not optional permissions. Each feature will consist of a unit of scopes and/or permissions, allowing app developers to determine which features are optional and which are mandatory. This will be coming in future milestones, while initially all permissions will be an all-or-nothing approval. App installation will still require all permissions (as currently), and when an admin is reviewing the additional permissions the app is requesting, it would also be an all-or-nothing proposal, until we get optional features in.

Some of you prefer if Atlassian implements an opt-out automatic app updates, including those with elevated permissions, after a notice period. While we understand the appeal of this approach, it does not align with the feedback we have received from customers. Customers tell us they need full control over app access to data, and any elevation of permissions must be blocked until consent is provided.

What did we change?

Display Conditions & Atlassian app events - We will introduce support for checking permissions in conditional display logic and Atlassian app events so you choose whether to hide features and disable triggers that won’t work without the permissions being granted

Test permission changes - We will allow you to install previous major versions and test with different level of permissions. For example, install code version 2 with the permissions of version 1 to simulate an auto-update so you can test your app under different permission scenarios.

What is being considered ?

Separate Deployment from Release - We will explore ways of giving developers more control over rollout timing

Permission feed per installation - We are exploring options for enabling app developers to track installations that have/haven’t adopted the scopes and permissions

Fallback system error - We are considering options for surfacing an error message when something fails due to permissions

Scope changes that might not warrant admin consent - We are exploring pathways for scope changes that may not require admin consent, such as storage, app-data scopes, and other equivalent scope changes that may be platform-driven.

What Is Coming Next?

• Ongoing Engagement: We will continue to engage with the community through Community posts, test our concepts with customers and will provide updates and gather further feedback.

  Admin awareness of unapproved permissions: We acknowledge the need to improve how we deliver app update notifications to admins and streamline release notes. We are exploring valuable improvements for future milestones.

• Optional features: as outlined above

Thank you for your valuable feedback and for helping us improve the scope for the initial milestone. We look forward to your continued engagement and support throughout each of the upcoming milestones!

Angelina