"How To Break Apps & Alienate Partners", coming to Jira instances near you!

14 Likes

In case there is any doubt as to what I’m trying to imply with the title: yes, Atlassian, you should have updated your partners about this change prior to making an announcement on CAC, or at least do a developer focussed announcement at the same time.

This is a monumental change to apps and Atlassian Marketplace Partners given that we tend to adhere to Jira permission models and will have to make changes to our back-ends to ensure these new permissions and their intentions are supported. Failing to do so might even result in bug bounty reports as we might accidentally expose data by not adhering to these new permissions.

Shame on you Atlassian.

18 Likes

Agreed. I find it incomprehensible that a change like this is thrust upon app vendors and customers with no warning or even time for vendors to validate that their apps still work. The potential to break customer’s workflows (not talking Jira workflow here) is huge. I’d suggest this change flies in the face of at least two of the Atlassian company values.

Flabbergasted

13 Likes

Totally agree with Remie here.

I guess as long as things as announced on a community site - then it’s fine. Guess it’s time to start monitoring Stackoverflow as well for Atlassian changes as well

This is a whole new level which will just cause confusion and cause issues left and right - both security wise and support wise.

Really poor form.

9 Likes

If you’re using project entity properties - I’m assuming this will affect you.

1 Like

@danielwester @remie @jmort — I agree this is an unfair and unrealistic notice period.

We’re treating this as an incident, and have paged the team involved and made them aware of the imminent impact to ecosystem apps and our mutual customers. They have confirmed that the feature has not yet started to roll out and we are currently discussing next steps.

I’ll share further updates as soon as I have them.

EDIT: To clarify, we’re treating this as an incident due to imminent impact to partners, apps, and customers if/when this feature roles out. Since it hasn’t started to roll out yet you won’t see communications on our Statuspages about it, but we’ll keep you updated here.

13 Likes

So this is like a pre-incident and in future we should raise a minority report?

8 Likes

Hi @tpettersen ,

Thank you for engaging!

There are specific, simple ideas for making the Atlassian platform great regardless of whether it is Forge, Connect or P2. They are nearly identical to what I wrote here over two years ago. That earlier post is equally descriptive anytime from seven years ago to today.

One reason I’m pointing out that post is that one of the 24 likes was from Atlassian cofounder Scott Farquhar. I see that as approval for engaging in said improvements! Hopefully, the engineering teams will read it as such. :slight_smile:

The ecosystem vendors all want Atlassian’s platforms to be wildly successful for both Atlassian and 3rd party developers. There are simple and effective steps Atlassian can take for that to happen.

Atlassian once took the right steps for their P2 platform circa 2012. That was when Atlassian dogfooded their platform (as many other companies now do). Atlassian still rides that momentum.

The type of messaging from Atlassian prompting this post is ongoing and has shown no change in trajectory for about seven years.

What can vendors do to help Atlassian succeed as a world-class ecosystem that other companies and their vendors look up to?

What can we do to help you create the platform that gets developers excited (as it once was for P2)?

What can we do to help Atlassian get excited to use their platform for actual feature development as many other platforms now do?

We have the answers to these questions, but what Atlassian believes the answers to be is what matters.

Cheers!

6 Likes

Hey folks,

As promised, I have an update for you. Fortunately I have some (generally speaking) very good news!

What happened?

After following up with a few teams internally, it appears that this was largely a communications mistake, rather than an actual change rollout. The announcement on the Atlassian Community was posted mistakenly before a more detailed post notifying partners of the upcoming changes. There are more detailed communications for the Developer Community planned and in the process of being drafted. Generally speaking we endeavour to notify partners ahead of customers, particularly when impact to apps is anticipated. These partner-facing comms should have been finalised and posted well ahead of the customer announcement.

Did we consider the impact to apps?

In other positive news, the Jira Configuration Experience team did anticipate that this change would impact apps, and—even better—has taken pains to make the planned product behaviour and API changes backwards compatible. That is, apps that make authorisation checks against BROWSE_PROJECT today or otherwise work with permissions and permission schemes should continue working as this feature rolls out, without partners needing to make changes to their offering. This should be true even if users start applying the new permissions to their permission schemes (there is some sort of mapping/translation layer planned to facilitate this — more on this in a second).

There is a subsequent planned deprecation to remove the BROWSE_PROJECT permission which will involve some breaking API changes, but this will not roll out until T+6 months after formal notice is given as per our REST API deprecation policy.

Interestingly enough, it turns out that while these changes have not rolled out to customers, the new permissions and REST API changes are already live in the Jira Cloud Vendor First Release program, and have been for some time. In September last year there was a limited announcement to a closed partner EAP group (limited means some of you reading this will not have permissions to view this announcement, so don’t be distressed if that link is not accessible to you). Though the announcement was limited, the changes should be live for all partners who are enrolled in Jira Cloud Vendor First Release. Feel free to sign up here if you would like to participate in Jira Cloud Vendor First Release program going forward.

EDIT: My apologies, while the new permissions were initially rolled out to the Jira Cloud Vendor First Release program in March, it appears the feature was removed from that tenant cohort in June (Thanks @matthias for the heads up!). We’re working with the engineering team to re-enable it, alongside posting the more detailed guidance around the change itself, so that interested partners can experiment with the implementation.

Next Steps

The team are working on finalising the more detailed partner-facing guidance, which—as a bit of a silver lining to the stress this has undoubtedly caused you all—will be better informed by the feedback that you have all left here and via other channels. This should be posted on the Developer Community in the next few days.

Given the evident concern from the partner community, and the fact that the implementation detail of the backwards compatibility layer have not yet been shared with the community beyond the limited EAP mentioned above, the Jira Configuration Experience team have agreed to hold off on any roll-out to customers for FOUR WEEKS after posting the partner-facing communications on the developer community. This is intended to give partners a chance to review the planned changes, experiment with them in the Jira Cloud Vendor First Release if desired, and raise any objections. After this post has been published, if you have concerns that the approach to backwards compatibility may not work for one of your apps, please do comment on that post and we will endeavour to modify the rollout timeline to provide partners a more appropriate time period to address breaking changes.

What have we learned?

Once the smoke has cleared we will be running a formal retrospective with the teams involved and taking a long, hard look at our change management and communication processes to evaluate strategies and tactics to avoid this type of mistake happening in the future. Though we’ve managed to avoid impacting our mutual customers, it is clear that the poor job we’ve done on communications here has caused a large amount of stress for partners (and our partner-facing teams), and I would like to sincerely apologise on behalf of Atlassian for that. We are aware that this is not an isolated incident, and we are committed to improving our processes in this area.

Thank you especially to @remie for sounding the alarm, and the rest of you for your honest feedback and your continued forbearance. @BrendanPatterson thank you for sharing that post! There is a lot of great ideas there and I will follow up with more detailed thoughts once I’ve had a chance to digest them.

cheers,
Tim

14 Likes

Thank you for the fast reaction, @tpettersen!

I’ve got one question since you’ve mentioned that the rollout to the Jira Cloud Vendor First Release instances already happened. I’ve checked all our instances which are enrolled and haven’t seen these new permissions inside the permission schemes. Can you double-check if it atually has been rolled out or not?

I’m also happy to send you the list of instances in a DM if you need them.

4 Likes

Shoot, thanks for the heads up @matthias. I’ll follow up with the engineering team in to confirm that the appropriate feature flags are set for all tenants in the Jira Cloud Vendor First Release group as was previously indicated. If you’re able to share your sites that may help speed up the troubleshooting process.

2 Likes

Hi @tpettersen, thanks for that.

I understand that the partner-facing communications is still in the making, but allow me to share my main concerns on what will very likely cause problems to us:

  • Any endpoints that allow checking for permissions (such as mypermissions or permissions/check) throw a hard “400” when an unknown permission key is being queried for. This is pretty annoying in general, but could become a real blocker here: How am I supposed to know if the new permissions already exist on the customer’s instance? I’d have to make a seperate call and gracefully handle the 400, which is a performance concern and heaps of work.
  • I haven’t actually tried it, but I would expect this to cause similar problems with “has_project_permission” conditions in the Connect descriptor. Here, I have no control at all about how these checks are handled, so I can’t even gracefully ignore any errors.

The solution to the above, IMO, would be to fully roll out the backend changes first (so that the new permissions definitely exist on every site), then allow Marketplace partners to adapt, and only then roll out the frontend changes to customers.

Long-term, I would really encourage to change or introduce new permission endpoints that don’t throw hard "400"s for unknown permission keys, but that’s for another time.

8 Likes

Thanks for the early feedback @hannes-finesoftware! I’ll pass them onto the engineering team for consideration for their rollout and compatibility plan.

Yes, fully agree on your point around API errors. If we get to a Jira REST v4 at some point rationalising our error codes, messages, and approach will definitely be high on my personal hit list. I suspect it’ll all be GraphQL by then though.

1 Like

Just following up @matthias — it turns out the feature is no longer available in the Jira Cloud Vendor First Release group. I’ve updated the post above, and the team are endeavouring to re-enable it alongside the more detailed technical guidance planned to be published in the Developer Community.

Haha “pre-crime” style incidents that get caught and mitigated while the impact is still imminent (but not actual) are the best kind of incidents.

I’ll follow up with our Developer Support Manager @Andrew_Golokha to evaluate whether we can formally support imminent incident reporting via our shiny new incident reporting service desk, but in the shorter term we’ll certainly keep an eye out for posts like this.

4 Likes