Connect apps that are using various Connect modules will require READ scope

On 5 Jul 2022, various Connect modules being used by apps that are not using READ scope will be blocked from accessing product data. Please see the changelog for more details on which modules are affected.

EDIT:
As there might have been some confusion with the original post, we have updated the changelog to help clarify the deprecation notice. Here are the key items in the updated changelog:

  • Existing apps which use these modules with no READ scope will continue to work and will not be impacted till 5 Jan 2023.
  • From 5 Jul 2022, newly listed Connect apps that are using the various Connect modules will be required to update and use the READ scope.

Hope this helps clarify the deprecation notice and apologies for any confusion caused.

Seriously, stop with the “we will be breaking your app unless you drop everything and make this change” type of announcements for security improvements. A 2 week notice for a breaking change is not acceptable, especially not nearing holiday season.

Unless there is an active exploit with a CVE number and customer communications that shit has hit the fan, it can wait at least a month.

EDIT
Disclosure: we don’t have any apps affected, this is just a remark in general that I think that the matrix you use to determine the remediation window seems to lack any consideration for the impact on both vendors & customers. By the latter I don’t mean impact with regard to security, but with regard to the fact that they will experience outages due to vendors not being able to meet the deadline.

@AnthonyNguyen @JakeComito @bentley

28 Likes

Totally agreed with this sentiment, either there are no apps out there that don’t use the read scope - in which case this announcement is probably unnecessary - or the change window is unacceptably short.

5 Likes

We don’t have an affected app but I’m just going to echo what everyone else has said - really close change window (if you can it a window - perhaps more like a gap) during vacation season.

Why am I saying something when I don’t have an app affected? Next week it could be - so I really would love for atlassian to start respecting their change management policies.

Has anyone at atlassian reached out to the affected partners? When that is done - which contact is contacted and by who? Would love to get an understanding how atlassian performs these notices.

/Daniel

10 Likes

New apps installed that don’t use READ scope will be blocked from accessing product data such as issue or page parameters.

Is there a complete list of what will be blocked to take the guesswork out of this? Including on Confluence.

9 Likes

While I do echo what the other vendors have said, I would like to, again, also point out the insufficiency of Atlassian’s messaging towards admins when an update that requires manual intervention is available.

On one of our apps, we had rolled out this exact scope change (add READ scope where the app used to require NONE) last year in anticipation of making the app paid. Based on the roughly three months between the READ update and the paid update, it would take very long for all admins to approve the update.

10 Likes

Seems to violate https://developer.atlassian.com/platform/marketplace/atlassian-rest-api-policy/#time-frames

3 Likes

This is a huge problem for us. We’ve even started to look at providing our own in-app messaging, which if each vendor does this, causes a very fragmented user experience.

6 Likes

Hi @TonyGoughAdaptavist I believe the linked changelog has the detail you’re looking for (including Confluence). Please let me know if not.

Hi folks!

First up, I have some good news. The deprecation period is actually six months for existing apps.

Unfortunately we buried the lede here, and this critical detail is hidden in an expand in the changelog entry. The wording in both the CDAC post and the changelog entry are not great, both in terms of the detail and empathy for our partners.

So to clarify, enforcement of requiring the READ scope for existing apps will not occur until January 5th, 2023. “Existing apps” here being defined as any app that was published on Marketplace on or before June 27th (the day the changelog entry was published).

We’ll get the formal comms updated to make sure this is more clearly spelt out.

My sincerest apologies for the confusing and scary-looking messaging here. We’re well aware of the pain platform change management and deprecations are currently causing partners. In the coming months we are planning to work on improving not just our communications, but the underlying processes behind how deprecations are carried out to make them more predictable, transparent, and empathetic to partners.

Also, thank you for all your feedback on the above! It’s great to see partners looking out for each other and highlighting concerns around impact to apps even when their own offerings are not impacted :heart:

To address some of your specific concerns:

Unless there is an active exploit with a CVE number and customer communications that shit has hit the fan, it can wait at least a month…the matrix you use to determine the remediation window seems to lack any consideration for the impact on both vendors & customers. By the latter I don’t mean impact with regard to security, but with regard to the fact that they will experience outages due to vendors not being able to meet the deadline.

@remie First up—I fully agree. We absolutely need to better define how deprecation windows are determined and make this expectation transparent to partners. To shed a bit more light on why the process today is what it is: security is something Atlassian takes very seriously as a pillar of customer trust, and our security teams currently do not assess likelihood when raising vulnerabilities against internal Atlassian engineering teams. If a vulnerability has a remote chance of being exploited, it must be fixed within a tight SLA, no questions asked, without any consideration for roadmap impact. Atlassian has the resources to bake this into our engineering budgets. However it’s pretty clear that this same logic does not map well on to ecosystem, and it is unreasonable for partners to down tools and fix things that realistically present little or no actual risk to our mutual customers. Hence the varied enforcement in this deprecation for new apps versus existing apps — while the fact that the READ scope not being applied to the context being passed to certain extension points is clearly a gap in our security controls, it is vanishingly unlikely that existing partners are abusing this in a way that violates customer trust.

Totally agreed with this sentiment, either there are no apps out there that don’t use the read scope - in which case this announcement is probably unnecessary - or the change window is unacceptably short.

@ernst.stefan Our analysis does in fact show very limited impact on existing apps, but even in the case where there is (in theory) zero impact, we’re still aiming to standardise change messaging and deprecation processes in case our analysis is flawed. That said, I agree that the way this particular announcement was conveyed does unnecessarily cause concern and uncertainty, and we’ll evaluate how to improve future comms to make it a bit more obvious as to which changes may impact your existing apps.

Has anyone at atlassian reached out to the affected partners? When that is done - which contact is contacted and by who? Would love to get an understanding how atlassian performs these notices.

Is there a complete list of what will be blocked to take the guesswork out of this? Including on Confluence.

@danielwester @tobitheo though we have performed an impact analysis, but I can’t really publicly confirm whether apps are impacted or which apps are specifically impacted. That said, directly contacting impacted marketplace partners is something we do in certain circumstances relating to incidents and change management, and we’re evaluating how we make this a more consistent and reliable process in the future. Sorry for the vagaries here! Enumerating which apps may or may not be impacted by a vulnerability, even a low risk one, is not a great practice so we can’t really get specific on a public forum.

While I do echo what the other vendors have said, I would like to, again, also point out the insufficiency of Atlassian’s messaging towards admins when an update that requires manual intervention is available.

On one of our apps, we had rolled out this exact scope change (add READ scope where the app used to require NONE) last year in anticipation of making the app paid. Based on the roughly three months between the READ update and the paid update, it would take very long for all admins to approve the update.

This is a huge problem for us. We’ve even started to look at providing our own in-app messaging, which if each vendor does this, causes a very fragmented user experience.

@tobitheo @jeff great points, and the internal discussion around this deprecation has put some renewed focus on this issue. When determining the appropriate deprecation period for this change apps, we did a short analysis of scope changes (and more generally major version upgrades) and how customers responded to them. The results were mixed and require some deeper analysis, but there was enough concerning evidence to justify using the “standard” six month deprecation period (which @boris linked to) for existing apps instead of an expedited one, even though the root concern was a security vulnerability. Really the six months is a period for partners to update their apps and all of your customers to update to the new version (this fact should also have been highlighted in the comms).

@jeff we’ve raised a request with platform team to investigate how we might facilitate upgrades better at the platform level, but I’m afraid we don’t have a firm timeline on any improvements yet. I’m sure customer upgrade lag has implications for you all beyond security—revenue, app maintenance costs, user experience, etc—so I’d love to explore this topic in more detail with you both. Do you or @tobitheo (or anyone else reading this :slight_smile:) have any data points, anecdotes, or past posts you’ve shared on this topic that I could read up on? Or if you’d be up for chatting directly, I’d be keen to set up a zoom (feel free to DM me if you like!)

cheers,
Tim Pettersen
Head of Developer Experience

15 Likes

Thank you for taking the time to write such a considerate response and address all our concerns. I know Atlassian is working hard on improving communications with regard to changes.

I do wish to re-iterate that for me, this is another example of how the change logs are a bad way of announcing deprecation. If this wouldn’t have been posted on CDAC, we would not have been able to have this discussion. In addition, I think the change log format encourages taking communications shortcuts and hiding crucial information in info boxes. The courtesy CDAC post is a quick hit-and-run copy/paste job from the change logs instead of a more elaborate announcement that the CDAC format actually supports. I know that in the future, the CDAC posts will stop, and I’m wary that this will only mean that there will be more change logs type of announcements that will miss a lot of information & context of the changes.

It would really mean the world to me, and I expect other partners as well, if Atlassian would take a step back and re-evaluate the new content & communications strategy based on the recent examples (like the current post, but also this one) :pray:

4 Likes

I totally support what @remie said above. At the current stage, moving to a solution that does not allow a dialog or Q&A seems to be a bad idea. Announcements currently have no clear structure, are often minimal, and seem to lack sufficient consideration of the ecosystem.

Example 1 which even has a sequel.
Example 2

2 Likes

I agree that this is a pretty bad example of communicating a change, particularly since we made the deprecation seem significantly worse than the reality is and caused unnecessary FUD. As you noted we are taking steps to improve the communications process, and understanding partner expectations is a key part of that process. The changelog serves no purpose other than informing you all of upcoming changes so we’re keen to make sure it works for you!

If this wouldn’t have been posted on CDAC, we would not have been able to have this discussion.

I’d love to understand a bit more about your concern here as part of the idea behind standardising change management comms on the changelog is to ensure partners have a single channel / source of truth for partners to review changes. However I do share your concern that it may inhibit dialog. Are you concerned that partners may miss changes entirely because they’re not subscribed to the changelog? Or that the friction of raising a topic on CDAC concerning a changelog entry would mean that it doesn’t get raised at all? Or something else entirely?

In any event, I’d love to make sure we do have a way to continue to facilitate that dialog, as we harvest a ton of valuable insights from yourself and other ecosystem participants on threads like this. For example: a silver lining of the poor comms that started this thread is that we now have some great concrete examples of where stale app versions are causing partner pain that we can take back to the platform team to help prioritise.

I think the change log format encourages taking communications shortcuts and hiding crucial information in info boxes.

In this case I agree that burying the fact that this only impacts new apps in an expand was suboptimal, to say the least. But the changelog is a bespoke tool that we’ve built that can be updated fairly trivially. Once we’ve aligned on a better process and format for communicating changes we should be able to make the tool meet any formatting requirements.

It would really mean the world to me, and I expect other partners as well, if Atlassian would take a step back and re-evaluate the new content & communications strategy based on the recent examples.

Absolutely, thank you for the pointers and all your feedback to date!

1 Like

I fully agree that we need some mechanism to be able to discuss changes, though I’m less sure that cross-posting all changes to CDAC is that mechanism. Not to jump to a solution (but I’ll propose it anyway so you can tell me if it’s terrible :smiley: ) but perhaps we could have a “Discuss on the Developer Community” button on the Changelog so that we keep a single source of truth for changes but partners also have the opportunity to easily open a dialog.

For the specific threads you linked above, I actually think there’s a broader problem that is less about comms and more about missed requirements. While catching these issues on CDAC threads is better than nothing, we’d ideally be catching them much earlier during the requirements gathering phases of these projects or during an Early Access Program, when the cost and impact of change is less painful for everyone. Outside of just comms, we’re also starting to think through how we improve our release processes (e.g. more consistent EAP and Developer Preview phases) and gather feedback from partners earlier, so we’re not discovering issues whilst a deprecation clock is ticking or after a problematic change has already started to roll out to production. I don’t have firm news to share on this front yet, but we are very aware of the problem and beginning to explore solutions in this space.

2 Likes

Thank you for your reply, Tim, and for taking part in this discussion. I do agree with your point that there is a broader problem than just the comms. For partners, I guess it ultimately manifests itself with the announcements.

To me, this sounds like a workable solution.

2 Likes

This touches the subject of the proposed changes to content & communications and more specifically, my concerns with the fact that I consider announcements a separate type of communications that require more attention.

It makes a lot of sense to post content to the channel that is best suited, whether that is a developer blog, change log, partner portal, newsletter or here on CDAC. However, if that content does not come with a separate announcement on an aggregation platform, this means that partners will have to listen to a lot of different channels and filter out a lot of noise to stay on top of all things Atlassian.

It moves the burden of creating an aggregated platform for all things Atlassian on each individual vendor, automating collection of a bunch of RSS feeds, of which there is also no single centralised place to find them. I would not want to start as a new Marketplace Partner and having to figure out where to find all the required channels to get my information.

It also does not safeguard us against misclassification of content & comms. Mistakes will always happen. If partners are not listening to the right channels (perhaps because they don’t know that channel exists), they might miss an important announcement because of it.

For what it’s worth, I’m not pleading to keep posting announcements on CDAC. I don’t really care about which platform you choose. What I’m asking for is for Atlassian to make the investment to create a single source of truth for product & platform announcements. This does not have to be a technical solution: perhaps it might be as simple as adding an FTE to your marketing team to do that filtering & aggregation of posts from all the different platforms and cross-post them to a single space.

My only goal is to make sure that we do not end up with a situation in which the burden of finding relevant information is placed on each individual Atlassian Partner.

1 Like

Gotcha, thanks for expanding!

Hypothetically, how would you feel about an aggregated changelog, similar in nature to the Forge changelog, but containing all Atlassian cloud product and platform changes? Assuming for now that we improve the content and formatting as per your above comment :slight_smile:

EDIT: To be a bit more realistic, let’s say “all Atlassian cloud product and platform changes to APIs or otherwise anticipated to have an impact on apps or partners” — a changelog of all changes shipped to the Atlassian cloud is a very different matter.

Also to set expectations this doesn’t actually exist yet and a forum thread isn’t really the best place for solutioning, but I’m finding it a very useful exercise to understand your concerns and desires a bit better if you’ll indulge me!

1 Like

Hypothetically, how would you feel about an aggregated changelog, similar in nature to the Forge changelog, but containing all Atlassian cloud product and platform changes?

Yeah, that would work as long as you make sure that the bespoke solution & post guidelines are tweaked to support long-format to at least contain a description and link(s) to relevant content, change logs & documentation.

As an example, how would any of these posts be delivered in a world of separated content & announcement?

To be a bit more realistic, let’s say “all Atlassian cloud product and platform changes to APIs or otherwise anticipated to have an impact on apps or partners”

Yes, that makes sense.

EDIT:
One example of an announcement going wrong is this Introduing Content State API. There are three separate posts with three different cross-references, none of them linking directly to each other. That would still require listening to multiple channels to get the full picture.

1 Like

Eeek! That’s quite scary for somebody like me who unlike Remie - I doesn’t want the extra noise of Confluence, Bitbucket platform changes. If I start having to filtering things out - I’m back to the mess that is CDAC today in regards to announcements:

  • Too much filtering required
  • Quality and the structure of announcements is all over the place
  • Discussions on announcements aren’t necessarily about the announcements which causes the announcements to become zombie announcements (An example of this would be if I was posting about changelings on a thread about scope changes)

It would be neat if the changelogs was backed by some type of database that could track changes through their lifecycle, where users can query for the change announcements and such that they interested in…

2 Likes

As an example, how would any of these posts be delivered in a world of separated content & announcement?

It’s a great question, and one relevant to the existing changelogs as well as a hypothetical aggregated one. Those posts are the beginnings of a great reference set to illustrate the variety (and disparity) in types and styles of communication deployed today. I don’t have a ready answer for you, but let me bring this to the task force working on comms to see how they’re planning on back-testing any new processes on prior announcements.

You mentioned a Server EAP in your list as well. Server product versions, versioned libraries like Atlaskit, or tools like the Forge CLI are interesting cases where the version number is the reference point for availability rather than the date. For the Forge CLI changes are handled a bit clunkily by putting the version into the title of the change and then having separate entries for the different change types (e.g. https://developer.atlassian.com/platform/forge/changelog/#CHANGE-234 and the other CLI changes on that day). There’s probably a better pattern we can apply there.

I mentioned changes in cloud specifically in my original reply, but obviously many partners have offerings in both server and cloud, and some libraries could theoretically impact both, so it does seem reasonable to have a single aggregated view for changes across everything to avoid partners having to subscribe in multiple places… though I think we’d certainly need to allow you to easily opt in or out of what you subscribe to.

1 Like