We need to talk about Runs on Atlassian

[remie]

Dear Atlassian,

We need to talk about Runs on Atlassian.

When the program was first introduced, the definition was:

we are introducing a new badge called Runs on Atlassian that identifies Forge apps with no data egress and data residency-enabled storage.

When the program launched, the definition became:

All Runs on Atlassian apps are built on Forge, but not all Forge apps qualify for Runs on Atlassian. In order to get the badge, an app must:

• Only use Atlassian-hosted compute and storage
• Provide data residency in the same regions as Atlassian apps
• Allow customers to control data egress and block it at any time, if desired

This was a result of the implementation of custom analytics, which required egress. This was the first time Atlassian implemented the notion of controlled egress: customers would be able to enable/disable analytics, making the feature optional.

To make sure that this first loophole of data egress in the Runs on Atlassian program does not cause data leakage, Atlassian planned to create a comprehensive definition of allowed analytics tools:

We will be offering developers a method to flag their egress permissions directly in the app manifest. Atlassian plans to publish comprehensive guidelines outlining what qualifies as an analytics tool and what does not.

However, despite multiple requests and promises, Atlassian has not provided this definition.

So the Runs on Atlassian program has gone from no data egress to controlled data egress between announcement and release based on requests from Marketplace Partners and customers.

How controlled egress is already challenging the program requirements
By changing the requirements from no data egress to controlled data egress, Atlassian has opened the door for questionable architectural choices, both by Atlassian as well as partners.

With RFC-94 and RFC-97 the interpretation of the requirements of the Runs on Atlassian program are already challenged:

• RFC-94 introduces configurable egress and remotes, which is excluded from Runs on Atlassian even though it falls under controlled egress as they are customer defined and can be enabled/disabled.
• RFC-97 introduces Forge REST API capability and is included in Runs on Atlassian because it is controlled egress as the customer can enable/disable it.

Both RFCs introduce controlled egress but are treated differently with regard to Runs on Atlassian. Both RFCs introduce features that might end up being non-optional, meaning that controlled egress will become meaningless as it will basically mean customers either enable egress or cannot use the app. Both RFCs are diluting the meaning of Runs on Atlassian.

Atlassian, please go back to the drawing board
In the Runs on Atlassian launch day announcement, Atlassian made it really clear that not all apps qualify for the program

Because of the way Runs on Atlassian apps restrict access to external services, not every app and use case can qualify.

And that is fine. Runs on Atlassian is a trust signal, and in order for it to work, the requirements have to be crystal clear.

I urge Atlassian to take a step back and have a good look at the Runs on Atlassian requirements, as the current controlled data egress paradigm has proven to be to difficult to comprehend for Atlassian teams.

Personally, I would be fine with Runs on Atlassian going back to no data egress. That way, all parties involved have a clear understanding of what the program means. As Atlassian has put it herself not every app and use case can qualify. And that is OK.

[remie]

N.B. I’m currently running a LinkedIn poll on what my network expected from Runs on Atlassian with regard to data egress.

As my network consist of mostly Atlassian Marketplace Partners and Atlassian Solution Partners, these are Forge power users with in-depth and up-to-date knowledge of Atlassian Forge and the Atlassian programs.

It should come to no suprise that ~50% of all respondents have indicated that Runs on Atlassian should enforce no data egress. I strongly urge Atlassian to take this into consideration (and perhaps do their own research as well).

Now that we've seen al the Atlassian marketing messages about the Runs on...

The poll is still running, I will update this message once the poll has closed with the final results.

The poll has closed:

[PaoloCampanelli]

I’d be in favour of a more relaxed approach: an app that entirely runs on Atlassian with no egress whatsoever is most likely going to be small and will just address a missing feature in the host platform, arguably providing not a lot of value and equating “Runs on Atlassian” with “Doesn’t do much”.

Most large-ish apps will have options/features that ingest/send data from/to the outside world. Any app that can work without data ingress/egress, albeit in limited capacity, should qualify for RoA and ship with egress disabled by default, giving customers the option to open the floodgates in whatever way they deem acceptable.

In my opinion, the real advantage of RoA is smoother compliance talks with large customers, therefore it should be achievable for enterprise apps, otherwise we’re back at square one and each partner will have to take care of all the compliance requirements, fill in questionnaires… boring stuff.

Let me use a non-Appfire example so that I don’t seem biased: take ScriptRunner, it… runs scripts. As long as Adaptavist can make SR run on Atlassian servers, there is no need for those scripts to access external resources, and on its own it won’t fetch data from third parties.

That sounds like a reasonable definition for “Runs on Atlassian”: is it running? I hope so.
Is it entirely running on Atlassian? For sure.
Is it a fully functional app? Betcha.

Now a customer goes to our friends at Adaptavist and ask “hey, do you mind if I can fetch data from my ERP using a REST call?”. What do you do?
Do you say “Heck no! That would violate the sanctity and trust that RoA bestows upon our app” and get a bunch of angry customers who can’t do something extremely reasonable?
Do you say “Sure, bud!” and get rid of your shiny RoA badge? You’ll then have a harder time getting new customers.

As partners, we shouldn’t have to make such a choice.

Of course, an app that inherently requires 3rd party egress should not qualify for RoA: “Oracle DB connector for Confluence” (what a cursed concept) or “Send stuff to Slack” would never be RoA since they effectively are useless if the customer disables all egress, but that’s different from apps that get progressive enhancement with network access.

[ernst.stefan]

I guess it comes down to definitions. I would personally say I would consider a Slack integration as Runs On Atlassian as long as it only connects to the Slack APIs.

But besides that point, it is currently challenging to design an app with no data egress as we have missing features like for example the ability to manage binary data within Atlassian. So it is more of an aspirational badge.

Additionally, the point of no data egress is a bit absurd given the architecture of the Atlassian platform itself and how many third party data processors are involved.

[david]

I’m feeling that Forge without Connect on Forge is “Runs on Atlassian”, so everyone will get the badge at some point in the future.

[JamesDumay]

Hi Remie,

Our guiding principle is to meet customer needs, and we expect all our products to evolve based on customer feedback.

Before Runs On Atlassian, customers moving to Cloud were often concerned about where app data was processed and stored. These customers had already conducted thorough assessments of Atlassian’s trust posture, and applying the same level of scrutiny to each Marketplace app was both complex and burdensome for customers and partners alike. This challenge is even more pronounced with Connect and Forge Remote apps, where there’s uncertainty around how and where user-generated content is stored, processed, and managed.

We’re hearing from customers that Runs On Atlassian provides the assurance they need - that compute and storage happen on our platform, with the data lifecycle and egress controls they require, without the need for costly individual evaluations.

This is the core value of Runs On Atlassian: workloads and storage are handled on Atlassian’s platform, not the partner’s. To ensure this, we’ve disallowed all egress in the initial release and have intentionally left room in the positioning that some forms of egress may be considered in the future if customer needs demand it but only if the customer is in control. If the focus were solely on egress, we would have simply called it “No egress apps.”

We’re currently conducting a study to better understand how enterprise customers feel about optional egress and REST APIs for Runs On Atlassian apps. We’ll share our findings with partners, along with any implications for the program and the necessary controls and rules to make such changes feasible, if we choose to adjust it. If we change the requirements, it would be because we’ve ensured the technical solution raises the control bar for customers and prevents misrepresentation from bad actors.

We recognise that these RFCs haven’t fully captured our unified thinking they’re intended as starting points for discussion, not as policy or program changes.

We truly appreciate your feedback and will consider it, along with customer research, before making any adjustments to Runs On Atlassian.

Thanks,
James Dumay
Group Product Manager

[tbinna]

As Paolo said, I am also in favor of a more relaxed approach. However, I do not think there should be a distinction between apps that optionally or progressively enable egress and those that inherently have egress. Otherwise, I would seriously consider building my app with inherent egress as an app with optional/progressively enabled egress, simply because then it can be Runs on Atlassian. I do not see why this would not be allowed. The concept is just forcing me to work around it at the cost of usability.

We need to talk about Runs on Atlassian

I’m feeling that Forge without Connect on Forge is “Runs on Atlassian”, so everyone will get the badge at some point in the future.

This, and I would add that also without Forge Remote.

All “pure” Forge apps should be Runs on Atlassian - that’s the point of Forge (for me at least). I think it’s important to draw a simple, clear line. This benefits both customers and partners. If I had to explain Runs on Atlassian to a customer today, I would say it’s an evolving concept. But it does not have to be. If you exclude Connect on Forge and Forge Remote, Atlassian has full control over egress, storage, and compute. I think it should become clear from these discussions that app egress, if not at all, progressively or inherently, should all qualify for Runs on Atlassian. The point is that, for all of the above, Atlassian remains in control if an app is a “pure” Forge.

Please, please keep it simple (or tell me where I am going wrong with this).

[RaimundasJuska]

@JamesDumay

that compute and storage happen on our platform

I think Atlassian is taking the phrase “compute and storage on our platform” too literally and missing what customers really need. Customers don’t actually care which servers do the computing - what matters is that their data never leaves Atlassian’s infrastructure. They just want a clear guarantee that their information stays completely within Atlassian’s systems (no-egress).

[remie]

Hi James,

Thank you for your quick and considered response.

I do have a few questions.
Please keep in mind that this discussion is solely about Runs on Atlassian as a trust signal, and what it communicates to customers. This is best illustrated with this paragraph:

@JamesDumay

We’re hearing from customers that Runs On Atlassian provides the assurance they need - that compute and storage happen on our platform, with the data lifecycle and egress controls they require, without the need for costly individual evaluations.

So whenever a customer sees the Runs on Atlassian badge, they expect that the app will not require an individual evaluation.

I expect that the company will still require to review the individual ToS / EULA for the app, right? This will continue to be a requirement for Forge apps, even those with Runs on Atlassian.

Now, as soon as optional data egress comes in the picture, whether this is analytics / logs, custom forge remotes or Forge API endpoints, doesn’t this mean that by definition the customer will have to review the privacy & security policy for the vendor? The only difference now is that A) it is disable by default (is it?) and B) that they can choose to do the individual evaluation.

But isn’t that where it becomes odd?

From my 13+ years of experience, Risk & Compliance (R&C) is usually part of the procurement process. If optional data egress is disabled by default, and R&C departments rely on the Runs on Atlassian badge for evaluating app purchase, they might get the idea that they do not need to evaluate the ToC, privacy & security policies of the vendor because there will be no data egress.

However, R&C is not the department that will be using the app. Business users are. And they can then, once purchase has completed, enable any type of data egress that they seem fit for their solution.

Of course, we can argue that this is the responsibility of the customer, but we all know that shadow IT is rampant within organisations.

We, Atlassian and Partners alike, can all cleanse our hands and declare innocence, but in the end we are creating an illusion of privacy & security with Runs on Atlassian if we allow an app to pass R&C on the promise of privacy by default and then give the keys of the kingdom to business users that just need their problem to be solved.

Can you please elaborate on how Atlassian plans on addressing this problem?

[JamesDumay]

@RaimundasJuska

I think Atlassian is taking the phrase “compute and storage on our platform” too literally and missing what customers really need. Customers don’t actually care which servers do the computing - what matters is that their data never leaves Atlassian’s infrastructure. They just want a clear guarantee that their information stays completely within Atlassian’s systems (no-egress).

I am intentionally using that language to give feedback from our joint customers to you, our partner audience: Enterprise customers have made it clear to Atlassian that they do not want partners to act as custodians of or handle their data when customers use your apps. It is not necessarily that egress that is bad, its having no choice but egress.

I realise that this for many partners is a tough pill to swallow and I do not wish to sugar coat it. This means the model of Connect apps or Forge Remote apps that must use partner infrastructure to store or process their data, is fundamentally misaligned with customers needs and expectations on this platform. There are exceptions here, but expect that these exceptions will become rarer and rarer as time passes.

There may be circumstances where, for example, they accept this risk because your app is very valuable and they are willing to work through the evaluation process. Integrations and highly popular apps tend to be good examples of this cohort. However, we are seeing customers migrating from DC where they have 20 or so apps, and come to cloud with one, two or zero apps for the reasons stated.

[JamesDumay]

@remie

So whenever a customer sees the Runs on Atlassian badge, they expect that the app will not require an individual evaluation.

I expect that the company will still require to review the individual ToS / EULA for the app, right? This will continue to be a requirement for Forge apps, even those with Runs on Atlassian.

We don’t tell customers they can skip evaluating an app if it is Runs On Atlassian. Instead, we explain that it helps make the evaluation process less costly and time-consuming. Reviewing Terms of Service and EULAs is usually straightforward. However, the most expensive and complex part for customers is understanding how and where their data is stored, processed, and which subprocessors are involved. This is where most of the friction and cost in the evaluation process comes from. (Now times this friction by 20 apps if you are migrating from DC…)

@remie

From my 13+ years of experience, Risk & Compliance (R&C) is usually part of the procurement process. If optional data egress is disabled by default, and R&C departments rely on the Runs on Atlassian badge for evaluating app purchase, they might get the idea that they do not need to evaluate the ToC, privacy & security policies of the vendor because there will be no data egress.

Thank you for sharing this perspective - it’s a concern we share as well. We also have specialists on our team who are contributing their extensive knowledge and experience to this discussion and are key stakeholders in any decisions we make for this program.

@remie

We, Atlassian and Partners alike, can all cleanse our hands and declare innocence, but in the end we are creating an illusion of privacy & security with Runs on Atlassian if we allow an app to pass R&C on the promise of privacy by default and then give the keys of the kingdom to business users that just need their problem to be solved.

Can you please elaborate on how Atlassian plans on addressing this problem?

That’s why we are being very deliberate and thoughtful in our decisions about this trust signal. We’ve taken a strong initial stance on the requirements and I think our initial goals are being met. However, we are actively listening to our customers through our research to ensure we do not disrupt their existing mental model of trust associated with this signal.

If I had a definitive answer today, I’d be at PAC this week sharing back the policy change proposal rather than discussing it here on CDAC. I’m looking forward to seeing the results!

Have a great evening,
James Dumay
Group Product Manager

[JamesDumay]

@david

I’m feeling that Forge without Connect on Forge is “Runs on Atlassian”, so everyone will get the badge at some point in the future.

I am confused by this statement. Can you expand for me?

[RaimundasJuska]

@JamesDumay

handle their data when customers use your apps

Isn’t that exactly what happens once the Forge REST API is introduced? You just forward Jira’s REST API, pull the data externally, and then it’s business as usual - just with a shinier label.

[JamesDumay]

@ernst.stefan

I guess it comes down to definitions. I would personally say I would consider a Slack integration as Runs On Atlassian as long as it only connects to the Slack APIs.

I’m definitely hearing similar thoughts from customers in regards to the app trust discussion & research. Some are comfortable with Atlassian and Slack processing and storing their data, provided the app developer doesn’t have access to it and Slack has already been vetted by the customer.

[Nar_ChtN]

I agree with what Paolo said above. I think allowing optional egress (controlled egress) is the middle ground and should be the way forward.

The current RoA definations are way too strict to build useful apps and does not cover complex use-cases. As such, only small, self contained utility apps will qualify.

For example, there is no way to connect and fetch data from customer’s own domain. I think customers should be given a choice and we should look into improving audit and visibility of the egress instead.

[tbinna]

Hey James,

I think what David means is what I highlighted in my post above. If a Forge app does not have Connect on Forge or Forge Remote functionality, i.e., it’s a “pure” Forge app, then all “pure” Forge apps should qualify for Runs on Atlassian and receive the badge.

We need to talk about Runs on Atlassian

As Paolo said, I am also in favor of a more relaxed approach. However, I do not think there should be a distinction between apps that optionally or progressively enable egress and those that inherently have egress. Otherwise, I would seriously consider building my app with inherent egress as an app with optional/progressively enabled egress, simply because then it can be Runs on Atlassian. I do not see why this would not be allowed. The concept is just forcing me to work around it at the cost of…

[Stavros_EasyApps]

@remie again brings up good points. I want to take on the customer point of view.

‘Controlled data egress’ ‘no data egress’. Well which one is it? If ‘controlled data egress’ without more details I’m left wondering if I have to bring in the top guns in IT to figure it out which is a pain - more work, more delays.

Part of the power of a badge is that is it easy to understand. This is an issue with Cloud Fortified it’s not clear what it is. Think of it like great UI, it doesn’t require a lot of explanation.

Less can be more.

[remie]

I would like to invite someone from Atlassian legal to join this discussion, as it seems that the current implementation of custom analytics is in direct violation of GDPR Article 25.

When a Forge app enables Egress Permissions with category analytics, the in-scope End-User Data inScopeEUD option is enabled by default.

The analytics category for egress permission allows the use of indiscriminate data egress to any external remote. This can include end-user data (incl. PII) by default.

Although it can be disabled through the Connect Apps admin interface, Atlassian enables analytics egress by default for all Forge apps.

The combination of unfiltered data egress of to an external remote for which the inclusion end-user data (incl. PII) is enabled by default in a feature that is also enabled by default is in direct violation of Article 25 GDPR, section 2:

The controller shall implement appropriate technical and organisational measures for ensuring that, by default, only personal data which are necessary for each specific purpose of the processing are processed. That obligation applies to the amount of personal data collected, the extent of their processing, the period of their storage and their accessibility. 3In particular, such measures shall ensure that by default personal data are not made accessible without the individual’s intervention to an indefinite number of natural persons.

Given the complex legal nature of the Atlassian <> Customer <> Partner love triangle, Atlassian is possibly putting herself or the partner at risk by not implementing privacy by default on behalf of the partner.

This is especially damning in light of Runs on Atlassian, which creates the illusion for partners and customers alike that Atlassian enforces data egress controls to ensure privacy and security by default. Even though inScopeEUD must be disabled for Runs on Atlassian eligibility, there is still a possibility that end-user data (incl. PII) is transmitted in custom payloads.

This is no longer a hypothetical discussion. The current implementations of custom analytics and the implied privacy/security posture of the Runs on Atlassian program violates EU law and puts customers and partners at risk.

[danielwester]

@remie I think you’ll probably get a better response from Atlassian if you file an ecohelp ticket and handle your concerns that way rather than going the semi-bullying approach.

[remie]

I’ll leave that to others in the ecosystem
I’m sorry to hear that you consider this semi-bullying, given the power dynamics and the fact that Atlassian is a multi-billion dollar company I respectfully disagree with that qualification.