RFC-94: Configurable Egress and Remotes

General Request for Comments (RFC)
21

This is a really welcome RFC, but it only makes sense if Apps maintain RoA. It is not needed at all if, by using Configurable Egress, apps would be excluded from being RoA.
Please rethink this as it will be wasted effort if not.

11 Likes
22

:100: What @jmort said.

Apps with user-configurable egress must be able to get the Runs on Atlassian badge if they are otherwise eligible.

7 Likes
23

@JamesDumay we’d use configurable egress and remotes for the specific external services that customers want to integrate our app with. For example:

All of these would be third-party destinations or other Atlassian instances. Adding new remotes would be a rare event, usually right after installation or changing the configuration by an admin.

1 Like
24

@SeanBourke Thanks the for RFC and continued focus on Forging ahead.

Some questions though:

  • What is the limit on Dynamic Remotes?
  • If you configure a Dynamic Remote, do you also need to configure the Egress for that Remote? Either for the Frontend or the Backend?
    By (automated) configuration of the egress it would be great if the popup that user are navigating away from Jira would no longer be shown.
  • If a user that is not a site-admin uses an app feature to configure a Dynamic Remote but doesn’t have himself the permissions to approve this, does a site-admin get a notification to approve the remote? Or will it simply fail?
    In my use-case both site-admins and project-admins can configure new Jenkins sites to integrate with.
    It would be doable if a project-admin can add the remote, but require a site-admin to approve it.
    It would be a blocker in the project-admin would not be able to add the remote.
  • Can an app use Dynamic Remotes without making any manifest updates?
  • Can a Remote that is already listed in the manifest add a new Dynamic Remote? (useful for migrations)
  • Can a Remote access the configuration of a dynamic remote? (useful for migrations)
  • What custom configuration can be stored with a dynamic remote? Can this include secret/secure configuration?

If the remote the customer is configuring is in full control of the customer, does that mean that Runs on Atlassian still apply to the app?
I fully understand that adding remotes, like the Github.com example, would not work with RoA, but a customer owned system on customer owned hardware/datacenter technically is not leaving the boundaries of the customer and I think would work with RoA.
Given the focus on Egress and Remotes, I understand if you want to move the RoA discussion to a separate RFC.

In my use-case I would be connecting Jenkins instances (possibly multiple) that the customer owns and operates to the app.
The app would need to access data from Jenkins, but also be able to connect to Jenkins to send data (like trigger a job, provide work item links for related items upon request from Jenkins).
All Egress actions can originate from both Jira and Jenkins in my case.
In case of Jira, it would be user defined, either by manual action or automation rules configured by the customer.
In case of Jenkins, it would be user defined, through interacting in Jenkins resulting in requesting Jira data related to the view within Jenkins.

Ideally traffic would be directed to the customer owned Jenkins instance directly, but I can see that during a migration path traffic would temporary go to a Marvelution owned system.

4 Likes
26

“Runs on Atlassian” should technically be named “Runs on AWS” because it actually is.

At some point it’s better to educate customers on how cloud-based apps function and clearly communicate data flows than to use marketing to obfuscate the realities of software in the 21st century.

There is not a single company on the planet that is solely using Atlassian products as their only SaaS stack. For their businesses to necessarily function and be interoperable with their other software tools, egress is a requirement.

The complexity of attempting to marry the reality with Atlassian’s marketing goals will continue to spiral into an unmanageable mess for both developers and end-users.

This whole egress management model is not the way to solve the problem in my opinion.

3 Likes
27

Hey @jens,

Thanks for the feedback.

At the moment we are looking to implement the functionality both from an app-led experience, as well as a Connected Apps/admin experience. Are there specific concerns about users being able to add from here?

An app could still preconfigure egress in the manner it does today, by declaring it up-front in the Forge manifest. We are exploring the possibility of an app being able to attach context to egress (similar to remotes), which would support admin understanding when reviewing egress (which you’ve noted in your comment).

1 Like
28

As part of this capability, apps could perform both behaviours - where egress is necessary for an app’s core operation, the app could declare these as standard egress within their manifest. Where egress was either required for an optional feature which could be toggled on/off, or an app loaded data from locations depending upon customer inputs, then it could leverage configurable egress to support these.

1 Like
29

Thanks for the feedback @erikmidtun.

Thanks for the feedback - that is an example of one of the use cases where we see this being beneficial for apps and customers; where you’d otherwise have to declare egress to Slack where customers may not want to use this.

There are no immediate plans to support other protocols from Forge Remote, however I’ve created FRGE-1812 so that partners and developers can share their interest in this.

Forge remotes would continue to work as they do today, utilising invokeRemote to make a request the keyed reference. In the case of dynamic remotes, it would be necessary to store a reference to the key to ensure you’re aware of the keys available which can be interacted with.

1 Like
30

The proposed solution does not support filtering types to different types of egress, however it would support the ability to request and egress type which is supported today (including *). We’ve opted to approach it this way, as declaration per type increases complexity for apps while introducing additional load for admins to review the permission types. With that said, we are open to feedback from the community on whether there is a preference on whether you’d prefer to manage these individually.

2 Likes
31

Thanks @scott.dudley,

Given that admin-approved updates are guided by a principle of elevation of privilege, we don’t currently believe this would be required (however this is subject to change). Depending upon the implementation approach, a major update may or may not be available (based upon technical constraints); however the Forge bulk-update command would enable roll-outs.

If a configurable remote defined a default, then it would inherit the behaviours of a standard remote (i.e. assessment of whether it elevates permissions, or was the previous remote). If it was entirely configurable, then admin approval would be associated with a modification of that remote via the proposed setRemote method.

Remotes only support a single endpoint - today they are interacted with in a keyed manner when invoked from a Forge event, or front-end or back-end. Appreciating that dynamic declaration of remotes can come with some overheads, do you have some examples of the averages/maximum number of integrations you see today?

This is still being explored - as noted, we are assessing the implication of this to other marketplace badges (such as Runs on Atlassian). It is likely that we’ll flag somewhere within the installation screen that the app may ask for additional remotes/egress post-installation.

As an alternative to this; would it be acceptable if existing customers updating to a new version of your app which implemented configurable egress inherited/persisted permissions that they had previously accepted, but would now also have the ability to remove them?

With regards to the Connect to Forge migration, this may require a staged migration approach, where you bring existing customers to your latest Forge version, then roll-out a version with configurable egress.

1 Like
32

Thanks for sharing this use case @StepanBogatrjov. If we support wildcarding (i.e. *) in this feature, you may be able to present the admin with an option to enable loading from any location, or loading from specific locations? Do you see a use case where this would/could exist in your configuration?

1 Like
33

While there will be a limit, we have not arrived at a specific number. Appreciating your app’s circumstance, would you be comfortable sharing an indication of the average/max number of remote integrations you have to handle?

Good question - given the current approach does not specify which type of egress is defined as part of the user approval, there is a warranted case to support both. Do you find that you have to declare your remotes as egress as well today?

We have not scoped any approval requests as part of this initial release. Would you have a rough idea of how frequently a project admin configures your app compared to a site admin?

This is still to be determined. Only configurable remotes without a default would be permitted, as the administration approval for egress would occur as part of the configurable remote process.

It would be great to walk through these scenarios further to better understand how you envision the migration process working. I appreciate that remote migrations are more complex than egress, particularly when they are defined and stored outside of Forge.

Today a remote does not have an ability to store additional context/properties. It would be good to learn more about this use case.

34

To everyone who has responded regarding Runs on Atlassian, we appreciate your feedback. We are continuing to discuss the implications of configurable remotes/egress against our marketplace programs and your inputs and feedback supports these conversations.

As noted in the original post, it is unlikely we’ll share a distinct position as part of this RFC, however we will share more details once they are available.

4 Likes
35

Thanks, @SeanBourke! Both options would work well for our use case. Being able to load from any location via a wildcard is important for flexibility, especially when we can’t predict the data source in advance. At the same time, giving admins the ability to restrict loading to specific locations would be valuable for customers who want tighter control. Having both options available—loading from any or from specific URLs—would give us and our users the right balance between usability and security.

1 Like
36

Sure thing, mostly I see up to 3 remotes configured, but have seen outliers that have more then 10.

Currently I don’t configure any egress for my remotes, this is because only the customer knows then at time configuring the app. So now when a user uses a link to navigate to the linked Jenkins site, they get presented with “Warning navigating to …” warning box. It would be nice if this can be removed by using dynamic remotes/egress. Its just the question if dynamic remotes come with dynamic egress, or if this is something the app developer must handle.

This is only at configuration time, so would only be done a 1 - 2 times for a project, and would be done around installation time of the app or when a project gets onboarded.
The majority of times the site-admin is involved, but there are some cases where the customer wants to limit the integration and thus opts for per project configuration. This is especially useful for PoC type initial setups.

In my use-case I’m not able to define any remote in the manifest up front. This because my app integrates with Jenkins sites of customers. Its not like having a common github.com remote that customer can “customize” with a dynamic remote to support their onprem version.

Sure we can connect to discuss this, the jist would be that I trigger a migration process to move and Jenkins sites currently configured in my app, to be configured as Forge Remotes. This should not be done using a C&P approach, but should be something is done automatically but with oversight of an admin.

Primary to store configuration specific to the remote, any auth specific details, like a shared secret for JWT auth, user/password for basic auth, what URL of the remote should be used in the UI, same as the remote itself, or something different. Is there a context path that should be used next to supported domain for link generation. Should self-signed certificate validation be skipped or not.

37

That suggestion is necessary, but not sufficient. While this will solve the issue for existing customers, it does not solve the issue for new customers in terms of wanting egresses enabled by default. This effectively creates two cohorts of customers with different sets of base capabilities based on installation date.

Nobody (almost) is going to go to a hidden configuration screen to enable or configure default egresses for a specific app. Even if the egresses can be configured through the app’s UI through the proposed APIs, I write “hidden” because this still requires that the admins pass through the app’s UI in the first place. Many admins will just install the app and do nothing else, so the app’s features will be judged based on what it is perceived as being able to do without those egresses.

Even if the app has a “getting started” screen, it often takes a minute or two to install the app, and the installation-success message is presented only via a pop-up flag in the corner of the screen, so we have already likely lost the admin’s attention.

One way to do this that is both customer-friendly and vendor-friendly would be to present an interstitial screen before installation that looks something like this, with optional egresses enabled by default if the vendor so chooses:

This app has the following external egresses enabled:


                                 Egress Enabled
                                 ==============
Remote to `github.com`           [x] (optional)
Remote to `slack.com`            [ ] (optional)

                                 [Option to enable/disable all optional egress checkboxes]

Remote to `sentry.com`           MANDATORY
Load images from `unsplash.com`  MANDATORY

[OK]

Atlassian already has a screen where it presents egress and scope information, so perhaps this is an upgrade of that existing page.

This seems fine as long as neither requires admin approval (and it would be great if this part of the process could be documented in the C-to-F migration docs).

This is admittedly off-topic, but I do not understand the circumstances in which the new bulk update command is intended to be useful, so I must be missing something.

The definition of a major version says that a major version upgrade generally requires admin approval. In reading the changelog above, it says “a version is eligible for upgrade to any new version of the app that does not require an elevation of privileges”.

How would one even end up with a major version that would be targetable by the bulk-update command? (By going out of your way to manually ask Forge to create a new major version?)

Thanks!
Scott

38

Trying to answer these questions in one response :slight_smile:

What we want to achieve is a mix of pre-defined and customer-specific egress locations.

For example we would like to:

  • Send some analytics data to a self-managed endpoint on AWS. This would probably qualify for the analytics egress as it doesn’t contain any PII etc.
  • Send error data (on case-by-case user-consent) to our JSM instance. We’d like to pre-declare this egress location and add a little explanation text + documentation link for the administrator so they can decide whether they want to enable that or not.
  • Request data from several well-known endpoints related to other apps we’re integrating with. For example diagram image downloads. We could also pre-declare them as optional locations and admins would only enable them if they are using the corresponding app.
  • Let the site admin add custom egress locations in case they are including images from external sources into their Confluence pages and our app needs to load them. Examples: imgur, giphy, you name it. They should then be able to leave their own notes for these locations.
  • Add a feature where we (K15t) can curate and distribute (export) templates online using a self-managed service and app users could use then those templates directly without manually downloading and uploading them into their instance. This would be another optional pre-declared egress location.
  • Send content to 3rd party translation services. We could offer integration with multiple providers and admins enable only their preferred providers or none at all, if they don’t need the translation feature.

Of course we could build UIs that explain all of this to the user and help them set up the egress locations, but if we could simply provide necessary context and documentation links for the standard UI instead, that would probably be preferable because then admins wouldn’t need to go to different locations in each app for managing the egress.

2 Likes
39

Hey, I’m cross-posting this RFC, even though RoA is not in scope for this thread.

The fact that REST APIs would qualify for Runs on Atlassian, whereas configurable egress wouldn’t, is completely unexplainable, if I can’t understand it as a developer, I assume it won’t be easy to explain to customers.

Y’all are probably in different teams, but I really think that this should be worked out internally by Atlassian before these features are available to customers and partners.

4 Likes
40

For those within Atlassian who want to track how Forge power users have experienced the Runs on Atlassian marketing messages with regard to data egress:

Now that we’ve seen al the Atlassian marketing messages about the Runs on Atlassian program for Forge apps, let’s do a quick poll! Data egress is an important topic for Atlassian Forge. With Runs on Atlassian, what do you expect apps to be able to…

Use it to your benefit with regard to a unified approach to Runs on Atlassian and data egress.

41

Hey Sean! Really nice to see you sharing this RFC, thanks!

To begin with, I can’t think of any use case in our apps that could be affected by this in the short term. We haven’t been contacted by any customers about egress either, so at this point, we’d only be allocating time to it because of RoA.
I understand you mentioned this is out of the RFC scope, but, as some other partners have already mentioned, I’d encourage you to reconsider including it. This might be the only solid reason we have to justify dedicating time to this.

I’m also wondering if this could affect us down the road as we move from Connect apps to Forge using remotes. Since we’re already supporting Data Residency, maybe having configurable egress could be helpful, at least to give customers clarity that we’re handling their data through external processes within their region.