RFC-97:REST APIs in Forge

General Request for Comments (RFC)
, ,
21

As it was pointed out, Runs on Atlassian was announced as a label for apps with no egress. It now gets diluted to apps with no egress or controlled egress.

Below is my initial comment on the Runs on Atlassian announcement:

The points Remie made are intended to illustrate that there are already holes in the original Runs on Atlassian concept, and that it no longer aligns with the original communication. We all understand that it is not allowed to use any of these holes for anything other than their intended purpose. The point is, again, that these holes and the example in this RFC illustrate how the Runs on Atlassian concept starts to fall apart (compared to the original communication).

My 2 cents:

  • All Forge apps are “Runs on Atlassian” by definition because otherwise we could have just stuck to Connect
  • Since Forge apps run on Atlassian infrastructure, Atlassian can build a firewall around them and ensure the customer has control over egress and is well informed about it - this is where Atlassian’s focus should be

Why not focus more on these:

  • provide more tools for customers to identify what apps are egressing data and where they have permission to egress data to
  • provide tools for customers to manage/block egress
  • provide tools for apps to query what egress permissions they have, such that they can behave accordingly

If I provide an integration with a third-party tool and a customer blocks egress to that tool, then that’s the customer’s decision. My app should have a clean way to determine when it has been blocked and display a message to the customer indicating that it cannot perform its job. It is then up to the customer to unblock it or uninstall it.

Circling back to this RFC, I am no fan of these highly customized solutions to an industry-standard problem, such as a REST API. I would like to reiterate my point that Atlassian should focus on the interface for Forge apps and the aspects it can control, instead of trying to reach deep into apps and trying to control seemingly everything.

Ultimately, I am convinced this would also eliminate these awkward debates about what Runs on Atlassian should or should not be.

6 Likes
22

+1 for the points raised by @remie and @tbinna

From the perspective of an app working to become Runs On Atlassian, the incentive to achieve this is becoming less and less, especially when the goalposts are continuously being pushed back.

Multiple times here you mention that if we are to implement a behaviour that (at present) is possible using Forge but poses data egress issues, you will be ‘reaching out’. Then don’t you think the entire Forge data egress and Runs on Atlassian setup needs to be reviewed? You designed this. Just because apps have (yet another) badge that you aim to get partners to promote, doesn’t help.

You can’t migrate security conscious customers to Cloud if app use cases are not fully supported within Forge. Runs on Atlassian won’t matter if their solution requirements are lost.

Exactly this. The actual behaviour for Runs on Atlassian is to have no egress or external compute, which is not what ‘runs on’ classes as. I really hope you aren’t going to reconsider Runs on Atlassian and implement a multi-level system, where some apps are full ROA with no compute/external egress, whilst others rely on behaviour similar to this RFC to support certain actions that would typically be classes as egress and disqualify ROA?

As long as the Forge app has 0 connect modules, it’s still Runs on Atlassian (even then, the connectModules are also ran on the historic AC framework), but they are still out of Runs On Atlassian scope. This is primarily targeted for apps that have a lot less module requirements and a lot less complex, meaning Runs On Atlassian is more achievable.

Instead of enforcing this behaviour during app/module definition for app developers, exposing this information to both app owners and installed customers would help. If the customer wishes to restrict the egress or compute, they do so at the risk of the app not working as intended. If the app developer wishes to reduce/remove egress/compute, they can validate what is deemed non-ROA, and work from there.
Following that, as already mentioned, it would be up to the app developer to ensure the app works when egress is disabled, and up to the customer to enable/disable egress or analytics they don’t want/can do without.

It’s complex enough, another 2 modules for this specific scenario isn’t going to fix the hole being dug here.

1 Like
23

Given the RoA badge has already rolled out and eligibility is being applied inconsistently, my 2c would be this:

  1. If it’s a Forge app make it RoA-eligible regardless of egress. This aligns Atlassian’s short-term incentive goal of pushing developers to migrate their Connect apps.
  2. Proxy and log all egress in an admin dashboard. Allow developers to add metadata directly in the fetch request to specify what/why that data egress does and whether it can be blocked or not by admins.

That provides admins with data flow transparency which they can verify in a sandbox. And it allows the developer to clearly explain what/why there is data egress, while preventing any functionally necessary egress from being blocked by admins.

Then replace RoA badge with something else once everyone has migrated.

2 Likes
24

I echo the vendor comments above asking for simpler and transparent, both for RoA and for the REST API.

RoA = Managed Data Egress
Maybe RoA can be rebranded as “Managed Data Egress” where the admins can see and feature flag enable or block certain features that require data egress.

For example, All of my teams (connect) apps send email notifications to users. Because of that we will never achieve RoA in its current incarnation. For us having an admin enable/block the Email feature is reasonable trade off in complexity for what 99% of customers would require this notification.

REST API - KISS
On the REST API, please consider something much simpler. Find the smallest possible footprint in the manifest to allow inbound web requests to be handled. Consider something like the invokeRemote endpoints where the manifest only has the baseURL endpoint, and the path, method and payloads handling are all done in (vendor) code.

The manifest could be this simple…

  • app:
    • api:
      -enabled: true
      -function: apiRequestHandler

At runtime all requests to `https://<site-name>/gateway/api/svc/<product>/<appid>/*’ are routed to the app’s apiRequestHandler function. Let the vendor code handle it from there.

6 Likes
25

How will this work with Cross Context apps?

The current infrastructure supports apps which are linked to a product, which leads to the proposed URL structure. With cross-context apps, this is going to change. We will accordingly be working towards changing the URL for cross-context apps as well.

Will there be any requirements on the custom header names, like has to start with X- ?

A comprehensive list of requirements will be shared as we decide on the timeline for this feature.

Is the write scope a combination in effect of create and update ?

In the current example, yes. At the moment, we were considering only read, write, and delete actions.

Is there a way for the app to verify if an admin disabled the API?

This isn’t something planned for the initial release but we’ll take this suggestion and investigate.

Will there be a way for developers to show a message to warn administrators if they which to disable the API.

We plan on adding prompts every time an admin enables or disables an API. This will be in place for all apps using these modules.

We are only planning to support the standard OAuth 2 authorization grant flow as part of this release. We are leveraging core Atlassian infrastructure (e.g. Identity) so are somewhat limited in the flexibility we can provide. This is something we could look into in the future though.

Questions about RoA and major/minor updates are being addressed in parallel threads.

26

Hi Rashi,
From our perspective, we see the following potential needs in the future regarding this new module:

  • accepting payloads in binary format or other formats such as XML (although for now, it’s possible to imagine a workaround within the current solution),
  • a self-documenting API in a standard such as OpenAPI is important to us. It might be worth considering having the defined REST API module in Forge expose/generate such a document — and perhaps in the future, even go a step further and define services directly using this format. This could be helpful in the case of “extensive” REST APIs. It would eliminate the need to describe the same endpoint in multiple formats (one in the app descriptor and another equivalent, for example, in OpenAPI in other tools).
  • Would it be possible to define multiple methods (GET/POST/DELETE) for a single noun, e.g., “payroll”, all at once? I suspect that defining each operation separately might contribute to exhausting the overall module limit available for a single app (100). Especially since each module also requires an additional function in manifest.
3 Likes
27

Hi all,

The conversation about Runs On Atlassian is important, but it’s starting to overshadow the main RFC discussion regarding the Forge REST API modules that our team needs to build the right feature here..

Let’s keep this thread focused on how the module works, your team’s requirements, and any considerations we need to address.

If possible, could we move the Runs On Atlassian discussion to a separate post?

I’d like to revisit Runs On Atlassian topic later with a more detailed perspective from our team, especially as we consider features like REST APIs, Optional Egress, and Dynamic Egress.

Thanks,
James Dumay
Group Product Manager

1 Like
28

Bitbucket Cloud has a API proxy module that already solves a similar the problem. With a solution like that in place there is no direct egress from a vendor app.

1 Like
29

Aside of this, could you describe how 3LO could be used by two forge apps to use REST APIs for inter-app communication?

Yes, this will be an option for inter-app communication. Due to the subdomains being different it will also require ROADMAP-99: User-Defined Remotes but that is scheduled to be released about the same time.

We will also explore other mechanisms for more seamless inter-Forge app communication but no details to share on that yet.

“As with other Atlassian apps, the current rate limits will be applicable for these APIs.” Could you please confirm if these are per site?

These are at site+app+env+user level. More information here.

2 Likes
30

Would it make it easier for you and us if this change was taken into account now?
“Simply” creating the URL to work with single and multi contexts would save you time building and us time migrating.

Please include messaging that disabling the API could render the app useless. Or provide developers a way to make this clear.

Alternatively you would opt for developers to mark an API as required, this would block admins from disabling the app, and they would instead need to uninstall the app.

I completely understand the need for admins to have control, but disabling a feature like this could render the app useless or result in unexpected behaviour resulting in customer, support and developer dissatisfaction and everything that comes with it.

I would presume that the authentication methods used by Forge (FIT) and Connect (JWT) also run as core Atlassian infrastructure. Being able to use those methods of authentication ideally with custom keys/shared secrets would make it so the Rest API can be supported by far more systems.

For my use-cases, its system events that can trigger invocation of Rest APIs in connected systems, there is no user context to speak of or resolvable. For me app would only need asApp() to ingress data into the host.

I also really like the suggestion from @Chris_at_DigitalRose this would provide support all my use-cases.

Initially I thought I could work with 800 kB request payload limit, but my use-cases require more as they are also used to data ingress, and 800 kB per request is not useable.
If the request and response payload sizes where both 5 MB, then I can cover all my use-cases and have a couple of bits left over.

1 Like
31

@JamesDumay I don’t think these two things can be separated as it directly impacts the architectural decisions with regard to Forge REST API implementation.

I’m still waiting for feedback on my proposed alternative solutions which are focussed to make Forge REST API available AND make it make sense from RoA perspective:

32

It really sounds like an artificial limitation based on the current Forge “lambda” runtime implementation.
Will the same limitation be applied to the future container-based runtime? And are there any other aspects which may be different for container-based runtime?

33

Please move this discussion to the other thread.

34

Why? I’m proposing changes to the RFC?

35

Can we disable it by default but make it minor version please? The reason is that for our apps, only a small subset of customers want REST API.

If adding this feature makes a major version, all customers get stuck to this old version. I really want to avoid major version upgrade whenever possible because it is really painful to upgrade users and many don’t do at all.

5 Likes
36

I’ve stated this above, it is an important discussion but we need this feedback to build the right feature for partners and customers irrespective of our trust signals. I hope you can understand and respect that need.

37

Have you read my comment?

I’m proposing two alternatives to implementing Forge REST API. This is directly tied to this RFC as it provides direct feedback and does not have a place in a different thread.

I hope you can understand and respect that I would like Atlassian to respond to my alternative suggestions for implementing Forge REST API, as per requested by this RFC

38

Remie, I have asked politely and the persistence is bordering on disruptive. You have a different thread where we are engaging directly with you entirely dedicated to the Runs On Atlassian topic encompassing at last two different RFCs and a LinkedIn post as well. You are being heard loud and clear. Please use the other discussion thread.

39

This is becoming absurd. My comments are about this RFC, not about Runs on Atlassian. I am proposing a different implementation and would love to get feedback on my proposal. This has nothing to do with RoA. Why is Atlassian not engaging with my feedback on this RFC?

40

Hi Rashi,

Thank you for you comment. Of course, I understand that there might be iterations and improvements in the future, thank you.

I’d like to add that it is also difficult to support APIs as described in the proposal. Our APIs have more than 40 endpoints, and we’d need to re-document all of them. And then keep a separate OpenAPI spec anyway to generate documentation.

I think it’s good that we are making a step forward with this proposal. But without a more mature and standardised solution planned, at least as a fast-follower initiative, I’m afraid the solution becomes very unattractive.

How about evolving this proposal to a two-step proposal, where the second step actually accepts an OpenAPI spec for describing an API?

Thank you.