Resolution & Jexo Forge Hackathon Findings

State of Forge Report

Introduction

Between May 31st and June 4, developers from resolution GmbH and Jexo did a hackathon to explore Forge, the Atlassian development platform. It was the second time that the two companies held a hackathon dedicated to apps for Atlassian cloud applications.

The hackathon had two goals: understanding what can be done with Forge, and hitting the ground running with app ideas. A total of 5 teams were formed, and they created 4 different app MVPs.

This document includes our findings regarding what can be done with Forge, identifies pain points and limitations, and includes some recommendations.

While limited in scope, we think that the document can be useful in at least three ways:

  • As a learning kit for Ecosystem developers who are experienced in Connect and will or may start creating apps in Forge or migrating apps to Forge in the near future
  • As a foundation for shared ecosystem knowledge of how to develop apps with Forge
  • As a source of recommendations/suggestions for Atlassian in further developing Forge. Since our time was limited, it’s very possible that we didn’t find the right approach around certain topics. If that’s the case, we’d hugely appreciate it if anybody at the Forge team can contribute to the discussion and point us in the right direction.

Since the document is quite lengthy for a quick read, here is a basic outline of its findings.

Summary findings

  • Review of UI Kit & Custom UI: We found UI Kit is easy (particularly for developers with React experience), and okay for simple UIs. Custom UI offers more interactivity, but is still very a bit limited compared to Connect, specially when it comes to the Javascript APIs Connect provided under the AP global variable.
  • Review of tooling and templates: Benefits and limitations of Tunnel, Deploy, Forge CLI with Typescript and using our own templates.
  • Restrictions encountered with Forge’s execution environment and storage: Runtime, Snapshots, security considerations
  • Lack of support for Jira Service Management. This was the heaviest limitation we encountered with Forge and deserves its own mention.
  • Modules and Permissions. Review of Forge Scopes and suggestions to improve them, limitations of user impersonation, review of Forge modules, similarities to Connect
  • App Migration options from Connect to Forge. Known limitations of Connect on Forge and recommendations to prepare for the migration, including which programming languages should be used and how to best build adapters.
55 Likes

UI Kit vs Custom UI vs Connect

Those are the two options you have, when you want to give your Forge app a user interface. There are some similarities between the two. Both use JSX as a syntax for writing these user interfaces and (depending on which framework is used for Custom UI) offer functionalities similar to React Hooks. And that’s where the similarities end.

UI Kit

UI Kit functions are executed in Forge’s FaaS environment and ultimately output a single tree of HTML elements that get put on the user’s web interface. There is no way of putting your own HTML in there somewhere. You have to use Atlassian’s UI Kit components. Half of these are wrappers that need to be used for certain modules, so the selection is pretty limited.

Interactivity Restrictions

There is also less interactivity. We mentioned that the code that generates the output is run inside Forge’s FaaS environment. It’s also important to know that these functions have a runtime restriction of 10 seconds. There is also no intermediate output before the function is fully done with all sync and async operations. This means that the loading screen from when the function has started executing can be no different from just before the last piece of content is loaded. Your function’s output will not be displayed, until all async calls are finished loading.

This also means there are no set Interval calls that continuously update your app every so often.

The only way to achieve an update of your UI, is by having the user click a button (or submit a form, which is also just clicking a button). Then your function will be called again and you can update your UI. But this also means there are no controlled inputs for text boxes, selects and other form components. No onClick, no onChange, no real-time validation, no interactivity. If you need interactivity, you have to use a button. If you don’t want that, you have to use Custom UI.

Good enough, sometimes

It is sufficient for simple UIs though: If you have a simple settings UI, like for a Confluence macro, or your app only needs to display some small amount of data in a table inside a Jira custom glance, then you are good to go with UI Kit. And it does have some advantages over Custom UI.

For example, the React-hooks-style API supports async functions everywhere. In React, this would wreak havoc, but since Forge is not exactly React, it’s okay to have an async initialization function in useState, since there is no incomplete state rendered anyway.

Simple to use

It is also easier to set up than the Custom UI. The file that generates the UI is simply a Forge function, which exports a render function. It lives in the same package as your other Forge functions and uses the same config files. A Custom UI is a bit more complicated.

Custom UI

A Custom UI resource basically consists of your own HTML, CSS and JS (and other types of files, like images). At deploy time, it is all bundled up and sent to Forge. When your module’s entrypoint is called, your HTML page is loaded inside a sandboxed iframe and delivered with a somewhat customizable, yet still restrictive content security policy. The experience is somewhat similar to a well-secured Connect app.

Bundling

How you get to that bundle is basically up to the developer. Typically, there will be another package.json somewhere inside or next to the Forge package’s one. This alone is, in our opinion, not great. The templates by Atlassian all use create-react-app. This seems fine, until you need that one feature, which requires you to eject. But you can also set up webpack or parcel or just write vanilla JS. Do note that external scripts and such need to be allow-listed in the manifest if you want to use them. The same goes for external media.

Custom UI Bridge

It is also not possible to make calls to external services from the frontend, but fortunately, there is a way to call your own backend hosted on Forge. This concept is called the Custom UI bridge, and it effectively lets a Custom UI iframe communicate with a Forge FaaS function. This function can then make calls to allow-listed external services, such as your own backend or third-party APIs you use. On top of that, Forge functions have access to credentials injected via environment variables, so your function can actually authenticate with that third-party API. Neat!

Of course this restriction about Custom UI not being able to call external services directly, means there is probably no simple inclusion of (Google) analytics or (Sentry) crash reporting snippets. All of the actual communication with the servers is possible via the bridge though, but has to be done manually.

Edit: As pointed out by @asridhara, you can in fact call external services if declared in your manifest. I had not seen the section in the docs about this. Sorry, my bad.

Routing

Another, slightly awkward restriction of Custom UI is the fact that if you have multiple modules using Custom UI, you need to do one of three things: Either declare multiple resources for the different modules and get your bundler to output all resources once per entrypoint. You can also have multiple, completely independent resource directories with separate package.json, bundler configs, etc… Or you declare one resource and do moduleKey-based routing. You can do code-splitting for that last one, but it will increase your load times. Neither of these is particularly elegant in my opinion. Ideally, we would be able to explicitly declare the file to use within one resource in the manifest. Currently, this always uses index.html, which effectively prevents us from using the regular entrypoint mechanism of bundlers, where for different entrypoints, different files are loaded.

Further Custom UI restriction

It also needs to be said that only some of the Javascript Connect API functions (exposed via window.AP) that are available in Connect iframes, have an equivalent replacement in Forge. But like most things, this area is improving steadily.

Unlike Connect, there are also fewer allowed sandbox parameters on the Custom UI iframes.

These permissions are not present on Custom UI:

  • allow-popups
  • allow-top-navigation-by-user-activation
  • allow-storage-access-by-user-activation

To be fair though, popups and top navigation, which is possible in Connect by using a regular HTML anchor with a target attribute, can be achieved by using the Custom UI bridge’s router Javascript API.

Atlassian Design

And for a UI library, you should probably look into Atlaskit or its reduced-UI pack version for matching styles with Atlassian if you’re using Custom UI. Of course that is not a requirement, but this makes your app feel more integrated with the host product.

Conclusion

So, which way should you go with your app’s UI? It depends.

Are you okay with it being simplistic because it’s a simple settings UI? Go with the UI Kit. It’s easiest to get started with. And if you already know React, you’ll feel right at home.

Do you need more interactivity but can accept living in a very sandboxed iframe? Go with Custom UI.

Do you need all the power that only an iframe unrestricted by a content security policy can wield? Go with Connect. For you, Forge is not there yet.

Do you need to reach deep into the inner workings of the page, the editor or the authentication mechanisms in Jira or Confluence? Hi P2 developers, fancy seeing you here :stuck_out_tongue_winking_eye: stick with P2 for a couple more years.

Atlassian Cloud is far from this level of integration.

Tooling & Templates

The centerpiece of tooling around Forge is the Forge CLI, which you need to install to interact with the Forge platform. In our opinion, it’s a good piece of software that makes a lot of the tasks that need to be done around building a Forge app fairly easy. The documentation is good and most functions work well enough. Some are a bit rough around the edges though.

Tunnel

The tunnel functionality in particular was pointed out several times as being very helpful with fast development cycles. It simulates being run inside Forge’s FaaS environment by running inside a Docker container. All requests to your app are being redirected to that container while the tunnel is active. That being said, we have hit some minor road bumps during our hackathon.

Reload it like it’s hot

If a function is being invoked while the tunnel is recreating the bundle, one of three things can happen: The old function answers the request, the tunnel waits for the new function to be ready and then that answers the request, or the tunnel errors out and the Forge app needs to be reinitialized. We would have liked for this behavior to be more consistent, ideally by letting all requests wait until the tunnel has reloaded, even if it takes a couple of seconds.

We have not experimented with connecting to our own dev server for Custom UI, but it sounds like a good solution to this issue for Custom UI resources.

Other issues

It would also sometimes happen that the tunnel gave a 402 response. In that case, it had to be restarted. This might also have been the underlying ngrok instance though.

Using a monorepo with workspaces can also be an issue. Due to the tunnel using Docker, the only directory that’s passed into the container is the one where the manifest.yml file is located, as well as all child directories. If you use a workspace, by default all dependencies in the node_modules directory are hoisted to the workspace root, where they are not included in the Docker container. Therefore, the nohoist option would have to be used, which is currently only supported by yarn.

Deploy & Manifest

The forge deploy command will bundle and upload your code to a given environment (development by default). It actually uploads the bundle to AWS S3, where Atlassian picks it up. Then it does some GraphQL calls. There are some limitations though.

Speedbumps

This is not a huge issue since the tunnel exists, but waiting for the deployment process to finish can be a bit tedious. It’s not an issue with bundle size, we think. It may take a minute or so regardless of it. We have not investigated further which part of the process exactly is causing the slowdown, but in general, you can take a short break when deploying.

It would also sometimes just randomly fail with some error. We discussed this and it definitely happened less often than during our previous Hackathon in mid-December, but it did still happen. Then the deployment would have to be repeated, which took time again. Better error messages would have been appreciated, even if not running in verbose mode. Because they generally do exist, but are not exposed to the user in non-verbose mode.

When an upgrade is necessary because something security-related like CSP exemptions or scopes changed, the CLI is nice enough to point out that instances with the app installed needed to be upgraded. This is nice! The upgrade process can be a bit tedious though, since for every command, the forge install --upgrade command needs to be called. This takes time, so it would be nice, if one could just upgrade all. Especially for those early development times, when these sections in the manifest get updated regularly.

We also found it weird that there are additional products in an instance like “Ecosystem” and “Identity” to upgrade.

Limitations

It can happen that you accidentally try to deploy a bundle more than the 50MB limit in size. We think that’s actually fine, but it did happen to us.

The manifest itself also has one limitation that seems unintuitive and arbitrary: Keys can not be longer than 23 characters and have to be unique across the whole manifest. This is not a huge issue, but some of us found it weird. We like to use long keys to make the identifiers more descriptive. And having “scoped” keys, such as using the same keys for a macro and the function that served that macro would have made it easier to navigate the file. Kubernetes does this and it makes it very obvious which resources belong together.

Templates & Documentation

Forge CLI provides a way to easily create new Forge apps. This is nice for first time developers, but it has some downsides for more experienced people.

TYPESCRIPT ALL THE THINGS!!!1!11!1one!!1one!eleven

We like to use Typescript for all of our Node and web-related projects these days. Javascript is nice for quick scripts, but as soon as projects become even a bit bigger, using Typescript just helps soooo much with so many things. From remembering which fields a class or an object has, to strict type enforcement so you never forget that something could be undefined or null, Typescript is here to help.

This is not to suggest that Typescript doesn’t have its downsides like missing built-in support for runtime type validation and a sometimes very verbose syntax, but it helped us so much in our development of cross-team medium to large projects. This is why the small number of Typescript templates surprised us.

There are two Typescript-based templates in the template library and neither of them include Custom UI. As discussed above, we realize that developers are free to choose their own way of ultimately getting a Custom UI resource bundle. But it would have been nice to see one anyway.

Looking at the node_modules/@forge directory, it seems like pretty much everything about the Forge libraries is already built in Typescript. Everything ships with typings and there are .d.ts files all over the out directories.

Why, then, is it not the other way around, having vastly more templates with Typescript than Javascript?

Outdated library versions (and sometimes templates and documentation)

Another issue with the templates is that, while they should ideally work out of the box, sometimes they don’t. This is usually due to outdated @forge libraries used in those templates. We have hit this issue several times in the past and have been able to learn from it. So it wasn’t a big issue for us, but it’s not a great experience for first-time forgers.

One solution to this could be upgrading all packages after forge creating a project. But this would have to be done regularly at Atlassian’s site as well, because we also stumbled upon outdated syntax Forge UI syntax in a template. This led to breakage when upgrading the @forge/ui package.

Some tutorials on the webpage had similar problems, also with scopes.

Don’t get us wrong here, we love the speed at which Forge is being developed and we appreciate that it can be a bit much to keep all templates up-to-date every time there’s a new version. But some of this could be automated and we think it might be a worthy time investment.

BYOT (Bring Your Own Template)

These points lead to us trying to design our own templates. We have created one UI Kit template and one Custom UI template. The latter also tries to use code splitting and internal module-key-based routing, but it’s not as good as it could be with a couple more days of work.

Of course, it would be nice to be able to bring your own template so “forge create” can use it. And luckily, the command has an optional CLI option for giving a template path. But internally, this is only resolved within the Atlassian Bitbucket namespace.

We have managed to break out of this namespace by using a directory traversal :innocent:, but then your directory names are screwed up and the Forge CLI has no problem walking all across your filesystem when you give it the right template name. This is not a huge issue, but we would appreciate it if the “forge create” code would be able to recognize non-Atlassian links and accept them as well.

You can find out our templates at

https://bitbucket.org/resolutiongmbh/forge-typescript-ui-kit-starter/src/master/

https://bitbucket.org/resolutiongmbh/forge-typescript-custom-ui-starter/src/master/

They are already outdated :sweat_smile: because of Atlassian changes.

Execution Environment & Storage

Forge functions run in a Node.js-like environment with a bunch of restrictions. These can make it hard to include some libraries and one should always be aware of this fact. There are also some restrictions to the data storage Forge provides.

Runtime Restrictions

You can find the runtime restrictions at Runtime and the platform quotas and limits at Platform quotas and limits. Especially note how the snapshotting works and which restrictions it has. This can save you some headache when debugging.

Most of these were not really an issue. Some were though and the error messaging is not always helpful.

Issues with libraries

We encountered some issues in the past when we tried to include libraries and this did not change for this hackathon. The error messages were sometimes not very helpful, for example the tunnel just crashes when our code or library code tries to include the crypto package:

Removing the reference to the crypto package does the trick. Deploying with snapshots enabled also seems to work. Our interpretation of this is that the tunnel environment differs from the actual execution environment, which is also not optimal.

Disabling snapshots actually fixes this issue, but that is not optimal either.

Snapshots

Snapshotting is used as a mechanism to accelerate function execution common in FaaS environments among others. Basically, a Node.js instance (modified to have the restrictions Forge has) is created and the Forge app’s file bundle is loaded, but no function is executed yet. For example, if you have a simple file where you import a library and export a function, those imports are executed (and everything related to those, like subsequent imports), and your exported function is parsed and created, but not executed. Then, a snapshot of the state of Node.js is taken and stored, hence the name snapshot.

When there is a request, Node.js is started again, but with this snapshot loaded back into memory. Then your function is executed. The advantage of this technique lies in the saved time: The initialization logic is only executed once for a code bundle, which means that any subsequent start of Node.js is faster and costs less money. But it also means that the UI is more responsive.

We don’t know exactly how this stuff works internally, but it does have some downsides: Environment variables are not available, and anything stored outside of functions can exhibit some weird behavior as it might be reset between function executions since a snapshot might be restored. This is not necessarily really problematic, but it is something that a developer needs to understand and be aware of when developing and especially when debugging.

As for cryptographic operations, we have found that jsrsasign works well for us. We used to check some signatures and create and validate some JWTs, all of which worked well with that library.

Forge App Storage

As with a lot of things related to Forge, Atlassian must not be envied for its balancing act between trying to prevent abuse of the platform and keeping it useful for developers with legitimately complex use cases that push those limitations.

It often feels like Forge’s app storage is a prime example of these considerations.

Background

App storage is a simple key-value store. As such, it is mostly well-suited for app settings, but only non-security relevant settings should be stored. Think of it as a replacement for the app properties API that exists in Connect.

The storage currently has some restrictions, for example it’s not possible to have relationships between entities or define schemas. Therefore, it can not currently be considered a replacement for a real DBMS that a Connect app might be connected to. But then again, for a lot of apps, that’s totally sufficient.

There is actually one functionality that makes it more interesting than a pure key-value store: You can run queries against and do paging, optionally filtering by key prefix. This, along with the ability to have 100 byte long keys, makes it possible to have a kind of hierarchical key structure, with each level separated by a dot.

As with everything Forge, app storage is also under constant development, so the information in this document might be outdated by the time you read this.

Security & OAuth

App storage accessible from both the frontend (in Custom UI) and (edit: I got this wrong, it is only available from the backend, a Custom UI needs to use a resolver to access storage) the backend (i.e. UI Kit functions and Custom UI resolvers). As such, it can not be considered safe from user influence and should not be used for storing security-related values such as application secrets, API keys, OAuth access tokens or passwords.

OAuth in general is not currently possible to do purely in Forge, but Atlassian has hinted that they are looking at an alternative that would also solve the secret and URL management problem.

Modules and Permissions

In this section, the supported manifest modules and scopes are discussed. Let’s start with the big one, which caused us the most trouble.

No Support for Jira Service Management

This is probably the biggest problem we have with Forge right now.

A lot of things related to Jira Service Management projects right now are broken: There are no events for those projects and its data can not be accessed via REST API. This is apparently due to Forge not supporting any of the scopes necessary to use JSM projects.

Some things do work though. For instance, adding a Jira project page module in your manifest will cause that page to also appear on JSM projects.

But with most other functionality missing, several of the projects we attempted at this hackathon were severely restricted.

Existing Scopes

Scopes in Forge apps are basically permissions on the host product the app requests when it is being installed. They give the installing person information about what the app needs access to. This concept is not new, however there are some details that are special to Atlassian.

Forge vs Connect scopes

One thing to keep in mind when developing Forge apps as opposed to Connect apps is the fact that Forge uses the OAuth scopes that were previously reserved for OAuth apps. They are much more granular than the Connect scopes and allow our apps to request only those permissions we need.

In security, this is called the principle of least privilege. This way of doing things is recommended because even if an attacker were to gain access to the credentials or the code of the app, they couldn’t do as much damage as they could, if the app had all the permissions to do everything.

Way to Improve Scopes

In general, we think the scopes are well implemented in Forge. They offer a nice mix of granularity without being annoying because every function has its own permission. The documentation around scopes is also very well written and understandable.

If Atlassian wanted to improve this system a bit further, we have some ideas.

Firstly, they could allow us app developers to add reasons why we request all scopes. The texts Atlassian shows around what a certain scope allows are nice, but they do not reflect exactly how this particular app uses this information and why we need it.

Also, there could be optional permissions. Similar to what smartphone operating systems have done for some time now with permissions like location access, some permissions could be optional to the functionality of the app. For example, a Jira app might offer a functionality to watch issue comments for certain attachments, but would also work without that.

Lastly, there is no way to distinguish between scopes needed for the app user and scopes needed when impersonating a user. An admin might not trust an app to edit comments using impersonation as it would make tracing edits impossible from an audit trail perspective.

User Impersonation

Forge functions sometimes have the option to run certain operations as the requesting user. This is similar to the Connect scope ACT_AS_USER, but does not really cover all the use cases.

Limited locations for user impersonation

A Connect app with the ACT_AS_USER scope could decide at any point to run any REST request as any user. This allowed for great flexibility in our code.

In Forge, only a very limited number of function locations are allowed to use user impersonation. Basically, we’re limited to UI kit handlers and Custom UI resolvers.

There is no user impersonation for host product event handlers where one would assume the app could run operations as the user that triggered the event. There is also no impersonation for web triggers, but we understand that there would be no reference user to run as. Which brings us to the next restriction with user impersonation.

Ability to select the user to impersonate

It is not currently possible to select the user to impersonate in the app’s code. If a request is being run as a user, then it is always the requesting user. But this is not always the correct user to run as.

From an app’s perspective, it might be necessary to be able to edit a comment as the author for example, even if it is in response to another user’s action.

We would argue that we could then, in most cases, simply run requests as the app user, but this is also not possible sometimes.

App User does not always have the right permissions

It is a known bug at the moment that app users don’t have the correct permissions for all the scopes. Combined with the aforementioned restrictions on user impersonation, this can make it hard to execute certain actions. It’s also hard to find this issue and if you don’t know about it, you could easily lose hours asking yourself why the project property API doesn’t work (for example).

There are workarounds noted on that issue though, so check them out to see if they fit your use case.

Modules

The modules you can define in the manifest basically describe which places in the host application your Forge app can hook into. They roughly correspond to modules in Connect apps, except they’re simplified in Forge.

Like Connect Modules But a Bit Different

For example, a Forge module for a Confluence context menu action can be declared in the manifest. When clicked, it will open a dialog which contains either a UI Kit page or a Custom UI. Besides the actual module definition, you need to define either your handler function or your Custom UI resource and then reference it in the context menu module.

In Connect, this would look a little bit different. You would define a module of type web item with a location of “system.content.action/secondary” for example, then point it to a dialog URL or a module of type dialog.

Finding this location requires the use of the web fragment finder app on the Marketplace, because there is no documentation on the available locations. Yes, the Atlassian documentation links to that non-Atlassian Marketplace app. This is not necessarily a really bad thing. Wittified, the vendor of the app, is a well-known and reputable company within the Atlassian ecosystem, but it does feel a bit strange that using the app is basically a requirement for developing Connect apps.

In Forge, the locations and their locations are somewhat less flexible, since a certain location always requires a specific type of container: page, dialog or inline dialog. But the Forge way does have the advantage of being well-documented and constrained in a way that leads to a consistent UX for the end user that we appreciated. Sometimes less is more.

Limited Selection, For Now

As nice as the new module definitions are, we were missing some locations in our Hackathon. For example, there is no way of putting a page in the Jira project settings at this point. We were able to work around this by defining a project page and using display conditions to only display the link to admins. Note that the conditions you need to use are different between team-managed and company-managed projects!

We also noticed that it’s not possible to add multiple Jira project pages. This feels a bit arbitrary, since more complex apps could have very good reasons for adding more than one project page. There are probably similar limitations on other modules, but we didn’t test them.

Another limitation we hit was the lack of a custom edit UI for Confluence macros. Currently, it’s only possible to define a UI Kit settings page, which looks similar to the Connect macros, when no custom edit UI is given, but rather a list of parameters.

All of this is certainly not optimal, but acceptable for now. Forge is under constant development and so far, the team at Atlassian has been very responsive when it comes to suggestions. It also resonates with what feels like the goal of Forge.

38 Likes

Connect on Forge

You have two paths to move from Connect app to Forge app.

The first one is to rewrite the whole app from scratch using only Forge. This might be possible for small and simple apps, but can be a challenge for large and complex apps to completely refactor all codebase, including how you persist data.

The second path is to register your Connect app on Forge using Forge CLI and at your own pace start adopting Forge features. Your app will still be hosted at your remote server, but will already be managed by Forge infrastructure.


Paths from Connect to Forge - Source: Atlassian Developer Day

What is Connect on Forge

Connect on Forge offers a way to continue using your Connect app while benefiting from Forge specific features. In the future it will certainly become a way to migrate from Connect to Forge gradually. Why in the future? Because at the moment, it’s still an alpha release with issues and some serious limitations.

Although it’s not production-ready and you can’t publish it on Atlassian Marketplace yet, you can try to convert your Connect app and use Forge tools to experiment.

Known limitations

The main limitation is that it’s not yet possible to list a Connect on Forge app on the Atlassian Marketplace.

Atlassian has not yet found a proper solution to migrate a Connect app to Forge app with the same key on the Marketplace. This can bring some issues if you depend on the app key. During Atlassian Developer Day on the Connect and Forge session, Joe Clark said that you’ll be able to replace your existing Connect app on the Marketplace and that Atlassian will help you manage the upgrade with your customers. But there’s no details on how that would happen.

When you create permissions on the descriptor using jiraGlobalPermissions or jiraProjectPermissions, these permissions endup having it’s key pre-fixed with the app key. If your app is assigned a new key, these permissions will be recreated with another key and you lose all permissions that were already configured using the old key.

Dynamic modules are not yet supported. They would have the same problem as described for permissions, because dynamic modules also prefix it’s key with the app key.

Connect on Forge is not allowed to impersonate a user and make requests on the Jira API. In other words, ACT_AS_USER permission scope cannot be used.

A list of all known limitations can be found here.

Conclusion

Creative Constraints

As a developer, there are often multiple ways of doing things: Should that button stick to the bottom of the content or go to the bottom of that screen? Should we use a dialog here or open a new page?

Forge feels like it’s trying to answer some of these questions for app developers. It’s a bit hard to judge whether this is a good thing or not. For simpler apps, it probably is. But more complex apps might need the flexibility that Forge does not yet offer.

Ultimately, if you are a developer who needs to decide whether to build your next app with Forge or Connect, try to build it on Forge and fall back to Connect if it turns out Forge is not right for you. Atlassian has not yet announced the deprecation of Connect, but a lot of people in the developer community are convinced that that is going to come.

Building Connect Apps with Forge in Mind

But even if you decide that a Connect app is the way to go for now, there are ways you can prepare your apps for moving your apps to Forge in the future:

  • Use Typescript or Javascript. No Spring and Java, all atlassian-connect-express (or your own version thereof).
  • Modularize your backend code in a way that you can easily replace any REST or GraphQL routes with Forge resolver functions. Pretend you are already using Forge resolvers and have your REST route call that resolver function. Or just use a FaaS like AWS Lambda in the first place
  • On the frontend, use cacheable iframes and a content security policy in preparation for Forge serving your code. Deliver that frontend as a static bundle either from a CDN or form your backend. No server-side magic!
  • Build adapters for storage which you can easily swap out for Forge app storage or just use entity/content properties directly
  • For any callbacks, try to stay under the 10 second limit. Jira expressions are your friend for slow-performing REST calls to Jira.
  • Use adapters for accessing the host product so you can easily swap them out for Forge-specific ones
  • Try to avoid using the Javascript Connect API too much since Forge does not support a lot of it yet. If you do use it, use an adapter so you can easily switch to Forge’s own APIs in the future

There are probably more ways you can plan your app so it can easily be migrated to Forge that we haven’t listed here. Doing a Forge hackathon is a good way for getting a feeling for the platform, especially for the more architecturally minded software developers.

This document is the result of a Hackathon conducted between May 31 and June 4 2021 by 4 teams of developers at resolution GmbH and Jexo. It contains the result of our own experience and tests, which may differ to that of other teams. Any misrepresentation, if any, is there in good faith and with a spirit of collaboration. If you find any errors or oversights, please point them out here and we will try to integrate them into the document.

40 Likes

That’s an awesome write-up. Thank you for sharing your experience. We are working with Forge since over six month now with Easy Subtask Templates. I totally agree with all of your points. I can make two short additions:

  • While you have not experimented with this we heavily rely on connecting or own dev server for Custom UI for tunneling purposes. This functionality of Forge works great. Even with multiple local servers on different ports.
  • In my eyes, the limitations of the current Storage Api cannot be emphasized often enough. The 32kb limit of values and the few querying functionalities cause us headaches. This is a heavy blocker if you want to implement use-cases solely within Forge without hosting your own service somewhere out of the Atlassian Cloud. We really hope, that these limits will be lifted in the next time.

Thank you again for you detailed post. :smiling_face_with_three_hearts: It somehow calms me that other Forge devs have similar issues and thoughts about the platform.

13 Likes

What a fantastic write-up! Thanks for taking the time to share this. I just wanted to let you know that we actively monitor this entire community to try and understand the needs and problems that our developers face and everything in the post above is being broken down and incorporated into our planning process and roadmap. I can’t promise that we’ll fix everything tomorrow, but I do want to say that many of the concerns above we are actively working on.

From here, I’ll probably pull out individual sections from above and comment on them where I can.

When you deploy a Forge App these rough steps happen:

  1. You bundle your app code, via the Forge CLI, on your own machine.
  2. The CLI uploads it to AWS.
  3. Your deployment gets added to a queue.
  4. Your deployment gets picked up and provisioned in Atlassian infrastructure.
  5. The result of your deployment is sent back to you.

I just want to let you know that recently we poured some time into making step (3) significantly faster. This change was shipped at around June 4th. Now each individual deployer get’s their own virtual deployment queue. Everybodies queues run in parallel (to a maximum limit which is quite large) and within your own queue the deployments run sequentially. That means that, according to our metrics, most deployments, most of the time, see no delays due to queuing: the provisioning starts effectively immediately.

Screen Shot 2021-07-06 at 8.08.29 am

For “provisioning time” of your actual deployment (4), most of the time it takes around 20s to actually make happen. Some of that is out of our control as there are AWS restrictions, a few seconds are within our control and we may aim to make that situation better in the future though it is not a priority right now.

Screen Shot 2021-07-06 at 8.11.39 am

The message that I want to send is that, for most people, if you assume that when you run the forge deploy command that it will take around 20s that’s probably a good average guestimate for how long your deployment is going to take you. This obviously varies from app to app, deployment to deployment.

I also want people to know that we actively monitor this and are trying to make your development loop experience as fast as we possibly can. Speaking from personal experience, my Connect Apps that I host in EC2 take significantly longer than this. Only my “frontend-only” static Atlassian Connect Apps deploy at a similar speed. So Forge is certainly has the faster development speed for me.

I hope that gives you an interesting insight into our deployments at the moment and shows that the deployment process should feel and be significantly faster than when you ran your Hackathons. If you have any questions please ask away.

14 Likes

With respect to this, I can tell you that we did have a very lengthy discussion about this internally and it is still open to change.

What we knew, for sure, was that we would need the JavaScript examples, so we focussed on them first. This is because there are simply more JS developers than there are Typescript developers.

We were waiting for more feedback on the template options people chose to know wether or not to invest more in Typescript, maybe even getting to the point where every template was written in both Typescript and JavaScript. It’s really up to community pressure to get us there.

You could even use this comment as a proxy. If you want there to be a Typescript equivalent template for every JavaScript template then hit that like button. Then our PMs will use our various sources of data to make a measured decision.

22 Likes

Yeah, we are aware that this deviation from a standard Node.js development experience is not fantastic, but the performance wins from Snapshotting were just too good to not incorporate into the platform.

Our Smartlinks saw a 17% function invocation performance improvement on average (time reduction).

Some functions, especially code with larger bundle sizes, saw ~65% reductions in invocation times.

This is why we have Snapshotting on by default, it’s just too good to ignore. We hope that developers will be able to learn about, and work with the nuances this does place on the development experience. If there is anything that we could do to make it better, just let us know.

5 Likes

I have confirmed that this should not be the case, you should only be able to access App Storage while being inside a Forge function. You should not be able to go straight from Custom UI frontend directly to App Storage as a User. If you have found a way to do so, please send me a DM with the steps.

The intention is that only the App can access its own App Storage.

1 Like

Just tried it and it is indeed much faster. The deployment of my test app takes about 30s now and it also feels more reasonable now. Most of that is actually the upload, which is not saturating my internet connection for some reason. Maybe S3 is just slow :man_shrugging: We must’ve just missed the improvement with our hackathon happening a week before that.

I agree that snapshotting is worth it. This point in the writeup was really only to emphasize that the developer needs to be aware of it. I know that someone from the Forge team has documented this behavior and done a good job with it IMO. I’m not sure how it could be improved… maybe some error messages when our code is trying to access process.env during snapshotting if that’s possible?

You’re right, I did get this one wrong. I have corrected the post. Sorry :grimacing:

I’m not sure then: why aren’t we supposed to store secrets in Forge storage? Is it only the lack of encryption at rest? Couldn’t we just hardcode an encryption key in our Forge apps via environment variables and encrypt and decrypt the stored secrets? The environment variables are safe to use for secrets like API keys, right?

6 Likes

Yep, encrypted environment variables (forge variables:set --encrypt example) were designed for this exact use case.

The recommendation to avoid using Forge storage for credential store is because it currently isn’t designed for it over being a general purpose key/value store. The data is encrypted at rest but there’s currently no detailed audit log, ability to rotate secrets, admin visibility of secrets/ability to revoke, setting additional encryption context, etc. I’ll update https://developer.atlassian.com/platform/forge/runtime-reference/storage-api-reference/ to call this out.

Good question, it’s only part of the problem for credential storage. In this case your app would need to take on the burden of encryption key management, for example what happens if (when) you need to rotate the encryption key?

8 Likes

Good point. Hadn’t thought of that. It might be an okay solution until Atlassian ships the proper solution, but it’s all hypothetical for now anyway. Very helpful information! :+1:

Great conversation, thank you all for chiming in :slight_smile: This productive conversation was exactly what we had hoped this report would spark. Forge seems very interesting from a technology perspective and I we can probably roughly imagine how it works, but I would definitely watch a presentation on the inner workings if one were to be made at a future event :slight_smile:

5 Likes

Thanks so much @tobitheo, this is an incredibly thorough review of the platform and it was an absolute joy to read. Feedback is a gift and this page is especially so.

We’re eagerly following up on each of these insights and will address them cohesively soon – just following up amongst ourselves internally on a few bits and pieces first :running_man:

Thanks again!

8 Likes

@CarrieMamantov I’d like to petition for this post to be considered for codegeist even though it was posted a few days too soon if an exception hasn’t already been made. This feels like exactly the sort of post you were trying to encourage with the feedback and story telling prizes.

14 Likes

@rmassaioli I would like to chime in on this point and say that in my opinion good Typescript support or even better a TS-first approach is hugely important for Forge and should be a no-brainer - please read on.

In my opinion, deferring this decision to the community to wait and see who shouts the loudest is a pretty weak move. Atlassian should take the lead with this, set the standards and decide on their own what it takes to write enterprise-level apps and is best for the community and the ecosystem.

If Atlassian decides static typing leads to better software and better apps it should provide curated high-quality TS templates and great documentation to showcase what a basic high-quality Forge app looks like. I am convinced that the community will adopt and follow the lead - which will be for the benefit of everyone. However, this mindset has to exist within Atlassian otherwise it will not work.

To your point on which language has more developers, I think the point is irrelevant. TS as a superset of JS will very likely always have fewer developers. Following the same argument, I guess every TS developer can also be counted as a JS developer.

It is clear that I am sitting in the TS camp and I am happy to chat more about this. I have sent @JordanKoukides an email a while back (in response to his post) with some more concrete points and observations regarding the current state of TS support. I will copy those (slightly modified) below for reference.

Forge DX - TS support

… I have shared this before with other teams but I am convinced that precise typings for Forge are one of the key features to building enterprise-grade Forge apps. For my taste, the current typings still have way too many “any” references. This is convenient for Forge platform developers but does not help the end-user at all. In many cases, precise types can often compensate for lacking documentation.

To give you a concrete example:

The method getContext in the custom UI bridge is currently typed as () => Promise<FullContext> with FullContext being typed as FullContext = {}. As a Typescript developer, I am getting almost no type information from this declaration. I just also noticed that the docs on getContext contain a deprecation warning regarding the extensionContext property being deprecated. Again, if there were precise types for FullContext, then the extensionContext property could be marked as @deprecated and all Typescript library consumers would know that they should fix their apps.

6 Likes

@tbinna :100:
I also encourage full support of TS in every aspect of the Atlassian Ecosystem, whether it is Forge, ACE or AtlasKit

3 Likes

My thoughts on this are different: Please support javascript first. That’s the more widely used language.

1 Like

Is TS vs JS the new Tabs vs Spaces?

Seems like it. I’ll just add in that I hope that Atlassian doesn’t pick a side…

And if you do - I’m going to pitch Perl as the language of choice for Forge.

1 Like

I agree… don’t pick sides. Just add vanilla JS & TS examples. It’s not uncommon to show examples in multiples languages in documentation.

If you do decide to go for Javascript, please also remove all ES6 import statements, given that this is not natively supported by NodeJS. Basically any code that requires a transpiler (looking at you BabelJS) should be banned.

On a more serious note: Atlassian has picked sides before. They chose to use React for AtlasKit. All examples are in React. And Forge UI Kit is also React inspired. So Atlassian has taken sides before. They can do it again.

Wait, did I miss a memo? Has this been resolved?