Resolution & Jexo Forge Hackathon Findings

UI Kit vs Custom UI vs Connect

Those are the two options you have, when you want to give your Forge app a user interface. There are some similarities between the two. Both use JSX as a syntax for writing these user interfaces and (depending on which framework is used for Custom UI) offer functionalities similar to React Hooks. And that’s where the similarities end.

UI Kit

UI Kit functions are executed in Forge’s FaaS environment and ultimately output a single tree of HTML elements that get put on the user’s web interface. There is no way of putting your own HTML in there somewhere. You have to use Atlassian’s UI Kit components. Half of these are wrappers that need to be used for certain modules, so the selection is pretty limited.

Interactivity Restrictions

There is also less interactivity. We mentioned that the code that generates the output is run inside Forge’s FaaS environment. It’s also important to know that these functions have a runtime restriction of 10 seconds. There is also no intermediate output before the function is fully done with all sync and async operations. This means that the loading screen from when the function has started executing can be no different from just before the last piece of content is loaded. Your function’s output will not be displayed, until all async calls are finished loading.

This also means there are no set Interval calls that continuously update your app every so often.

The only way to achieve an update of your UI, is by having the user click a button (or submit a form, which is also just clicking a button). Then your function will be called again and you can update your UI. But this also means there are no controlled inputs for text boxes, selects and other form components. No onClick, no onChange, no real-time validation, no interactivity. If you need interactivity, you have to use a button. If you don’t want that, you have to use Custom UI.

Good enough, sometimes

It is sufficient for simple UIs though: If you have a simple settings UI, like for a Confluence macro, or your app only needs to display some small amount of data in a table inside a Jira custom glance, then you are good to go with UI Kit. And it does have some advantages over Custom UI.

For example, the React-hooks-style API supports async functions everywhere. In React, this would wreak havoc, but since Forge is not exactly React, it’s okay to have an async initialization function in useState, since there is no incomplete state rendered anyway.

Simple to use

It is also easier to set up than the Custom UI. The file that generates the UI is simply a Forge function, which exports a render function. It lives in the same package as your other Forge functions and uses the same config files. A Custom UI is a bit more complicated.

Custom UI

A Custom UI resource basically consists of your own HTML, CSS and JS (and other types of files, like images). At deploy time, it is all bundled up and sent to Forge. When your module’s entrypoint is called, your HTML page is loaded inside a sandboxed iframe and delivered with a somewhat customizable, yet still restrictive content security policy. The experience is somewhat similar to a well-secured Connect app.

Bundling

How you get to that bundle is basically up to the developer. Typically, there will be another package.json somewhere inside or next to the Forge package’s one. This alone is, in our opinion, not great. The templates by Atlassian all use create-react-app. This seems fine, until you need that one feature, which requires you to eject. But you can also set up webpack or parcel or just write vanilla JS. Do note that external scripts and such need to be allow-listed in the manifest if you want to use them. The same goes for external media.

Custom UI Bridge

It is also not possible to make calls to external services from the frontend, but fortunately, there is a way to call your own backend hosted on Forge. This concept is called the Custom UI bridge, and it effectively lets a Custom UI iframe communicate with a Forge FaaS function. This function can then make calls to allow-listed external services, such as your own backend or third-party APIs you use. On top of that, Forge functions have access to credentials injected via environment variables, so your function can actually authenticate with that third-party API. Neat!

Of course this restriction about Custom UI not being able to call external services directly, means there is probably no simple inclusion of (Google) analytics or (Sentry) crash reporting snippets. All of the actual communication with the servers is possible via the bridge though, but has to be done manually.

Edit: As pointed out by @asridhara, you can in fact call external services if declared in your manifest. I had not seen the section in the docs about this. Sorry, my bad.

Routing

Another, slightly awkward restriction of Custom UI is the fact that if you have multiple modules using Custom UI, you need to do one of three things: Either declare multiple resources for the different modules and get your bundler to output all resources once per entrypoint. You can also have multiple, completely independent resource directories with separate package.json, bundler configs, etc… Or you declare one resource and do moduleKey-based routing. You can do code-splitting for that last one, but it will increase your load times. Neither of these is particularly elegant in my opinion. Ideally, we would be able to explicitly declare the file to use within one resource in the manifest. Currently, this always uses index.html, which effectively prevents us from using the regular entrypoint mechanism of bundlers, where for different entrypoints, different files are loaded.

Further Custom UI restriction

It also needs to be said that only some of the Javascript Connect API functions (exposed via window.AP) that are available in Connect iframes, have an equivalent replacement in Forge. But like most things, this area is improving steadily.

Unlike Connect, there are also fewer allowed sandbox parameters on the Custom UI iframes.

These permissions are not present on Custom UI:

  • allow-popups
  • allow-top-navigation-by-user-activation
  • allow-storage-access-by-user-activation

To be fair though, popups and top navigation, which is possible in Connect by using a regular HTML anchor with a target attribute, can be achieved by using the Custom UI bridge’s router Javascript API.

Atlassian Design

And for a UI library, you should probably look into Atlaskit or its reduced-UI pack version for matching styles with Atlassian if you’re using Custom UI. Of course that is not a requirement, but this makes your app feel more integrated with the host product.

Conclusion

So, which way should you go with your app’s UI? It depends.

Are you okay with it being simplistic because it’s a simple settings UI? Go with the UI Kit. It’s easiest to get started with. And if you already know React, you’ll feel right at home.

Do you need more interactivity but can accept living in a very sandboxed iframe? Go with Custom UI.

Do you need all the power that only an iframe unrestricted by a content security policy can wield? Go with Connect. For you, Forge is not there yet.

Do you need to reach deep into the inner workings of the page, the editor or the authentication mechanisms in Jira or Confluence? Hi P2 developers, fancy seeing you here :stuck_out_tongue_winking_eye: stick with P2 for a couple more years.

Atlassian Cloud is far from this level of integration.

Tooling & Templates

The centerpiece of tooling around Forge is the Forge CLI, which you need to install to interact with the Forge platform. In our opinion, it’s a good piece of software that makes a lot of the tasks that need to be done around building a Forge app fairly easy. The documentation is good and most functions work well enough. Some are a bit rough around the edges though.

Tunnel

The tunnel functionality in particular was pointed out several times as being very helpful with fast development cycles. It simulates being run inside Forge’s FaaS environment by running inside a Docker container. All requests to your app are being redirected to that container while the tunnel is active. That being said, we have hit some minor road bumps during our hackathon.

Reload it like it’s hot

If a function is being invoked while the tunnel is recreating the bundle, one of three things can happen: The old function answers the request, the tunnel waits for the new function to be ready and then that answers the request, or the tunnel errors out and the Forge app needs to be reinitialized. We would have liked for this behavior to be more consistent, ideally by letting all requests wait until the tunnel has reloaded, even if it takes a couple of seconds.

We have not experimented with connecting to our own dev server for Custom UI, but it sounds like a good solution to this issue for Custom UI resources.

Other issues

It would also sometimes happen that the tunnel gave a 402 response. In that case, it had to be restarted. This might also have been the underlying ngrok instance though.

Using a monorepo with workspaces can also be an issue. Due to the tunnel using Docker, the only directory that’s passed into the container is the one where the manifest.yml file is located, as well as all child directories. If you use a workspace, by default all dependencies in the node_modules directory are hoisted to the workspace root, where they are not included in the Docker container. Therefore, the nohoist option would have to be used, which is currently only supported by yarn.

Deploy & Manifest

The forge deploy command will bundle and upload your code to a given environment (development by default). It actually uploads the bundle to AWS S3, where Atlassian picks it up. Then it does some GraphQL calls. There are some limitations though.

Speedbumps

This is not a huge issue since the tunnel exists, but waiting for the deployment process to finish can be a bit tedious. It’s not an issue with bundle size, we think. It may take a minute or so regardless of it. We have not investigated further which part of the process exactly is causing the slowdown, but in general, you can take a short break when deploying.

It would also sometimes just randomly fail with some error. We discussed this and it definitely happened less often than during our previous Hackathon in mid-December, but it did still happen. Then the deployment would have to be repeated, which took time again. Better error messages would have been appreciated, even if not running in verbose mode. Because they generally do exist, but are not exposed to the user in non-verbose mode.

When an upgrade is necessary because something security-related like CSP exemptions or scopes changed, the CLI is nice enough to point out that instances with the app installed needed to be upgraded. This is nice! The upgrade process can be a bit tedious though, since for every command, the forge install --upgrade command needs to be called. This takes time, so it would be nice, if one could just upgrade all. Especially for those early development times, when these sections in the manifest get updated regularly.

We also found it weird that there are additional products in an instance like “Ecosystem” and “Identity” to upgrade.

Limitations

It can happen that you accidentally try to deploy a bundle more than the 50MB limit in size. We think that’s actually fine, but it did happen to us.

The manifest itself also has one limitation that seems unintuitive and arbitrary: Keys can not be longer than 23 characters and have to be unique across the whole manifest. This is not a huge issue, but some of us found it weird. We like to use long keys to make the identifiers more descriptive. And having “scoped” keys, such as using the same keys for a macro and the function that served that macro would have made it easier to navigate the file. Kubernetes does this and it makes it very obvious which resources belong together.

Templates & Documentation

Forge CLI provides a way to easily create new Forge apps. This is nice for first time developers, but it has some downsides for more experienced people.

TYPESCRIPT ALL THE THINGS!!!1!11!1one!!1one!eleven

We like to use Typescript for all of our Node and web-related projects these days. Javascript is nice for quick scripts, but as soon as projects become even a bit bigger, using Typescript just helps soooo much with so many things. From remembering which fields a class or an object has, to strict type enforcement so you never forget that something could be undefined or null, Typescript is here to help.

This is not to suggest that Typescript doesn’t have its downsides like missing built-in support for runtime type validation and a sometimes very verbose syntax, but it helped us so much in our development of cross-team medium to large projects. This is why the small number of Typescript templates surprised us.

There are two Typescript-based templates in the template library and neither of them include Custom UI. As discussed above, we realize that developers are free to choose their own way of ultimately getting a Custom UI resource bundle. But it would have been nice to see one anyway.

Looking at the node_modules/@forge directory, it seems like pretty much everything about the Forge libraries is already built in Typescript. Everything ships with typings and there are .d.ts files all over the out directories.

Why, then, is it not the other way around, having vastly more templates with Typescript than Javascript?

Outdated library versions (and sometimes templates and documentation)

Another issue with the templates is that, while they should ideally work out of the box, sometimes they don’t. This is usually due to outdated @forge libraries used in those templates. We have hit this issue several times in the past and have been able to learn from it. So it wasn’t a big issue for us, but it’s not a great experience for first-time forgers.

One solution to this could be upgrading all packages after forge creating a project. But this would have to be done regularly at Atlassian’s site as well, because we also stumbled upon outdated syntax Forge UI syntax in a template. This led to breakage when upgrading the @forge/ui package.

Some tutorials on the webpage had similar problems, also with scopes.

Don’t get us wrong here, we love the speed at which Forge is being developed and we appreciate that it can be a bit much to keep all templates up-to-date every time there’s a new version. But some of this could be automated and we think it might be a worthy time investment.

BYOT (Bring Your Own Template)

These points lead to us trying to design our own templates. We have created one UI Kit template and one Custom UI template. The latter also tries to use code splitting and internal module-key-based routing, but it’s not as good as it could be with a couple more days of work.

Of course, it would be nice to be able to bring your own template so “forge create” can use it. And luckily, the command has an optional CLI option for giving a template path. But internally, this is only resolved within the Atlassian Bitbucket namespace.

We have managed to break out of this namespace by using a directory traversal :innocent:, but then your directory names are screwed up and the Forge CLI has no problem walking all across your filesystem when you give it the right template name. This is not a huge issue, but we would appreciate it if the “forge create” code would be able to recognize non-Atlassian links and accept them as well.

You can find out our templates at

https://bitbucket.org/resolutiongmbh/forge-typescript-ui-kit-starter/src/master/

https://bitbucket.org/resolutiongmbh/forge-typescript-custom-ui-starter/src/master/

They are already outdated :sweat_smile: because of Atlassian changes.

Execution Environment & Storage

Forge functions run in a Node.js-like environment with a bunch of restrictions. These can make it hard to include some libraries and one should always be aware of this fact. There are also some restrictions to the data storage Forge provides.

Runtime Restrictions

You can find the runtime restrictions at Runtime and the platform quotas and limits at Platform quotas and limits. Especially note how the snapshotting works and which restrictions it has. This can save you some headache when debugging.

Most of these were not really an issue. Some were though and the error messaging is not always helpful.

Issues with libraries

We encountered some issues in the past when we tried to include libraries and this did not change for this hackathon. The error messages were sometimes not very helpful, for example the tunnel just crashes when our code or library code tries to include the crypto package:

Removing the reference to the crypto package does the trick. Deploying with snapshots enabled also seems to work. Our interpretation of this is that the tunnel environment differs from the actual execution environment, which is also not optimal.

Disabling snapshots actually fixes this issue, but that is not optimal either.

Snapshots

Snapshotting is used as a mechanism to accelerate function execution common in FaaS environments among others. Basically, a Node.js instance (modified to have the restrictions Forge has) is created and the Forge app’s file bundle is loaded, but no function is executed yet. For example, if you have a simple file where you import a library and export a function, those imports are executed (and everything related to those, like subsequent imports), and your exported function is parsed and created, but not executed. Then, a snapshot of the state of Node.js is taken and stored, hence the name snapshot.

When there is a request, Node.js is started again, but with this snapshot loaded back into memory. Then your function is executed. The advantage of this technique lies in the saved time: The initialization logic is only executed once for a code bundle, which means that any subsequent start of Node.js is faster and costs less money. But it also means that the UI is more responsive.

We don’t know exactly how this stuff works internally, but it does have some downsides: Environment variables are not available, and anything stored outside of functions can exhibit some weird behavior as it might be reset between function executions since a snapshot might be restored. This is not necessarily really problematic, but it is something that a developer needs to understand and be aware of when developing and especially when debugging.

As for cryptographic operations, we have found that jsrsasign works well for us. We used to check some signatures and create and validate some JWTs, all of which worked well with that library.

Forge App Storage

As with a lot of things related to Forge, Atlassian must not be envied for its balancing act between trying to prevent abuse of the platform and keeping it useful for developers with legitimately complex use cases that push those limitations.

It often feels like Forge’s app storage is a prime example of these considerations.

Background

App storage is a simple key-value store. As such, it is mostly well-suited for app settings, but only non-security relevant settings should be stored. Think of it as a replacement for the app properties API that exists in Connect.

The storage currently has some restrictions, for example it’s not possible to have relationships between entities or define schemas. Therefore, it can not currently be considered a replacement for a real DBMS that a Connect app might be connected to. But then again, for a lot of apps, that’s totally sufficient.

There is actually one functionality that makes it more interesting than a pure key-value store: You can run queries against and do paging, optionally filtering by key prefix. This, along with the ability to have 100 byte long keys, makes it possible to have a kind of hierarchical key structure, with each level separated by a dot.

As with everything Forge, app storage is also under constant development, so the information in this document might be outdated by the time you read this.

Security & OAuth

App storage accessible from both the frontend (in Custom UI) and (edit: I got this wrong, it is only available from the backend, a Custom UI needs to use a resolver to access storage) the backend (i.e. UI Kit functions and Custom UI resolvers). As such, it can not be considered safe from user influence and should not be used for storing security-related values such as application secrets, API keys, OAuth access tokens or passwords.

OAuth in general is not currently possible to do purely in Forge, but Atlassian has hinted that they are looking at an alternative that would also solve the secret and URL management problem.

Modules and Permissions

In this section, the supported manifest modules and scopes are discussed. Let’s start with the big one, which caused us the most trouble.

No Support for Jira Service Management

This is probably the biggest problem we have with Forge right now.

A lot of things related to Jira Service Management projects right now are broken: There are no events for those projects and its data can not be accessed via REST API. This is apparently due to Forge not supporting any of the scopes necessary to use JSM projects.

Some things do work though. For instance, adding a Jira project page module in your manifest will cause that page to also appear on JSM projects.

But with most other functionality missing, several of the projects we attempted at this hackathon were severely restricted.

Existing Scopes

Scopes in Forge apps are basically permissions on the host product the app requests when it is being installed. They give the installing person information about what the app needs access to. This concept is not new, however there are some details that are special to Atlassian.

Forge vs Connect scopes

One thing to keep in mind when developing Forge apps as opposed to Connect apps is the fact that Forge uses the OAuth scopes that were previously reserved for OAuth apps. They are much more granular than the Connect scopes and allow our apps to request only those permissions we need.

In security, this is called the principle of least privilege. This way of doing things is recommended because even if an attacker were to gain access to the credentials or the code of the app, they couldn’t do as much damage as they could, if the app had all the permissions to do everything.

Way to Improve Scopes

In general, we think the scopes are well implemented in Forge. They offer a nice mix of granularity without being annoying because every function has its own permission. The documentation around scopes is also very well written and understandable.

If Atlassian wanted to improve this system a bit further, we have some ideas.

Firstly, they could allow us app developers to add reasons why we request all scopes. The texts Atlassian shows around what a certain scope allows are nice, but they do not reflect exactly how this particular app uses this information and why we need it.

Also, there could be optional permissions. Similar to what smartphone operating systems have done for some time now with permissions like location access, some permissions could be optional to the functionality of the app. For example, a Jira app might offer a functionality to watch issue comments for certain attachments, but would also work without that.

Lastly, there is no way to distinguish between scopes needed for the app user and scopes needed when impersonating a user. An admin might not trust an app to edit comments using impersonation as it would make tracing edits impossible from an audit trail perspective.

User Impersonation

Forge functions sometimes have the option to run certain operations as the requesting user. This is similar to the Connect scope ACT_AS_USER, but does not really cover all the use cases.

Limited locations for user impersonation

A Connect app with the ACT_AS_USER scope could decide at any point to run any REST request as any user. This allowed for great flexibility in our code.

In Forge, only a very limited number of function locations are allowed to use user impersonation. Basically, we’re limited to UI kit handlers and Custom UI resolvers.

There is no user impersonation for host product event handlers where one would assume the app could run operations as the user that triggered the event. There is also no impersonation for web triggers, but we understand that there would be no reference user to run as. Which brings us to the next restriction with user impersonation.

Ability to select the user to impersonate

It is not currently possible to select the user to impersonate in the app’s code. If a request is being run as a user, then it is always the requesting user. But this is not always the correct user to run as.

From an app’s perspective, it might be necessary to be able to edit a comment as the author for example, even if it is in response to another user’s action.

We would argue that we could then, in most cases, simply run requests as the app user, but this is also not possible sometimes.

App User does not always have the right permissions

It is a known bug at the moment that app users don’t have the correct permissions for all the scopes. Combined with the aforementioned restrictions on user impersonation, this can make it hard to execute certain actions. It’s also hard to find this issue and if you don’t know about it, you could easily lose hours asking yourself why the project property API doesn’t work (for example).

There are workarounds noted on that issue though, so check them out to see if they fit your use case.

Modules

The modules you can define in the manifest basically describe which places in the host application your Forge app can hook into. They roughly correspond to modules in Connect apps, except they’re simplified in Forge.

Like Connect Modules But a Bit Different

For example, a Forge module for a Confluence context menu action can be declared in the manifest. When clicked, it will open a dialog which contains either a UI Kit page or a Custom UI. Besides the actual module definition, you need to define either your handler function or your Custom UI resource and then reference it in the context menu module.

In Connect, this would look a little bit different. You would define a module of type web item with a location of “system.content.action/secondary” for example, then point it to a dialog URL or a module of type dialog.

Finding this location requires the use of the web fragment finder app on the Marketplace, because there is no documentation on the available locations. Yes, the Atlassian documentation links to that non-Atlassian Marketplace app. This is not necessarily a really bad thing. Wittified, the vendor of the app, is a well-known and reputable company within the Atlassian ecosystem, but it does feel a bit strange that using the app is basically a requirement for developing Connect apps.

In Forge, the locations and their locations are somewhat less flexible, since a certain location always requires a specific type of container: page, dialog or inline dialog. But the Forge way does have the advantage of being well-documented and constrained in a way that leads to a consistent UX for the end user that we appreciated. Sometimes less is more.

Limited Selection, For Now

As nice as the new module definitions are, we were missing some locations in our Hackathon. For example, there is no way of putting a page in the Jira project settings at this point. We were able to work around this by defining a project page and using display conditions to only display the link to admins. Note that the conditions you need to use are different between team-managed and company-managed projects!

We also noticed that it’s not possible to add multiple Jira project pages. This feels a bit arbitrary, since more complex apps could have very good reasons for adding more than one project page. There are probably similar limitations on other modules, but we didn’t test them.

Another limitation we hit was the lack of a custom edit UI for Confluence macros. Currently, it’s only possible to define a UI Kit settings page, which looks similar to the Connect macros, when no custom edit UI is given, but rather a list of parameters.

All of this is certainly not optimal, but acceptable for now. Forge is under constant development and so far, the team at Atlassian has been very responsive when it comes to suggestions. It also resonates with what feels like the goal of Forge.

38 Likes