Forge update broke our app in 100s of customer instances and we can't do anything about it 😭

Ok, first thing first. I already raised this issue to the relevant teams at Atlassian. I even spoke to some of them at AtlasCamp and the issue was ignored.

So, why am I even writing this here? I don’t know, just felt like sharing it.

Background

We used Forge to build an app that now has a few thousand installations. We naturally used Forge Storage to store data. Our app fetched 50 items per query from storage, so far so good. Let’s say this was version v1.

This version was working fine. In order to apply some important platform changes, we had to release few major versions, so now we have versions v2 and v3.

The Problem

In December, Forge Storage had some performance issues, so they decided to add 20 limits on query result. It returns an error if you query more than that.

Error: Page limit value (50) should be >=1 and <=20

This change broke our app, especially V1. The change itself is not a bad thing even though it is not backward compatible. However, combination of this with Forge versioning is a HUGE issue.

Note: a major version has to be manually upgraded by admin.

There are still 300 customers on broken V1 but we cannot do anything:

  • We cannot update V1 to apply fix as Forge does not allow updating old versions
  • We cannot deprecate or mark V1 as legacy version
  • We cannot reach out to customers because the app was free and we don’t have contact info
  • There is no in-app way to inform users about new versions

Customers are totally confused why app started showing blank screen. Some eventually find a way to contact us, and we help them upgrade. Others leave negative review in the Marketplace. This whole thing has been very painful for us.

Conclusion

  • Making query backward compatible and just returning 20 results with pagination would have avoided it. I feel like Forge team didn’t understand the huge concern we raised. I guess they track Forge Storage metrics, though our app alone is raising 100s of errors every day because of the above.
  • For the love of God, please do something about issues related to old versions (I know there are multiple requests in Forge issue tracker)
  • For developers, make sure you have a section in the app with just plain HTML that does not depend on any Forge features and add support links there. That way, customers can at least contact your support, when something like above happens.

Related issues:

32 Likes

Permission scopes are also a mess. If you built a Forge app more than 12 months ago they’ve made multiple unannounced changes on scope requirements and naming.

So in addition to unfixable errors from old versions, you end up with unfixable permission errors everywhere. Bump the @forge/cli by a single version and the linter will start throwing new errors. Repeat a month later and they’ve reverted those changes.

There’s also something funky going on with permission scopes between the bizarre move of creating separate incompatible UI Kit 1 and UI Kit 2 packages. Has broken many of our apps. The lack of foresight has clearly created an overly complex developer platform.

eg when I built apps using UI Kit the useContentProperty sounded like a good idea:

import ForgeUI, { Button } from '@forge/ui';
import { useContentProperty } from '@forge/ui-confluence';

In UI Kit 2 this is how you use the same hook:

import { Button, useContentProperty } from "@forge/react";

I think whoever is trying to juggle this tangle of incompatible @forge/* packages is just as confused as everyone else, and has deployed scope permission requirements that are not compatible with UI Kit 1.

I’ve been putting off migrating to UI Kit 2 because the crystal clear message from Atlassian (eg v2 Confluence API) is to never migrate until the very last second. But at this point it appears they’ve unofficially dropped support for UI Kit 1 and will continue to break those apps in the future.

[I could go on for literally 12 hours on other issues but will stop there while it’s still somewhat related to OP]

1 Like

I’d like to add that this whole mess even encourages vendors to build less secure apps. To avoid major releases at every cost, it is feasible to launch apps with more scopes and egress permissions than would actually be required.

10 Likes

Please vote for [FRGE-720] - Ecosystem Jira if you think this should be implemented on the Forge platform

4 Likes

From my experience customers are really lazy in manual updating. We released a major version in July 2023 and still 40% (> 500 sites) of our installations are running in the old version.

At least, there must be ways to “encourage” site admins to update!

2 Likes

Yes, that’s also our experience.

I don’t think they are lazy though, they just don’t know a new version exists and needs to be manually updated.

3 Likes

I’m not sure if this is relevant for Forge apps, but maybe my experience can help. Last year we went through something similar where we added a scope to our CONNECT app, with the expectation that customers would need to manually update the version, accepting the new scope before the app could be updated.

Before the new version with the new scope was released, we pushed a change to our app to include a dynamic banner across the major pages, whereby if the installed -AC Cloud version was NOT the latest version as reported in the Marketplace listing (there’s an API for that somewhere) then the banner was shown advising the customer to upgrade with a handy direct link to the in-app UPM. If they were on the latest version then no banner is shown.

Maybe that is something that can be considered for a Forge app too?

2 Likes

@RyanRules1 there’s no ability to get version data with Forge apps: No API available to get app version for Forge Apps?

If there was the ability we could very easily show the users an “update app to continue” message to block all functionality until they updated.

Although since Forge app versions are bundled and immutably held for ransom on Atlassian’s slow infra, you could only do this for future app versions. So it still needs an external trigger option in the Developer Console.

[edit] They’re finally listening: An update on managing App Versions on Forge

4 Likes

Hi all,

I just want to let you know we are actively working on a solution for this problem and if you want to contribute to or discuss the solutions, please comment on my Forge post here.

Thanks
James

3 Likes