Discussion: Connect and Forge together

Hello everyone!

This is the discussion thread for the Connect and Forge together session from Atlassian Developer Day 2021. You can get to this discussion thread by following the go.atlassian.com/connect-and-forge link in the session content.

@jhazelwood and I (@HeyJoe) are here to discuss any questions, feedback or concerns you have may have about the content we covered in this session.

10 Likes

Just watched the session on-demand - thanks for the great talk.

Something that seems to have been quickly glossed over in the video, and I’m not sure if that is because its still yet to be figured out - you mentioned the two pathways:

  1. Rewrite
  2. Connect-on-forge

The remainder of the session & demo naturally focussed on the second pathway; and nothing further was said about the rewrite pathway except (paraphrasing):

You rewrite your entire Connect app as a Forge app and we’ll help you migrate

Are you able to share any more about the migration aspect?

We have some simple connect apps (built on atlassian-connect-express) that we’re intending to rewrite from the ground up as Forge apps.

However these apps (collectively) have thousands of active installs, with the install data sitting in our own remotely hosted Postgres databases (i.e. the AddonSettings table). We don’t store any other customer data other than this table.

Assuming that in future there will be a way to update our Marketplace listings to point to a Forge version of the app instead of the existing Connect version, what will be the process for migrating the AddonSettings table to Forge? (or is the expectation Atlassian will manage this, and we can simply discard this data once the migration of all active customers to the Forge version is complete?)

Uncertainty about the migration process is the main thing that has prevented us from exploring Forge fully.

Thanks in advance.

7 Likes

Hi @scottohara,
Thanks for your feedback and interest.

a way to update our Marketplace listings to point to a Forge version of the app instead of the existing Connect version

We’re just getting started spiking and investigating this, so if you have any ideas on how you’d like this to work on your side, we’d love to hear them.

what will be the process for migrating the AddonSettings table to Forge? (or is the expectation Atlassian will manage this, and we can simply discard this data once the migration of all active customers to the Forge version is complete?)

In a scenario where you’ve re-written your app to purely use forge, the auth data in AddonSettings (i.e. shared secret keyed against clientKey) will become obselete, because Forge authentication works differently.

If you’re talking about other settings data that specific to your app, that’s not so straightforward, and we’ve had less chance to think about migrating that would work, but my current thinking is that down the track we’ll provide a way to access forge storage from your app server, at which point your app could push the relevant settings data in there (the alternative being, perhaps, for the Forge side to pull the data out via an api exposed from your app server).

We’d also love to hear your ideas on how you’d expect / like to see it work because we haven’t built it yet :slight_smile:

Thanks again,
James Hazelwood

6 Likes

Thanks for the additional info @jhazelwood .

We’re just getting started spiking and investigating this, so if you have any ideas on how you’d like this to work on your side, we’d love to hear them.

I guess that’s my real concern.

Forge was first announced to the community nearly 2 years ago at Atlas Camp 2019, yet there seems to be no clearly articulated vision for how even the simplest of Connect apps (let alone the large corpus of all Connect apps) can move to Forge.

Imagine a simple Hello World Connect app, built on atlassian-connect-express:

  • It is a “Paid via Atlassian” Marketplace app
  • It does nothing but render “Hello World” somewhere in the host product.
  • It has no other functionality, and no other data storage.
  • But for whatever reason, let’s assume that this app is wildly successful in the Marketplace and has thousands of active, paying customers.
  • The vendor understands the benefits of Forge, and is prepared to put in the effort to rebuild the app on Forge
  • But they want the transition from Connect → Forge to be seamless and transparent to their existing customers. (After all, Connect vs Forge is an implementation detail that end-customers shouldn’t know or care about).

It would be very helpful for Atlassian to provide, as a strawman, some detail (even at a very high level) on what your thinking is around simple migrations like this.

We’d also love to hear your ideas on how you’d expect / like to see it work because we haven’t built it yet

As a vendor:

  1. I’d like to build & test the Forge version of my app independently from my Connect app
  2. When ready, I’d like to link my existing Cloud Marketplace listing to both the Forge and Connect versions
  3. I’d like to have some sort of canary/piloting mechanism, so that I can gradually rollout the Forge version to some customers (with the bulk of customers remaining on the Connect version)
  4. At some point, I’d like to toggle the Forge version to be the default for NEW subscribers
  5. At some point, I’d like to toggle the Forge version to be the default for ALL subscribers
  6. I’d like the migration of customer registrations from Connect → Forge handled automatically for me by the platform (with no disruption to billing cycles, subscription terms etc.)
  7. I’d like to have reporting/monitoring tools to show how many customers are on the Connect vs Forge versions, so that I know when I can deprecate the Connect version and tear down all of the supporting infrastructure to save costs

Atlassian have been very clear that Connect is not going away, but equally the messaging is very strong that Forge is the future. We want to be part of that future.

6 Likes

It would be very helpful for Atlassian to provide, as a strawman, some detail (even at a very high level) on what your thinking is around simple migrations like this.

Poor choice of words on my part; to say “we just started spiking and investigating” makes it sound like we haven’t thought about it until now, which isn’t the case.

Our plan, still subject to change, is for a migration to look like this:

  • Forge-ify your app, and test internally and with interested customers (most likely using a separate instance at first).
  • When you’re ready to release, create a new version in your Connect app marketplace listing, but select the forge app as the new “version”
  • The forge version is rolled out to new and existing subscribers (preserving the app key, installations, billing cycles, etc).

The last point has some open questions around it such as whether we can offer a canary/piloting mechanism of some sort, and whether to require the customer re-consent for every such migration, or only if the scopes change, that sort of thing.

Hope that’s helpful, and thanks so much for the list you’ve supplied; that’s really valuable.

3 Likes

Hi, I want to create a Forge app but not all of the functionality is there yet. So development I need to do a connect and Forge app. I have a few questions around this

  • Does the Forge app need to be released to the Marketplace (as private) before I can link it to Forge?
  • Will the connect app get all of the lifecycle events when the app is installed?
  • What is the best way to communicate between the connect and forge parts of the app?
  • Is there an example app that uses connect and Forge?

Thanks
Paul

This is exactly the sort of information I was hoping for. Thanks very much @jhazelwood !

Hi, Paul,

I need to do a connect and Forge app

“Connect and Forge” apps are still in alpha so I’d recommend sticking to Connect for now if you want to build a production app that can’t be built using exclusively Forge features.

Will the connect app get all of the lifecycle events when the app is installed?

You can declare install and uninstall handlers (but not enabled or disabled).

What is the best way to communicate between the connect and forge parts of the app?

That’s a great question. We don’t offer an easy way to facilitate such communication, yet. If you’d still like to try, though, you could experiment with sharing data via entity properties, with web triggers or with custom authentication between forge functions and your app server over HTTP.

Is there an example app that uses connect and Forge?

Besides the worked example from the talk, there’s this one linked from the Build a Connect on Forge app page.

Hope that helps.

2 Likes

@jhazelwood thanks for those answers.

It was probably said in the talk (but I have forgotten) but when is connect → Forge going to be ready. I don’t want to use just connect because I need the internal storage (and data residency) of Forge but its event system isn’t quite then yet so I need a hybrid

I don’t have the slides from the session handy, but I think the ballpark figure was the first half of next year because there’s a fair bit to do.

As for data residency, we are working on supporting that in Connect, but it will put more of the onus on the app developer.

I don’t have the slides from the session handy, but I think the ballpark figure was the first half of next year because there’s a fair bit to do.

The dates weren’t in the session because they’re fairly rough estimates at the moment, but yes, ideally we are aiming for early 2022 as the current target.

Bugger, that far away. I was hoping this would be a workaround for not getting the deleted comment text in delete comment events. Is there any other way I can get the deleted comment text for a forge app without using Forge and Connect?

I’m afraid I don’t know when we’re aiming to address that, sorry. I’ll ask around.

@SamPurchase has directed me to this discussion between the two of you, and the Forge Ticket that was raised as a result. That about covers it; I don’t have any more to add besides that, sorry.

You’ve already thought of the one possible workaround that occurred to me: keeping track of everything so you already have the comment content stored when the deletion event comes through (and you also thought of the main drawbacks: you are likely to run into storage limits, and it’s a lot of customer data to be storing).

1 Like

@jhazelwood , thanks for that. Yeah, I have been asking around trying to find ways around this. Thought someone might come up with a way that I hadn’t yet thought of. At least I now have one date, the Forge connect bridge.
Thanks
Paul

Hi again @scottohara and other interested developers, we’ve finished investigating and planning and have begun in earnest on:

a way to update our Marketplace listings to point to a Forge version of the app instead of the existing Connect version

Our planned first step is to make it possible to set the connect key of your forge app in your manifest, like so (in future, this will become fixed to the marketplace key at listing time):

app:
  id: abc-123
  connectKey: com.pluginsquad.awesome-planning-poker

This can vary per-environment, but there’s only the one slot in the manifest so you’ll need to adjust it before deploying to a different environment.

As a next step, we want to add support for installing a Forge app environment over the top of a classic connect app with a matching key.

Please share any feedback or questions on the plan, or let us know if you’re interested in trying an early version of this feature when it’s ready!

Cheers,
James

4 Likes

Would it be possible to make the connectKey be a hash? Since the manifest in Forge does a lot more than in Connect (ie it declares the backend resolvers etc) - having multiple manifests floating around would seem kinda scary - especially since it would lead to “oh crap I forgot to declare the new function in the production manifest” and we would literally be testing in production…

My suggestion would be the connectKey be a hash in the manifest (you can always rewrite it when things are deployed) of something like:

app:
   id: abc-123
   connectKeys:
       production: com.pluginsquad.awesome-planning-poker
       development: com.pluginsquad.awesome-planning-poker-dev
1 Like

Hi, it’s certainly possible, and we considered something like this. We were hesitant, however, because are other things that we might want to make per-environment, like remotes[0].baseUrl and environment variables, so we should probably give more thought to how to introduce the concept of per-environment config before we introduce this new concept.

Would you want to also have a per-environment baseUrl, and do you have any thoughts on what a format for that would look like?

As for the current proposed design, the usage pattern I imagined wasn’t multiple manifests, but rather some kind of script that wrapped the forge deploy command and subbed the correct connectKey and baseUrl into the manifest file before running the deployment. I agree that having the official tool / manifest format take care of it would be nicer, though.

I’ll take this back to the team responsible for the manifest format and discuss it as well.

Thanks for the feedback!

I’ve only just discovered this thread, as I’m not actively developing on Forge yet (I had a look last year, but I’ve been focused on addressing breaking changes over on connect this year).

A shout out to @HeyJoe & @jhazelwood as your Developer Day presentation Connect and Forge together | Developer Day 2021 - YouTube is excellent.

I noticed it only has 27 views but should be essentially viewing for everyone developing Connect right now.

4 Likes

Hi everyone,

We would like to update you on some of the changes we’ve implemented to support migrating your Connect on Forge apps. These changes are part of an alpha release.

In order to allow you to replace your Connect app with a Forge version of the app, we have introduced an app.connect section in the manifest file so that you can reference Connect portions of your app. In particular, the app.connect.key will give you the ability to control the app key for testing your Connect on Forge app, and the app.connect.remote key will allow you to reference the baseUrl of your existing Connect app.

For example:

app:
  connect:
    key: hello-world
    remote: connect-app-server
remotes:
  - key: connect-app-server
    baseUrl: https://hello-world-app.example.com

It’s important to note that the app.connect.key of your Forge app’s production environment should match the existing key of the Connect app you wish to replace. For development and staging, we recommend using the production key with a suitable suffix, e.g. hello-world.development . The tutorial linked below offers one way to juggle the different environments’ keys.

At this stage, changing the key is not supported, but support is coming soon.

If you have an existing Connect on Forge app you wish to continue using, you will need to set app.connect.key of its environments to match its existing (randomly-generated) key.

For more detailed instructions on how to configure the app.connect section, please see: https://developer.atlassian.com/platform/forge/build-a-connect-on-forge-app

As always, we invite you to share any feedback or bugs so that we can improve the migration journey from Connect to Forge.

Thank you :smiley:

7 Likes