Forge versioning strikes again! A release triggered a major version update for seemingly no reason

Today we deployed an update of our Forge app that contained:

  • no manifest changes
  • no new or changed modules
  • no permission/scope changes
  • no egress changes

Just minor dependency bumps of our Custom UI npm modules, and a change to our resolver functions to check license.active instead of the deprecated license.isActive.

When deployed to our staging environment, it was correctly treated as a minor version

  • 3.0.0 → 3.1.0

To our surprise, when deployed to production it was treated as a major version

  • 3.0.0 → 4.0.0, and the CLI message indicated that permission or egress changes were detected (which there aren’t any).

When we run the command

forge version compare \
  --environment1 production --version1 3 \
  --environment2 production --version2 4

…we can see that the only “differences” are that the modules are returned in the opposite order (despite no changes to our manifest), and the icon property of our module has a different UUID in it (which appears to be generated at deploy time anyway, so not a change that we initiated).

That’s it. No mention of any differences to scopes, remotes, policies or egress.

Here’s a redacted summary version of the compare results, showing the order of the macro and confluence:globalSettings modules reversed, but no other changes:

Version 3.x [production]

[                                                                                                                                                                                                                                                                                                                                        
    {                                                                                                                                                                                                                                                                                                                                      
      "items": [                                                                                                                                                                                                                                                                                                                           
        {                                                                                                                                                                                                                                                                                                                                  
          "key": "example",
          "properties": {                                                                                                                                                                                                                                                                                                                  
            "adfExport": {
              "function": "export-example"
            },                             
            "categories": [                                                                                                                                                                                                                                                                                                                
              "...",
            ],         
            "config": {
              "icon": "resource:example-resource;example-icon.png",
              "openOnInsert": true,                                  
              "resource": "config-resource",                                                                                                                                                                                                                                                                                               
              "title": "...",
              "viewportSize": "xlarge"                                                                                                                                                                                                                                                                                                     
            },                        
            "description": "...",
            "icon": "https://icon.cdn.prod.atlassian-dev.net/{3 x UUID segments}/example-resource/example-icon.png",                                                                                                                          
            "resolver": {                                                                                                                                                                                        
              "function": "example-resolver"                                                                                                                                                                                                                                                                                              
            },                               
            "resource": "example-resource",                                                                                                                                                                                                                                                                                               
            "title": "..."
          },                                           
          "type": "xen:macro"                                                                                                                                                                                                                                                                                                              
        }                    
      ],                                                                                                                                                                                                                                                                                                                                   
      "type": "macro"
    },               
    { 
      "items": [
        {       
          "key": "example-settings",
          "properties": {            
            "displayConditions": {                                                                                                                                                                                                                                                                                                         
              "appIsLicensed": true,
              "isAdmin": true,                                                                                                                                                                                                                                                                                                             
              "isLoggedIn": true
            },                  
            "render": "native",
            "resolver": {      
              "function": "settings-resolver"                                                                                                                                                                                                                                                                                              
            },                               
            "resource": "settings-resource",                                                                                                                                                                                                                                                                                               
            "title": "..."
          },                                                                   
          "type": "confluence:globalSettings"
        }
      ],                                                                                                                                                                                                                                                                                                                                   
      "type": "confluence:globalSettings"
    }                                                                                                                                                                                                                                                                                                                                      
  ]

Version 4.x [production]

[
    {
      "items": [
        {       
          "key": "example-settings",                                                                                                                                                                                                                                                                                                      
          "properties": {            
            "displayConditions": {                                                                                                                                                                                                                                                                                                         
              "appIsLicensed": true,
              "isAdmin": true,      
              "isLoggedIn": true
            },                                                                                                                                                                                                                                                                                                                             
            "render": "native",
            "resolver": {                                                                                                                                                                                                                                                                                                                  
              "function": "settings-resolver"
            },                               
            "resource": "settings-resource",
            "title": "..."
          },                                                                   
          "type": "confluence:globalSettings"                                                                                                                                                                                                                                                                                              
        }                                    
      ],                                                                                                                                                                                                                                                                                                                                   
      "type": "confluence:globalSettings"
    },                                   
    { 
      "items": [
        {       
          "key": "example",                                                                                                                                                                                                                                                                                                               
          "properties": {   
            "adfExport": {                                                                                                                                                                                                                                                                                                                 
              "function": "export-example"
            },                             
            "categories": [
              "...",
            ],         
            "config": {                                                                                                                                                                                                                                                                                                                    
              "icon": "resource:example-resource;example-icon.png",
              "openOnInsert": true,                                  
              "resource": "config-resource",
              "title": "...",                                                                                                                                                                                                                                                                 
              "viewportSize": "xlarge"                                    
            },                                                                                                                                                                                                                                                                                                                             
            "description": "...",
            "icon": "https://icon.cdn.prod.atlassian-dev.net/{3 x UUID segments}/example-resource/example-icon.png",
            "resolver": {                                                                                                                                                                                                                                                                                                                  
              "function": "example-resolver"                                                                                                                                                                                                                                                                                              
            },                                                                                                                                                                                                                                                                                                                             
            "resource": "example-resource",                                                                                                                                                                                                                                                                                               
            "title": "..."
          },                                           
          "type": "xen:macro"
        }                    
      ],                                                                                                                                                                                                                                                                                                                                   
      "type": "macro"
    }                                                                                                                                                                                                                                                                                                                                      
  ]

Why does this happen?

Why are the items in a different order, and regardless, why would that constitute a new major version bump?

Does the order difference cause the major version bump, or was it due to some other unforeseen reason?

Look, I’m sure we can work around this with yet another forge version bulk-upgrade run, but the point is: we shouldn’t need to.

We didn’t change anything that would require extra consent from an admin. It should be a minor version that gets rolled out automatically.

We’re quickly losing faith in Forge here.

@rmassaioli this is versioning on Forge.

And remember the moment a major version is triggered it requires manual admin update.

Majority of admins never take that action, users get stuck on stale versions, app usage drops, admins cull apps with low usage, partners lose revenue.

@rmassaioli For reference, I have raised ECOHELP-126883 for this issue.

We have had initial engagement from the ticket assignee who asked us to provide a screenshot of the forge version compare command output (which as noted above shows no evidence of changes that trigger a major version).

We await further explanation as to what happened in this instance.

@scottohara Could be this? It was extended: https://developer.atlassian.com/changelog/#CHANGE-3166

I don’t think so, as we haven’t changed pricing either, or licensing (app has always been licensed).

Though as mentioned we did edit our license check code to look at license.active instead of license.isActive. I certainly hope that wasn’t the cause, as that would surely be a bug on Atlassian’s part if so.

Sorry wrong link… https://developer.atlassian.com/changelog/#CHANGE-3141

Ah right, that does sound like a plausible explanation.

But what a ridiculous thing to do in the first place?

“We’re making changes to our backend upgrade pipeline, so we going to make all of your releases major versions while we do it.”

Version numbering at this point truly has no meaning in Forge.

I’m trying to think how we will respond to this inevitable question from a customer at some point:

Customer: “Can you tell us what changed from 3.0.0 to 4.0.0?”

Us: “Nothing really. Just some internal changes, nothing user visible, no new features, no breaking changes.”

Customer: So why is it 4.0.0 then?"

Us: “Umm…”

What Atlassian are calling “version number” isn’t being treated like a version number at all (in the historical sense of that term, let alone semantic versioning). It’s more a “deployment sequence number”, which should not be public facing.

The exact same code build can be assigned a completely different number depending on:

  • which environment it is deployed to, and how many previous deployments that environment has had (iterating in development burns through version numbers at a high rate)
  • total user count for that environment (<50k = minor version, >50k = major version)
  • when it is deployed (before, during or after Atlassian’s work concludes on their upgrade pipeline)

Version numbers have meaning, both to the app vendor and to customers, who are trained to understand a major version typically implies significant or breaking changes.

Atlassian’s complete disregard for the importance of version numbers and the implied meaning that they carry is astounding.

Also (sorry to keep editing this post, but I keep thinking of more bizarre things about this whole change), why does the change mentioned in the changelog apply when user count is >50k? What does user count have to do with upgrades, which happen at an installation level? (I can assure you we don’t have 50k installs of our app)

What specifically in the upgrade pipeline is affected by total number of end users?

None of this makes any sense to me.