How to use Forge CLI in a CI environment?

Re new topic - Good point, I’ll do that. Thanks.

I don’t have an answers but I’m running into the same issue. Using Ubuntu under the Windows Subsystem for Linux.
Strangely everything was working fine with Forge 1.5, but I started getting the Keytar error after upgrading to 1.6. The issue continued to happen even after uninstalling 1.5 and re-installing 1.6, so I wonder if it’s a change in a new version of a dependency.

2 Likes

Maybe I do have an answer, I looked in the code and saw that there is a fallback credentials option if

require('keytar')

fails, so I went a deleted the keytar folder from my global node_modules folder (.nvm/versions/node/v14.17.0/lib/node_modules/@forge/cli/node_modules/ for me).
Obviously not the nicest way to do things but it seems to work.
Matt

1 Like

Hey @matthew.grover,

I’m on the Forge team and I believe I know how to make this work now.
So the trick is to:

  1. Set usage analytics (as @sven.schatter has mentioned above, although I personally think it’s worth it to enable it: forge settings set usage-analytics true)
  2. set FORGE_EMAIL and FORGE_API_TOKEN environment variables. I use BB Pipelines now and I have configured these variables in BB Pipelines UI. This was enough for Forge CLI to run forge deploy, forge install and other commands.

Let me know if it doesn’t help.

5 Likes

Awesome!

So skip the forge login and stuff and just continue to use the Api token? Can that be official through documentation (since I suspect a lot of folks will be setting up deployment pipelines like me. Also, that way the functionality doesn’t disappear without us having anything to point to :slight_smile: ).

But that got me going for now on this hurdle. Thank you!!!

/Daniel

3 Likes

@danielwester that’s a valid question. The functionality from my reply won’t disappear. However, Forge team has recently confirmed that the recommended way to use Forge CLI in the CI is this:

forge login -e <email> -t <token> --non-interactive

There will be a changelog entry about this soon.

3 Likes

@Dmitrii,

Is there anyway we could make keytar an optional dependency? For me, the version (2 major versions behind current) required by the Forge CLI won’t compile for me on 2 different flavors of Linux. I’ve tried all kinds of things to get the libsecrets-1 library, setting compile paths, and whatnot. While I understand the day-to-day convenience, it seems overly aggressive in the CI/CD environment when that secrets management is just going to be destroyed anyway.

5 Likes

For me it looks like it’s already an optional dependency. That is, I’m seeing the node-gyp errors followed by an npm warning message that it’s skipping optional dependency keytar.

2 Likes

Unfortunately, the same error still persists even after doing exactly that.

Keytar error detected: Cannot autolaunch D-Bus without X11 $DISPLAY
Something went wrong while accessing Secret Service API.
1 Like

Hi @ibuchanan

The keytar is an optional dependency of the CLI. So even if it cannot compile in your environment, this should not block the usage of the CLI.

You might see warnings and errors during the installation step, but the CLI will be usable as the keytar codepath will be ignored when running commands.

Hope this clarifies things.

2 Likes

I can confirm @XavierCaron’s assessment, though there are some gotchas that are easy to run into, here’s our current recipe:

  1. do not depend on @forge/cli via package.json (we had it as a dev dependency, which worked up to 1.5.0 and somehow broke in 1.6.0)
  2. ensure to run npm install again before committing package-lock.json after removing any @forge/cli dependency as per 1), otherwise keytar remains configured based on your local interactive environment
  3. instead, install @forge/cli via a separate step in your build environment to ensure it is configured w/o the interactive features (of course, you can optimize build times via a custom build image with the @forge/cli preinstalled) - the mentioned warnings can be ignored, or better yet avoided by ignoring the optional keytar dependency (thanks @remie):
    • npm 6.x: npm install @forge/cli@2.0.1 --no-optional
    • npm 7.x: npm install @forge/cli@2.0.1 --omit optional
  4. ensure to add the --non-interactive flag to applicable commands like forge login and forge install
7 Likes

To add to this, if you remove forge-cli from your package.json and package-lock.json files, then this will work for a basic deploy:

name: Deploy to forge app to atlassian cloud site development environment on push

on:
  push:
    branches: [dev]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Build the source code for deployment
    steps:
      - uses: actions/checkout@main
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm install @forge/cli@2.0.1 --no-optional
      - run: npm run static:install
      - name: disable analytics
        run: npx forge settings set usage-analytics false
      - name: Deploy to atlassian cloud site
        run: |
          npm run static:build
          npx forge login --email ${FORGE_EMAIL} --token ${FORGE_TOKEN} --non-interactive
          npx forge deploy
        env:
          FORGE_EMAIL: ${{ secrets.FORGE_EMAIL }}
          FORGE_TOKEN: ${{ secrets.FORGE_TOKEN }}

Atlassian, the fact this took 3 people hours to figure out and we have no meaningful response from anyone on this thread for a month shows how GA Forge really is. I can’t believe I wasted my entire afternoon on this.

Major Kudos to Remie and Steffen for spending the time to help me un@#$% this.

8 Likes

This is quite scary, that I need to use my main account credentials in CI. Deployment keys per app would be much more secure and practical.

2 Likes

Thanks everyone for helping identify Forge’s CI/CD capabilities. I think CI/CD is an important topic and given Forge’s unique hosting setup, we need to provide a dedicated guide on Forge CI/CD. I’ve created FRGE-813: Inadequate documentation of Forge CI/CD support to request this.

1 Like

Thanks @dmorrow, appreciate the initiative! The lack of CI/CD guidance has also just been discussed (and acknowledged) in Join us for a Developer AMA with Tim Pettersen - #9 by sopel, where I also referenced a recent Atlassian blog post that seems to be a good starting point for a guide (Bitbucket Pipelines though, ideally a guide would also cover GitHub indeed):

  • […] I just stumbled over Damien Lauberton’s recent and comprehensive blog post How to Configure CI/CD for an Atlassian Forge App […], and it has neither been announced in the community nor referenced from the Forge docs yet, maybe you could derive (and maintain) a tutorial from it?
1 Like

Hi @sopel ,

Thanks for these references and in particular the blog post.

Regards,
Dugald

We’d gone back to v4.5.2 and then converted over to use FORGE_EMAIL and FORGE_API_TOKEN env variables instead of forge login as described in the getting started. This worked.

After the last few posts here I decided to try the latest cli version again and re-add the login using the token option but for me this still does not work and I get the following error:

Error: The CLI couldn’t securely store your login credentials in a local keychain. Ensure you have libsecret installed. If a local keychain is not available, use environment variables before trying again. See https://go.atlassian.com/dac/platform/forge/getting-started/#log-in-with-an-atlassian-api-token for more.

Simply removing the forge login and using the environment variables seems to work fine though so thats what we’re going with - and per the getting started docs that seems to be the prescribed way.

1 Like

Hi folks,

In response to this topic, a new article on continuous delivery has been created. The best place for comments on the guide is probably the topic it was announced in.

Regards,
Dugald

Adding an updated GH Actions workflow in case people run across this thread.

Uses node 18, caching, supports custom UI, error logging, and is hopefully written simply enough that it’s easy to understand.

name: Deploy to forge app to atlassian cloud site staging and prod environments on push

on:
  push:
    branches: [main, production]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Build the source code for deployment
    steps:
      - uses: actions/checkout@main
      - name: Use Node.js 18.x
        uses: actions/setup-node@v4
        with:
          node-version: 18
          cache: 'npm'
          cache-dependency-path: |
            package-lock.json
            static/admin-page/package-lock.json
            static/jsm-portal/package-lock.json
      - run: npm ci
      - run: npm install -g @forge/cli@latest --no-optional
      - name: Build admin page
        run: |
          cd static/admin-page
          npm install
          npm run build
      - name: Build jsm portal page
        run: |
          cd static/jsm-portal
          npm install
          npm run build
      - name: disable analytics
        run: forge settings set usage-analytics false
      - name: Deploy to atlassian cloud site
        run: |
          branch=${{ github.ref }}
          if [ $branch == "refs/heads/main" ]
          then
            forge deploy -e staging
            forge install --upgrade --non-interactive --site abrega.atlassian.net -e staging -p jira
          elif [ $branch == "refs/heads/production" ]
          then
            forge deploy -e production
          fi
        env:
          FORGE_EMAIL: ${{ secrets.FORGE_EMAIL }}
          FORGE_API_TOKEN: ${{ secrets.FORGE_TOKEN }}
      - name: Archive npm failure logs
        uses: actions/upload-artifact@v2
        if: failure()
        with:
          name: npm-logs
          path: ~/.npm/_logs

2 Likes

Thank you, @BPB. Already spent quite a few github actions runs until I landed on a text that would lead me to this thread.

Thanks for posting the full workflow yml, it saved hours, I’m sure.