How to use Forge CLI in a CI environment?

Hi!

I’ve trying to run Forge CLI in our GitHub workflows but I keep receiving the following message:

Error: Keytar error detected: Cannot autolaunch D-Bus without X11 $DISPLAY
Something went wrong while accessing Secret Service API.

As this is running in the CI, there is no interactive prompt available. This happens both with the login and lint commands (I haven’t tried others yet).

I’ve also tried executing the login:ci command and setting the output token to the env variable FORGE_CI_TOKEN, but the result is the same.

Any ideas?

3 Likes

Have you tried the --non-interactive flag already? This is what we use in Bitbucket Pipelines:

forge settings set usage-analytics false
forge login --email $FORGE_EMAIL --token $FORGE_TOKEN --non-interactive

Cheers,
Sven

1 Like

Thanks for you reply Sven.

I already had the usage-analytics setting set to false, but I missed the non-interactive option from the login.

Unfortunately adding it yields the same result (I’ve also added the --verbose flag to get more details):

Run npm install -g @forge/cli
  npm install -g @forge/cli
  forge settings set usage-analytics false
  forge login --email *** --token *** --non-interactive --verbose
  shell: /usr/bin/bash -e {0}
/opt/hostedtoolcache/node/12.21.0/x64/bin/forge -> /opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/out/bin/cli.js

> keytar@5.6.0 install /opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/node_modules/keytar
> prebuild-install || node-gyp rebuild


> @forge/cli@1.3.0 postinstall /opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli
> node -e "process.exitCode = fs.existsSync('./out/bin/postinstall.js')" || node ./out/bin/postinstall.js

+ @forge/cli@1.3.0
added 792 packages from 456 contributors in 11.294s
✔ Successfully set usage-analytics as false.

Log in to your Atlassian account
Press Ctrl+C to cancel.

Logging you in...

  ▶️  GraphQL https://api.atlassian.com/graphql
Query: {me {user {name accountStatus accountId}}}
Variables: {}
  ◀️  GraphQL
Request ID: 71d3774d706a9ccc
Result: {
  "me": {
    "user": {
      "name": "Rodrigo Nogués",
      "accountStatus": "active",
      "accountId": "xxxxx"
    }
  }
}

Error: Keytar error detected: Cannot autolaunch D-Bus without X11 $DISPLAY
Something went wrong while accessing Secret Service API.

Error: Keytar error detected: Cannot autolaunch D-Bus without X11 $DISPLAY
Something went wrong while accessing Secret Service API.
    at KeytarPersonalApiCredentialsStore.wrapKeytarError (/opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/node_modules/@forge/cli-shared/out/auth/personal/token-store-keytar.js:90:15)
    at /opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/node_modules/@forge/cli-shared/out/auth/personal/token-store-keytar.js:67:32
    at async LoginCommand.execute (/opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/node_modules/@forge/cli-shared/out/auth/personal/login.js:15:9)
    at async CommandLineUI.displayProgress (/opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/node_modules/@forge/cli-shared/out/ui/command-line-ui.js:44:28)
    at async loginCommandHandler (/opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/out/command-line/register-authentication-command.js:32:38)
    at async Command.actionProcessor (/opt/hostedtoolcache/node/12.21.0/x64/lib/node_modules/@forge/cli/out/command-line/command.js:74:32)
Error: Process completed with exit code 1.

Regards,
Rodrigo.

2 Likes

@sven.schatter, is the forge install step part of your pipeline?

I am trying to build & deploy a forge app from a pipeline and I notice that there is no --non-interactive flag on the forge install command. So, even though I pass through the --site, --environment, --product, --confirm-scopes, and --upgrade flags the install command still prompts me for confirmation.

Shouldn’t the install support the --non-interactive flag as well?

2 Likes

Hi @jeffryan,

We’re only using forge deploy in our pipeline so I don’t have any idea about that, sorry. Would probably make sense to create a fresh topic for this. :slight_smile:

Cheers,
Sven

Re new topic - Good point, I’ll do that. Thanks.

I don’t have an answers but I’m running into the same issue. Using Ubuntu under the Windows Subsystem for Linux.
Strangely everything was working fine with Forge 1.5, but I started getting the Keytar error after upgrading to 1.6. The issue continued to happen even after uninstalling 1.5 and re-installing 1.6, so I wonder if it’s a change in a new version of a dependency.

2 Likes

Maybe I do have an answer, I looked in the code and saw that there is a fallback credentials option if

require('keytar')

fails, so I went a deleted the keytar folder from my global node_modules folder (.nvm/versions/node/v14.17.0/lib/node_modules/@forge/cli/node_modules/ for me).
Obviously not the nicest way to do things but it seems to work.
Matt

1 Like

Hey @matthew.grover,

I’m on the Forge team and I believe I know how to make this work now.
So the trick is to:

  1. Set usage analytics (as @sven.schatter has mentioned above, although I personally think it’s worth it to enable it: forge settings set usage-analytics true)
  2. set FORGE_EMAIL and FORGE_API_TOKEN environment variables. I use BB Pipelines now and I have configured these variables in BB Pipelines UI. This was enough for Forge CLI to run forge deploy, forge install and other commands.

Let me know if it doesn’t help.

5 Likes

Awesome!

So skip the forge login and stuff and just continue to use the Api token? Can that be official through documentation (since I suspect a lot of folks will be setting up deployment pipelines like me. Also, that way the functionality doesn’t disappear without us having anything to point to :slight_smile: ).

But that got me going for now on this hurdle. Thank you!!!

/Daniel

3 Likes

@danielwester that’s a valid question. The functionality from my reply won’t disappear. However, Forge team has recently confirmed that the recommended way to use Forge CLI in the CI is this:

forge login -e <email> -t <token> --non-interactive

There will be a changelog entry about this soon.

3 Likes

@Dmitrii,

Is there anyway we could make keytar an optional dependency? For me, the version (2 major versions behind current) required by the Forge CLI won’t compile for me on 2 different flavors of Linux. I’ve tried all kinds of things to get the libsecrets-1 library, setting compile paths, and whatnot. While I understand the day-to-day convenience, it seems overly aggressive in the CI/CD environment when that secrets management is just going to be destroyed anyway.

5 Likes

For me it looks like it’s already an optional dependency. That is, I’m seeing the node-gyp errors followed by an npm warning message that it’s skipping optional dependency keytar.

2 Likes

Unfortunately, the same error still persists even after doing exactly that.

Keytar error detected: Cannot autolaunch D-Bus without X11 $DISPLAY
Something went wrong while accessing Secret Service API.
1 Like

Hi @ibuchanan

The keytar is an optional dependency of the CLI. So even if it cannot compile in your environment, this should not block the usage of the CLI.

You might see warnings and errors during the installation step, but the CLI will be usable as the keytar codepath will be ignored when running commands.

Hope this clarifies things.

2 Likes

I can confirm @XavierCaron’s assessment, though there are some gotchas that are easy to run into, here’s our current recipe:

  1. do not depend on @forge/cli via package.json (we had it as a dev dependency, which worked up to 1.5.0 and somehow broke in 1.6.0)
  2. ensure to run npm install again before committing package-lock.json after removing any @forge/cli dependency as per 1), otherwise keytar remains configured based on your local interactive environment
  3. instead, install @forge/cli via a separate step in your build environment to ensure it is configured w/o the interactive features (of course, you can optimize build times via a custom build image with the @forge/cli preinstalled) - the mentioned warnings can be ignored, or better yet avoided by ignoring the optional keytar dependency (thanks @remie):
    • npm 6.x: npm install @forge/cli@2.0.1 --no-optional
    • npm 7.x: npm install @forge/cli@2.0.1 --omit optional
  4. ensure to add the --non-interactive flag to applicable commands like forge login and forge install
7 Likes

To add to this, if you remove forge-cli from your package.json and package-lock.json files, then this will work for a basic deploy:

name: Deploy to forge app to atlassian cloud site development environment on push

on:
  push:
    branches: [dev]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Build the source code for deployment
    steps:
      - uses: actions/checkout@main
      - name: Use Node.js ${{ matrix.node-version }}
        uses: actions/setup-node@v1
        with:
          node-version: ${{ matrix.node-version }}
      - run: npm ci
      - run: npm install @forge/cli@2.0.1 --no-optional
      - run: npm run static:install
      - name: disable analytics
        run: npx forge settings set usage-analytics false
      - name: Deploy to atlassian cloud site
        run: |
          npm run static:build
          npx forge login --email ${FORGE_EMAIL} --token ${FORGE_TOKEN} --non-interactive
          npx forge deploy
        env:
          FORGE_EMAIL: ${{ secrets.FORGE_EMAIL }}
          FORGE_TOKEN: ${{ secrets.FORGE_TOKEN }}

Atlassian, the fact this took 3 people hours to figure out and we have no meaningful response from anyone on this thread for a month shows how GA Forge really is. I can’t believe I wasted my entire afternoon on this.

Major Kudos to Remie and Steffen for spending the time to help me un@#$% this.

8 Likes

This is quite scary, that I need to use my main account credentials in CI. Deployment keys per app would be much more secure and practical.

2 Likes

Thanks everyone for helping identify Forge’s CI/CD capabilities. I think CI/CD is an important topic and given Forge’s unique hosting setup, we need to provide a dedicated guide on Forge CI/CD. I’ve created FRGE-813: Inadequate documentation of Forge CI/CD support to request this.

1 Like

Thanks @dmorrow, appreciate the initiative! The lack of CI/CD guidance has also just been discussed (and acknowledged) in Join us for a Developer AMA with Tim Pettersen - #9 by sopel, where I also referenced a recent Atlassian blog post that seems to be a good starting point for a guide (Bitbucket Pipelines though, ideally a guide would also cover GitHub indeed):

  • […] I just stumbled over Damien Lauberton’s recent and comprehensive blog post How to Configure CI/CD for an Atlassian Forge App […], and it has neither been announced in the community nor referenced from the Forge docs yet, maybe you could derive (and maintain) a tutorial from it?
1 Like