JSON is not well formed when adding new AddOn... but it is! (a.k.a. Visual Studio/Windows adds the BOM to the file)

Greetings! I am trying to get my first addon up and running on my atlassian cloud development site and running across this error:

Invalid Atlassian Connect descriptor: JSON is not well-formed (Unexpected character (‘’ (code 65279 / 0xfeff)): expected a valid value (number, String, array, object, ‘true’, ‘false’ or ‘null’) at [Source: ; line: 1, column: 2]). Please contact the add-on vendor for details.

But it is…

I run my connect json (shown at bottom of post) through the atlassian connect validator and it passes:

I upload the json to my Blob container in Azure and can access it from there via the browser so then I go to my Manage Add-Ons admin page, click Upload Add-on, paste in my connect URL and then see that very “descriptive” message above.

So my json isn’t formatted properly but the Atlassian tools says it is.

Please advise.

(Note: The ngrok session has ended so that won’t work)

{
  "key": "has-first-jira-addon",
  "name": "HAS - First Jira Add On",
  "description": "Happy App Software's very first Jira add-on hosted in Azure",
  "apiVersion": 1,
  "enableLicensing": false,
  "vendor": {
    "name": "Happy App Software LLC",
    "url": "http://www.happyappsoftware.com"
  },
  "baseUrl": "http://d5947734.ngrok.io",
  "links": {
    "self": "https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/atlassian-connect.json",
    "homepage": "https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/index.html",
    "documentation": "https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/index.html"
  },
  "authentication": {
    "type": "jwt"
  },
  "lifecycle": {
    "disabled": "/disabled",
    "enabled": "/enabled",
    "installed": "/installed",
    "uninstalled": "/uninstalled"
  },
  "scopes": [
    "read",
    "write"
  ],
  "modules": {
    "webhooks": [
      {
        "event": "jira:issue_created",
        "url": "/issue-created",
        "excludeBody": false
      },
      {
        "event": "jira:issue_deleted",
        "url": "/issue-deleted",
        "excludeBody": false
      },
      {
        "event": "jira:issue_updated",
        "url": "/issue-updated",
        "excludeBody": false
      },
      {
        "event": "comment_created",
        "url": "/comment-created",
        "excludeBody": false
      },
      {
        "event": "comment_deleted",
        "url": "/comment-deleted",
        "excludeBody": false
      },
      {
        "event": "comment_updated",
        "url": "/comment-updated",
        "excludeBody": false
      },
      {
        "event": "user_created",
        "url": "/user-created",
        "excludeBody": false
      },
      {
        "event": "user_deleted",
        "url": "/user-deleted",
        "excludeBody": false
      },
      {
        "event": "user_updated",
        "url": "/user-updated",
        "excludeBody": false
      }
    ]
  }
}

Hey Aarron!

First thing I have noticed (but shouldnt cause this error) is that your baseUrl needs to begin with https.
ngrok should provide this to you when the tunnel is open.

Otherwise my best guess would be a non-ASCII character hiding inside the JSON.
Line 2 says that perhaps that it may not like the new line character used. Would you be able to minify the JSON descriptor and try again?

If this is the issue, then I would think we should open a bug and try suss it out there :slight_smile:

Thanks,
Dave

Dave,

Thanks for the tips. Tried both suggestions and still have the same problem. :frowning:

Going to open a bug in regards to this.

On a side note, thanks for the really rapid response on a Friday night. Kudos to you sir!

Thanks,

Aarron

Hey @aarron.szalacinski,

only yourself and Atlassian Staff can see the DEVHELP tickets you’ve created, so nobody else can follow up on what’s happening on the ticket.

Have you tried hosting the file somewhere else than Azure? For example, running your app locally on your machine and using ngrok as a tunnel to get it installed?

It might be that Azure is adding something in front of the file when Jira tries to retrieve it.

So I just tried hosting it on Glitch: https://glitch.com/edit/#!/noiseless-desert

And tried installing it into my own Jira Cloud dev instance (using your exact descriptor) and I didn’t get a validation error. I did get a 404 because I didn’t configure the paths correctly in my glitch project.

But that could only happen after the descriptor already got validated.

So it might have something to do with where your descriptor is being hosted or the encoding being used for your descriptor file.

Thanks for the details Peter.

I think we are closing in on the issue.

I started digging around in the settings of my Blob container and noticed the ContentEncoding property/header wasn’t set.

I changed that to UTF-8 and the good news is that I don’t get the 0xFEFF error any more. The bad news is I get a new error:

Problem accessing the file https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/atlassian-connect.json.

So it’s getting somewhere.

I tried changing a bunch of combinations of header properties and the above is the new error message.

So where we are at is it appears the Atlassian AddOn mechanism does not like the set of headers coming back from Azure Blob Storage or is changing the casing of the URL.

Here are the headers coming back…

https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/headers.png

I can also add the following additional headers:

CacheControl
ContentDisposition
ContentLanguage

Azure Bob containers are case sensitive.

If you take the URL and change it from
https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/atlassian-connect.json
to
https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/Atlassian-connect.json

A 404 comes back and states the Blob couldn’t be found

So if the team can identify what headers need to come back AND what the URL is changing to, we can mark those investigative items as NOT the root cause. (Or with a little luck, we will have the business rules to let Azure users know how to serve static connect scripts from Blobs!)

With that said, I am going to follow your advice for a temporary workaround and host elsewhere but I need the azure blob option as a deployment strategy for static files.

Thanks for the assist!

-A

Set up my ngrok environment locally, serving the app descriptor from local web server.

Can download the json file with no issue in the web browser.

Still getting the same errors. :confused:

Watching ngrok inspect and my server logs I see the descriptor being served but then there is no call after that. Serving the file with Content-Encoding = UTF8 and the error changes to the one where there is a problem installing.

Going to try hosting from glitch and see if that does something different.

-A

So hosting the connect script in glitch worked perfectly.
The differences in headers may be the key here… only showing a few

From glitch:
Connection →keep-alive
Content-Length →1835
Content-Type →application/json; charset=utf-8
Date →Sat, 10 Mar 2018 19:51:31 GMT
X-Powered-By →Express
cache-control →max-age=0
etag →W/“72b-16211696637”
last-modified →Sat, 10 Mar 2018 19:35:42 GMT
vary →Origin

From my hosting
Accept-Ranges →bytes
Content-Type →UTF-8
Date →Sat, 10 Mar 2018 19:26:26 GMT
Etag →"93edee73a1b8d31:0"
Last-Modified →Sat, 10 Mar 2018 18:56:09 GMT
Server →Microsoft-IIS/10.0 Microsoft-HTTPAPI/2.0
Transfer-Encoding →chunked
X-Powered-By →ASP.NET
X-Sourcefiles →=?UTF-8?B?Qzpcc3JjXEhBUy5KaXJhXEhBUy5KaXJhLkFkZE9uLldlYi5GaXJzdEFkZE9uXGZpcnN0YWRkb24tY29ubmVjdC5qc29u?=

Going to see if Content-length and Content-Type formats are the breaking item.

1 Like

I suspect the Content-Length and Content-Type headers may be the key headers that need to be in place. However when I moved to attempting to server the json from IIS with the needed headers, I was still getting the same errors.

Gonna have to leave it to the SME’s to noodle on this one.

I am not a server expert so any details on how IIS should be setup and what headers, responses, etc are needed would be helpful.

Thanks!

-A

1 Like

Hi @aarron.szalacinski,

I replied to the DEVHELP ticket you created. It seems you’re dealing with byte order mark. Saving the file with an encoding with no BOM option using your text editor should remove the 0xFEFF (I used Sublime and I believe Notepad++ have this functionality). Using less in CLI to view the file contents shows the BOM in the first part of the file you attached.

Hope this helps,
Ian

2 Likes

This was totally the issue. I am a .NET developer (yes there are those of us who build add-ons using .net) and Visual Studio likes to save files with that BOM. Saved the UTF-8 files appropriately and it works now.
To fix it a user needs to:

Save the file using the Save As command:

https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/SaveAs.png

Click the dropdown to Save With Encoding:

https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/SaveFile.png

And then make sure the appropriate encoding is applied:

https://hasjiratestaddon030418.blob.core.windows.net/has-jira-firstaddon-assets/WithoutSignature.png

Glad to see that it was an easy fix and not something silly… like headers… I also confirmed that the file can be served directly from Azure Blob storage.

Apologize for the delay in getting a response. The day job got in the way this week.

Thanks for the assist.

-A

3 Likes