Weekly GDPR API status development update - April 12

akassab · April 12, 2019, 5:14pm

Apps are required to migrate to Atlassian AccountID and remove legacy user references (username and user key) by April 29, 2019. For more background on why - please see our first post .

Understanding migration status

We are tracking use of the apiMigrations flag in the Connect App descriptor as a way to understand developer readiness to deprecate username and user key from our REST APIs. This flag serves as both a mechanism to test the new API behaviors (when set to gdpr:true) and a communication tool to signal when you are blocked (when set to gdpr:false).

In order for us to understand where you are with migration to accountID and use that information to effectively communicate with our product teams we need you to signal your status using the “opt In mechanism”.

Learn more about how to properly use this flag.

Additional note on opt’ing into the new API behaviors

The flag described above controls Atlassian Connect behaviors. When opt’d in gdpr:true only the APIs related to Atlassian Connect will change this includes: Inbound Auth, Outbound Auth, App iFrames, App Lifecycle, Webhooks, and JavaScript API. To change the behaviors of Jira and Confluence REST APIs you will need to pass additional header/query parameters on every REST call. The header for Jira APIs is x-atlassian-force-account-id’:true. This header is also required to transform change logs from username to accountID. The query parameter for Confluence is privacyMode=true.

Note: Jira webhooks (query parameters and webhook payloads) still contain user_id and user_key even when opt’d into the new API behaviors via the connect descriptor. We are planning to address query parameters but not webhook payloads. Please comment below if continuing to receive these soon to be deprecated fields in webhooks during the opt in period creates a critical blocking issue for you.

Recently closed issues

Jira PD Cleaner converting user strings that are not exact usernames to ‘unknown’ (ACJIRA-1725). We rolled out a fix to the PD cleaner so that full name searches against active users are not migrated to accountID - instead of converting the results to “unknown”.
JQL transformed to remove legacy user references and replaced with accountID are un-readable (ACJIRA-1710) - We’ve built a JQL Editor Dialog component that will display the user’s current name in JQL rather than the accountID.
Missing avatarURL with opt in header (x-atlassian-force-account-id:true) (ACJIRA-1732. ) - We’ve rolled out a fix to present avatars when x-atlassian-force-account-id:true header is used.
Jira comment authors missing accountType information ( ACJIRA-1765) - Jira comment authors includes accountType.

Issues with workarounds

The below issues have workarounds and therefore have been de-prioritized from our list of blockers. Please refer to the tickets for more details on the workarounds.

Account ID not working in notify resource - This issue was raised in ACJIRA-1746. In order to get accountID in the notify resource you’ll need to include the “strict mode” header in your REST call (x-atlassian-force-account-id:true).
Jira sidebar fails to render for Project Admin Tab Panel without legacy user context parameters - This issue was raised in ACJIRA-1736. Jira completed their investigation of this issue and discovered that if the order of parameters is changed or any parameters are omitted, then the navigation fails to render. In terms of the upcoming API deprecations it should be possible to include the legacy context parameters to stop the navigation failing to render. Jira will look into improving the handling of parameters for this feature but that may not be done before the planned deprecation of legacy user references.
Jira does not recognize issue entity property aliases - This issue was raised in ACJIRA-1723. The guidance from the Jira team at this time is the same as the above (manually convert expressions that cannot be automatically converted by the PD Cleaner).
Jira does not have an API to edit workflows -This issue was raised in ACJIRA-1718. The guidance from the Jira team at this time is to update post function data from its configuration form.
Problem exporting / importing user data in Jira - Exporter Cloud - This issue was raised in ACJIRA-1751. Export as CSV currently produces user names, and importing of CSV data currently understands user names.

Items In Progress

Jira PD Cleaner converting user strings that are not exact usernames to ‘unknown’ - We are still experiencing an issue with the PD cleaner converting deactivated users to unknown. We do not have an estimated fix date for this. See ACJIRA-1725.
JQL transformed to remove legacy user references and replaced with accountID are un-readable - A new issue tracking the development of JQL rendered in advance mode was raised. We do not have an estimate fix date for this. See ACJIRA-1768
Jira incoming webhooks missing accountId - This work is still in progress. We do not have a new fix date. See ACJIRA-1692.

Testing Profile Visibility Controls

If you are ready to start testing profile visibility controls, please raise a ticket in our Developer Support Help Desk here .

Thank you

We realize this is a large amount of work to complete in a short timeframe. We appreciate you working with us through this process to deliver a more trusted experience for our customers.

If we’ve missed anything preventing you from completing the migration, please comment below.

ademoss · April 12, 2019, 5:32pm

Hi @akassab, is anyone manning the service desk for requesting profile visibility? We’ve been waiting for several days now for someone to enable the new controls for some of our users.

akassab · April 12, 2019, 5:59pm

Its me … sorry I’ve been a bit behind here!

vit · April 15, 2019, 7:49am

I guess I an on the laggards team… Just finished migrating all our apps last week, phew!

niels · April 15, 2019, 8:48am

Hi @akassab, I am in the process of migrating our app, and I can see that the changelog on jira:issue_updated webhooks still use the userKey on the to and from fields when I change the assignee on an issue.

The app’s gdpr-flag is set to true so I was expecting to get accountId here.

What can I expect the changelog to use after April 29th?

Here is an example:

    "changelog": {
        "id": "10004",
        "items": [
            {
                "field": "assignee",
                "fieldtype": "jira",
                "fieldId": "assignee",
                "from": "admin",    // <-- userKey
                "fromString": "Niels Frederiksen",
                "to": "niels+1",  // <-- userKey
                "toString": "niels frederiksen +1"
            }
        ]
    }

niels · April 15, 2019, 8:52am

Ohh I just saw ACJIRA-1692
Nevermind then

akassab · April 15, 2019, 5:07pm

@vit - Awesome news!!!

jack · April 16, 2019, 7:52am

We are still kind of blocked by the problem I reported in DEVHELP-2566 and described at Weekly GDPR API status development update - March 29 - #9 by jack

jbevan · April 16, 2019, 6:31pm

For clarity, we’re blocked by [ACJIRA-1779] - Ecosystem Jira (deactivated users being replaced by unknown in PDCleaner) and [ACJIRA-1692] - Ecosystem Jira (no accountIds in webhook changelogs)

akassab · April 16, 2019, 11:53pm

Hi @jack - I responded via the DEVHELP ticket but to summarize here, could you use the bulk migration endpoint to search instead? (that endpoint shouldn’t require global user permissions)

Thanks,
Alex

ivan.babikov · April 17, 2019, 1:20pm

Hello @akassab,

Our Atlassian Connect app calls “/rest/api/latest/search” GET method to get all Issues of a certain type from the project. We use the following JQL for it: “project = ‘ProjectKey’ AND type = ‘IssueTypeId’”.

Previously, to avoid the error “Forbidden” we had to use impersonation and we doing these calls with Sub Prefix “urn:atlassian:connect:userkey:admin”.

Currently, “urn:atlassian:connect:userkey” is going to be deprecated but we do not know the AccountId for “admin” to use it with “urn:atlassian:connect:useraccountid:”.

Is there any common AccountId value for “admin” or “urn:atlassian:connect:” construction for Jira-Administrators role or any other way to get a list of Issues from app with JWT authentication?

The error we are getting is:
“Add-on ‘qac-connect-plugin-…’ disallowed to impersonate the user because ‘no valid active user exists’”}

epehrson · April 17, 2019, 4:19pm

@ivan.babikov, just to reiterate, account IDs identify users globally in Atlassian Cloud, whereas the legacy user references (username and user key) are site-bound. So when you refer to the admin user, you are typically referring to different user for each site. (And to my knowledge, there was never any guarantee of Jira having a user with that key.)

No, there is no such symbolic account ID value or magic value for the OAuth 2.0 JWT Bearer Token assertion sub claim. What you describe sounds like an anti-pattern, but I suppose you should be able to find a user with the appropriate permissions using /viewissue/search or /user/permission/search as a short-term solution.

Instead of impersonating a user with permissions, it seems like you should make sure that your app user has the appropriate permissions. You should perhaps consider adding PROJECT_ADMIN or ADMIN scope. But in any case, the current version of your app must first be migrated away from deprecated APIs.

ivan.babikov · April 18, 2019, 12:06pm

Hello Einar Pehrson ( @epehrson ),

Actually we have all possible scopes in our plugin descriptor file:
“scopes”:[“READ”,“WRITE”,“ADMIN”,“PROJECT_ADMIN”,“DELETE”,“ACT_AS_USER”]
But for some reason the call of “/rest/api/latest/search” leads to “Forbidden” error and 2 years ago, on the development stage we had to impersonate this call on behalf of the “admin”. It worked fine on many clients’s Jira Cloud instances.

brendan · April 19, 2019, 6:49pm

Hi Team GDPR,

We’ve been noticing several of our customers whom we believe should have had their descriptors automatically upgraded seem to have not been. It is possible some of these instances are on older versions of the descriptor and recent versions require admin ‘approval’.

Questions:

What is supposed to happen if customers are on old versions of a descriptor?
Can the GDPR team do something extra to encourage admins to update? I’m guessing some of the time they don’t even see the ‘upgrade’ email if that is the only way they’re informed.
Is there a way that vendors with Atlassian Cloud apps can detect what version of a descriptor a customer is using so we can seek to encourage them to upgrade…perhaps insert a banner message for admins when using our app?

We have also noticed the documentation on descriptor updates is either out of date or contracdictory:

To propagate a change in the descriptor to the Atlassian products, you need to create a new version of the app in its Marketplace listing.
from: https://developer.atlassian.com/cloud/jira/platform/app-descriptor/

We poll the app descriptor URL that you included when you submitted your listing. When we detect a change, we automatically update your app in the Atlassian Marketplace with a new version
from: https://developer.atlassian.com/platform/marketplace/upgrading-and-versioning-cloud-apps/

These two pages are linked but it seems they directly contradict each other.

Thanks!
Brendan

jack · April 19, 2019, 7:31pm

@brendan,

With regards to bullet #3, you might try License API for cloud apps

brendan · April 19, 2019, 9:02pm

Thanks @jack ! Is there a kudos system here? Looking into that. I’d forgotten they do dynamically append that info to the descriptor, but looking into if it ends up where they say. Are you leveraging that? (ah just figured out the kudos system and sent, thx!!)

epehrson · April 23, 2019, 10:18am

@ivan.babikov, I think the Forbidden error is where we should focus our efforts. Could you please raise a bug in DEVHELP with steps to reproduce that issue using regular JWT authentication (without impersonation), ideally also providing an example tenant where you can reproduce it.

EDIT: I just saw you already did, ACJIRA-1790.

epehrson · April 23, 2019, 10:32am

@brendan, the API changes apply to all app installations, so generally, we expect app developers to ensure all versions with installations remain functional after the deprecation date.

But of course, after a scope increase, you are not required to maintain previous app versions indefinitely. And you may want to sunset older versions of the app after a notice period.

Can the GDPR team do something extra to encourage admins to update?

Normally no, but if you have special circumstances, you could describe them in a DEVHELP issue and we can take it from there.

ivan.babikov · April 24, 2019, 12:26pm

Hello,

Currently “/rest/api/latest/issue/{issueKey}” GET REST method in GDPR mode and with “x-atlassian-force-account-id” header returns Issue with user info fields (Assignee, Reporter etc) with user Emails even though the user has “Atlassian account/Privacy” option “Private (just you)”.

Is it normal or am I using the wrong place to set user Email visibility (the url is “Atlassian account”)?

maciej.dudziak · April 24, 2019, 1:35pm

Will [ACJIRA-1773] - Ecosystem Jira be ready before the deadline?

If key from user objects dissapears after 30.04, it will break our app.

It is the last thing that blocks us to set apiMigration=true for Issue Templates app