Weekly GDPR API status development update - March 29

akassab · March 29, 2019, 7:02pm

Apps are required to migrate to Atlassian AccountID and remove legacy user references (username and user key) by April 29, 2019. For more background on why - please see our first post .

Understanding migration status

We are tracking use of the apiMigrations flag in the Connect App descriptor as a way to understand developer readiness to deprecate username and user key from our REST APIs. This flag serves as both a mechanism to test the new API behaviors (when set to gdpr:true) and a communication tool to signal when you are blocked (when set to gdpr:false).

Today, we have 199 apps that have signaled that they are ready for the deprecation, 79 that have signaled they are blocked, and ~400 that have not indicated their status.

In order for us to understand where you are with migration to accountID and use that information to effectively communicate with our product teams we need you to signal your status using the “opt In mechanism”.

Learn more about how to properly use this flag.

Additional note on opt’ing into the new API behaviors

The flag described above controls Atlassian Connect behaviors. When opt’d in gdpr:true only the APIs related to Atlassian Connect will change this includes: Inbound Auth, Outbound Auth, App iFrames, App Lifecycle, Webhooks, and JavaScript API. To change the behaviors of Jira and Confluence REST APIs you will need to pass additional header/query parameters on every REST call. The header for Jira APIs is x-atlassian-force-account-id’:true. This header is also required to transform change logs from username to accountID. The query parameter for Confluence is privacyMode=true.

Note: Jira webhooks (query parameters and webhook payloads) still contain user_id and user_key even when opt’d into the new API behaviors via the connect descriptor. We are planning to address query parameters but not webhook payloads. Please comment below if continuing to receive these soon to be deprecated fields in webhooks during the opt in period creates a critical blocking issue for you.

Recently closed issues

Confluence documentation for new REST endpoint for migrations - Jira and Confluence have new migration APIs that are unaffected by the 29 April 2019 deprecation. (See here for Jira: /rest/api/3/migration ) The endpoint for Confluence is documented here.
Get current user’s locale ( ACJS-1073) - AP.user.getLocale() is now available. For performance concerns please consider using cacheable app iframes.
Jira user picker REST API missing account type - Raised in ACJIRA-1686. The fix for this is in process of being rolled out. Once roll out is complete the ACJIRA ticket will be closed.
JQL transformed to remove legacy user references and replaced with accountID are un-readable - We’ve built a JQL Editor Dialog component that will display the user’s current name in JQL rather than the accountID. There is an early access version of this API available now. To experiment, you can simply paste the showJQLEditor code snippet into any Jira app iframe. Additional development to convert from accountID to user lozenge and fix a bug known to display double scrollbars in some browsers is still under development. See ACJIRA-1710.
Jira post function /triggered calls missing accountID and account type - Jira released a fix to introduce a new “user” field in the /triggered call body this week. The “user” field does not currently contain an accountType subfield. We are working with the Jira team to get an estimate for adding accountType subfield added to the /triggered call body. See ACJIRA-1722.
Jira PD Cleaner converting user strings that are not exact usernames to ‘unknown’ - This issue was raised in ACJIRA-1712 but now is being tracked in ACJIRA-1725. The Jira team is updating the PD cleaner to return an error instead of converting the results to “unknown”. The fix for this is rolling out now and the ACJIRA ticket will be updated once complete.

Issues with workarounds

The below issues have workarounds and therefore have been de-prioritized from our list of blockers. Please refer to the tickets for more details on the workarounds.

Account ID not working in notify resource - This issue was raised in ACJIRA-1746. In order to get accountID in the notify resource you’ll need to include the “strict mode” header in your REST call (x-atlassian-force-account-id:true).
Jira sidebar fails to render for Project Admin Tab Panel without legacy user context parameters - This issue was raised in ACJIRA-1736. Jira completed their investigation of this issue and discovered that if the order of parameters is changed or any parameters are omitted, then the navigation fails to render. In terms of the upcoming API deprecations it should be possible to include the legacy context parameters to stop the navigation failing to render. Jira will look into improving the handling of parameters for this feature but that may not be done before the planned deprecation of legacy user references.
Jira does not recognize issue entity property aliases - This issue was raised in ACJIRA-1723. The guidance from the Jira team at this time is the same as the above (manually convert expressions that cannot be automatically converted by the PD Cleaner).
Jira does not have an API to edit workflows -This issue was raised in ACJIRA-1718. The guidance from the Jira team at this time is to update post function data from its configuration form.
Problem exporting / importing user data in Jira - Exporter Cloud - This issue was raised in ACJIRA-1751. Export as CSV currently produces user names, and importing of CSV data currently understands user names and commits to not breaking the import/export functionality until resolving issues raised in that ticket.

Items In Progress

Jira incoming webhooks missing accountId - This work is still in progress. We do not have a new fix date. See ACJIRA-1692.
Email address hidden by default - The Email API is being worked on now. We are delayed a bit (we expected the work to be complete this week). The new estimated date for this is April 5, 2019. See ACJIRA-1726. In the meantime, if you anticipate needing to use email address please see these guidelines on how to request access to the Email API.
Missing avatarURL with opt in header (x-atlassian-force-account-id:true) - Avatars should be present with the opt in. Jira is currently investigating this issue. We do not have an estimated fix date yet. See ACJIRA-1732.
Jira comment authors missing accountType information - We do not have an estimated fix date yet. See ACJIRA-1765.

Testing Profile Visibility Controls

If you are ready to start testing profile visibility controls, please raise a ticket in our Developer Support Help Desk here . We are not quite ready to set up developer environments but this guide will describe how testing will work and what you can expect once privacy controls are turned on.

Thank you

We realize this is a large amount of work to complete in a short timeframe. We appreciate you working with us through this process to deliver a more trusted experience for our customers.

If we’ve missed anything preventing you from completing the migration, please comment below.

jens · April 3, 2019, 9:43am

Hi @akassab,

any news on the profile visibility controls? Testing those is the last blocker for our Confluence apps.

Cheers,
Jens

maciej.dudziak · April 3, 2019, 3:49pm

Hi

Did anybody try to use AP.jira.openCreateIssueDialog?

AP.jira.openCreateIssueDialog(function(issues){
  alert(issues[0]['fields']['summary']);
}, {
  pid: 10000,
  issueType: 1,
  fields : {
    summary: "Hello World",
    environment : "My environment",
    priority : 2,
    assignee: "tom",
    reporter: "bob",
    labels : ["Mylabel","MyOtherLabel"],
    description : "My first Issue",
    duedate : "11/Oct/16",
    fixVersions : 10001,
    versions : 10000,
    components : "My component",
    timetracking_originalestimate: "2w",
    timetracking_remainingestimate: "3d",
    worklog_activate: true,
    worklog_timeLogged: "2"
  }
});

Before API migration you would pass user.key for assignee or reporter. Now I am trying to pass accountID, but it does not seem to work

Result is that Dialog opens with reporter field showing accountId (not user) and after clicking create, validation complains

akiyanovskiy · April 3, 2019, 6:52pm

Hi @akassab,

What about exporting accountIds in the ‘User management’ and ‘Customer Portal only users’ export? Is there a related request(s)?

Cheers,
Andrey.

maciej.dudziak · April 5, 2019, 12:04pm

https://ecosystem.atlassian.net/browse/ACJIRA-1773

jbevan · April 5, 2019, 1:19pm

Hi @akassab are there any tickets that we can watch (and if not can you raise them) for providing the account type in Confluence webhooks? Ie a Confluence equivalent of [ACJIRA-1674] - Ecosystem Jira

sluthra · April 5, 2019, 5:02pm

Jon, that work is already done. You can check /wiki/rest/api/user/current

jack · April 11, 2019, 6:47am

@akassab, @vendors

I want to report (ask about) the problem with /rest/api/2/user/search REST endpoint that looks like a bug.

We have performed the last step of the GDPR migration (replaced the user keys with accountID in our database). We used the aforementioned endpoint to search for users and their accountID by sending user keys in the query.

Most of the requests returned expected data. One of the expected responses was empty HTTP 200 which could mean that user is no longer there (we assumed that it could happen).

When analyzing logs, we discovered that for some Jira instances 100% responses returned empty HTTP 200. That looked worrying, so we started the investigation (please note that we asked only for a subset of Jira users so we might have bad luck getting 100% of empty responses).

Unfortunately, we were able to reproduce that behavior (an empty response for an existing user) on our production Jira instance by removing the “Browse users and groups” permission for “jira-servicedesk-users” group. And it looks like a serious issue.

Important facts:

calls were made from the server side as an app (which holds READ and ADMIN permissions)
in Jira UI it is no longer possible to see the app user and give/reject permissions on that basis.
we have asked/search for a regular user who is present in (and has access to) Jira Core and Software
restoring permissions (“Browse users and groups”) to group “jira-servicedesk-users” made the search endpoint return correct (non-empty) data again.

The above looks like a Jira bug that prevents us from migrating all stored user data to accountID.

Did anyone face it?

Thanks,
Jack

ivan.babikov · April 15, 2019, 10:29am

Hi @akassab,

Our Atlassian Connect app calls “/rest/api/latest/search” GET method to get all Issues of a certain type from the project. We use the following JQL for it: “project = ‘ProjectKey’ AND type = ‘IssueTypeId’”.

Previously, to avoid the error “Forbidden” we had to use impersonation and we doing these calls with Sub Prefix “urn:atlassian:connect:userkey:admin”.

Currently, “urn:atlassian:connect:userkey” is going to be deprecated but we do not know the AccountId for “admin” to use it with “urn:atlassian:connect:useraccountid:”.

It there any common AccountId value for “admin” or “urn:atlassian:connect:” construction for Jira-Administrators role or any other way to get a list of Issues from app with JWT authentication?

The error we are getting is:
“Add-on ‘qac-connect-plugin-…’ disallowed to impersonate the user because ‘no valid active user exists’”}

ThomasGrantt · July 25, 2019, 4:15pm

Hello Ivan,

Has anyone found a solution to this exact problem. I am also facing this issue.

Thank you.

ivan.babikov · July 26, 2019, 4:49am

Hello Thomas,

For my problem with “/rest/api/latest/search” GET there were 2 solutions:

Use “/rest/api/latest/search” POST method.
This solution fixed the actual reason of the problem. The problem was that we use Microsoft .Net client to access Jira which is written using Java. So, there proved to be a slight difference in HTTP special symbol encoding on our side and Jira. We fixed it by choosing another encoding method. If this is your case, I can tell more details.

maciej.dudziak · July 26, 2019, 10:43am

@ThomasGrantt

We still have: “apiMigrations”: {
“gdpr”: false
},

With this setting we still get in issue payload both accountIds and user keys and for now we are still using userKeys in this case.

I am guessing this may stop working at some point, but we had no other choice

ivan.babikov · July 26, 2019, 1:48pm

Our problem was with “gdpr”: true and related to wrong hash calcs caused by appearing AccountID field (with its special chars) in URL

ThomasGrantt · July 26, 2019, 1:55pm

Okay, thank you for you response.

ThomasGrantt · July 26, 2019, 2:08pm

@ivan.babikov

In my application, this information (userkey/acccountID) is passed by the Jira platform to my backend spring boot addon service. I simply extract the relative information from the AtlassianHost java object.

Do you think setting “gdpr: false” may temporarily fix the issue?

ivan.babikov · July 29, 2019, 10:30am

If your problem is related to the case “[ACJIRA-1773] - Ecosystem Jira” which you mentioned initially I cannot remember if we have had problems with posting user Ids after GDPR migration has been started.

AFAIK, now all apps are forced to work in “gdpr:true” mode.