Weekly GDPR API status development update - February 7th

Announcements
,
1

We’re making changes to our cloud products to provide users with more control over their profile information - both in response to recent changes to data privacy legislation (i.e. GDPR) and as part of a broader effort to improve customer trust.

These changes require apps to migrate their app and data stores to a new global identifier - the Atlassian AccountID and remove legacy user references (username and user key).

Recap on timeline

We announced the deprecation of legacy user references in October 2018. Per our standard deprecation policy, apps have 6 months to complete the migration to accountID. On 29 March 2019 Jira, Confluence, Bitbucket and Connect Cloud REST APIs currently containing username and user key will only return accountID.

In addition to the migration, you will need to prepare for profile visibility controls which we intend to launch in mid April 2019 . These profile visibility controls will create variability in the user object based upon preferences set by the user. This means that fields that are currently available (i.e. email address) will no longer return or return null.

We’ve recognized that there are a number of issues you’ve identified affecting your ability to migrate and prepare for profile visibility controls. In the spirit of Atlassian’s Open Company, No BS value, we’ll go over each of the open items that we are currently working on and provide estimates of completion. Please note that these dates may change. We will provide these updates weekly.

Recently closed issues

  • Deprecation of AP.user.getTZ() API - In early October we announced via the Connect API Migration guide that the Atlassian Connect JavaScript API, AP.user.getTZ() would be deprecated without a replacement. This API is used to get the logged in user’s current TZ. Our reason for removing this API is because user timezone is an indirect identifier that we are including in the user profile visibility control screen. This API provides unrestricted access to user timezone and for that reason we did not want to break customer trust by exposing a field that they’ve chosen to be hidden. After more careful consideration of the impact to the user experience, we’ve decided not to deprecate the API. We are publishing guidelines on the appropriate use of this API as a result and ‘un-deprecating’.
  • AP.getUser() Javascript API doesn’t return accountID - There’s a new method to retrieve the current user’s accountID. See AP.user.getCurrentUser()
  • Jira change logs containing username missing accountID - The Jira API migration guide describes an opt in mechanism which can be used to force accountIDs to be returned in Jira change logs. To convert Jira change logs from username to accountID use the x-atlassian-force-account-id: true header in your REST call to retrieve change logs.
  • Conversion of stored JQL containing legacy user references to accountID - The Jira API migration guide describes a ‘PD Cleaner’ operation which can be used to convert stored JQL containing legacy user references to accountID.

Items in progress

  • Jira and Confluence webhooks bodies that contain username and/or user key missing accountID - We are working to update webhooks to contain both legacy user references, accountID and account type. We expect this work to be complete for both Jira and Confluence in approximately 2 weeks time.
  • Issues with updating data stored in product - We’re aware that there are certain challenges with migrating data stored in the host product. For example, if the app requires a user action to update which may not have occurred during the deprecation period or if the app lost access to the storage location either by changes to permissions or uninstall. We are planning to facilitate migration of those legacy user references to accountID by providing a new API endpoint unaffected by the 29 March 2019 deprecation of username / user key which will continue to allow you to lookup old user references and convert to accountID. Jira has already released this endpoint (see: /rest/api/3/bulk/migration) and Confluence is planning to release this endpoint before end of February 2019. You should expect these APIs to be temporary, as it is important for us to remove use of username and user key.
  • Conversion of stored CQL containing legacy user references to accountID - The Confluence team is developing a similar solution to the one that’s currently available for Jira. We don’t yet have a confirmed date for completion but will provide an update in a week’s time.
  • Testing Profile Visibility Controls - We intend to provide you with time to test profile visibility controls before we launch. We will provide more information on how to test in 2 weeks time.
  • Email address hidden from public by default - Email address will be hidden from public by default which means that the email address will no longer be provided to you through our product APIs for the majority of users. Users will need to manually adjust their profile visibility control settings in order to provide access to email. We recognize that for some of you this restricts core functionality. We intend to enable access to email for approved use cases only. We will provide more detail in 2 weeks time.

Thank you

We realize this is a large amount of work to complete in a short timeframe. We appreciate you working with us through this process to deliver a more trusted experience for our customers.

If we’ve missed anything preventing you from completing the migration, please comment below. If you are no longer blocked please opt in to the new APIs using the opt in mechanisms described in the migration guides to signal to us that you’re ready.

11 Likes
2

This is great Alex, thanks for the weekly recap!

It’s been a very opaque process thus far with (seemingly) conflicting messages. This looks like a great approach to getting everyone on the same page.

Cheers,
Nick Muldoon, Easy Agile

2 Likes
3

Hi @akassab,
does this also mean you’ll un-deprecate the respective context parameter tz, which is still marked as deprecated in the docs?

https://developer.atlassian.com/cloud/confluence/context-parameters

It’s nice to get a replacement for the JS API but ideally our app servers could get that information without another callback to the host instance to improve performance. Since apps can get the data using the JS API I guess there’s no real reason for hiding it from the context parameters.

For the also deprecated loc context parameter we can always have a look at the HTTP language headers and hope the users stick to the ‘auto-detect’ mode in their user profiles but for the timezone there is no fast replacement if there is no context parameter.

Cheers,
Jens

1 Like
4

Thanks @akassab, this question about migrating userkeys stored in entity properties still hasn’t been addressed as far as I know.

1 Like
5

Hi @akassab,

we are building an issue synchronization app where we also allow to sync between server and cloud based Jiras. In order to map users, e.g. assignee or reporter, properly, we rely on user’s mail addresses. We’d be highly interested in becoming an approved use case - or telling our users an easy way to e.g. switch users’s privacy settings for their whole Jira site.
So we’ll eagerly awaiting your next news.

Cheers,
Matthias.

1 Like
6

Hi @akassab,
we are using the atlassian-connect-express framework, which has partially been updated to support the GDPR changes but not fully.
For example, what will happen to the HostClient.asUser() method, which is used for impersonation? I believe it should take care of calling the new “migration” REST API to convert the userKey into an accountId. Will this be implemented soon?

In fact, after checking the code, I realized it’s all currently supposed to be handled by the atlassian-oauth2 library (Node module), and that library calls the https://auth.atlassian.io server passing a userKey. Will https://auth.atlassian.io continue to support userKey-based claims?

2 Likes
7

@jbevan - You are right. I’ve raised with the Jira team and should have an update for you next week. (CC: @bkelley)

8

@jens - We have not as of yet, discussed the un-deprecation of timezone in context parameters. For now this decision only applies to the JS API. I will follow up on this to see if its feasible and provide an update in next week’s post.

9

@david2 -

Will https://auth.atlassian.io continue to support userKey-based claims?

No. The developer documentation calls out that the sub attribute needs to use accountID instead of user key.

In terms of updating the method to use the new migration REST API to facilitate conversions, this is not something we are currently working on, nor had in our immediate plans. The new migration end points are intended to be temporary so if this is something that will speed up migrations we may consider it. I will speak with the engineering teams and get back to you in next week’s update.

10

Hi @akassab.

I did not met mention of [ACJIRA-1625] - Ecosystem Jira (accountId in jql ‘status changed by admin’ (username but accountId not supported)).
Please clarify.

Thank you.

11

we are using the atlassian-connect-express framework, which has partially been updated to support the GDPR changes but not fully.

It is great to see you updating to a new version of connect-express. We believe a new version has been updated to handle all GDPR changes. I will address your examples below, but if you have any further concerns that we might have missed please let us know further details.

For example, what will happen to the HostClient.asUser() method, which is used for impersonation?

As per ACE readme - we recommend to move from using asUser() to asUserByAccountId(). Please see examples in the readme.

I believe it should take care of calling the new “migration” REST API to convert the userKey into an accountId. Will this be implemented soon?

The framework will not perform the migrations. The framework and API provide new accountId based methods for addons to use. For the present both the old mechanism and the new accountId based mechanism are supported allowing you to migrate your usages. After the deprecation period no connect call to app server should contain userKey values. Thus using asUser() APIs won’t be possible. So there will be no way to convert userKey to accountId. If your app is ready to use new APIs then it should use accountId received instead of userKey. This accountId can be used to the same effect as userKey - to make a call on behalf of a given user identified by that accountId. That is why there is no reason to convert a userKey into an accountId.

Hope that clarified intended use of the new version of ACE & API.

12

Hi @iryabov,
I believe you are not aware of the discussions that have been taking place between Atlassian and some Top Vendors about this issue (elsewhere on CDAC and on Trello).
The problem is that there is no API to modify existing workflows, so until customers modify them manually, our app will still be holding a user key as the “run as user” for post-functions created using JMWE. So we need to convert that userkey to an accountId, which can be done using the migration API. However, this is an asynchronous operation, whereas the asUser and asUserByAccountId methods are synchronous (the user key/accountId to impersonation token is generated later, when an actual Jira REST API is called). So it would make much more sense to do the user key to accountId mapping when requesting the OAuth token (since it is already in an asynchronous operation). Otherwise, it has a significant impact on our code (moving from synchronous calls to promises everywhere).

Anyway, we’ll continue using patch-package to patch up ACE, it won’t be our first patch and it won’t be our last :frowning:

13

Hi @david2, do you have a link to these discussions

Thanks
Paul

14

@paul they are scattered across multiple discussions, such as:

Etc.

But the bulk of our discussions with Atlassian happened on private channels (email and others) because our problems are only relevant to a handful of apps (those offering workflow post-functions).

15

Thanks for that, I see I’m in one of them :slight_smile:. Just looking for more answers
Paul

16

@jens - We will not reverse the decision to remove timezone and locale from context parameters because they would be passed by URL. We are requiring the action to be explicit when retrieving PD even timezone and locale which are indirect identifiers.

17

Thanks @david2. These comms have been all over the place (literally) which is why we’re now trying to consolidate known issues into a single weekly post. @paul please raise issues here if you’re blocked by issues not mentioned.

I’ve just released this week’s post which has the latest status for the above including those that were raised in comments: Weekly GDPR API status development update - February 13th

18

Hi @akassab

My main worry is what to do about user names stored in workflow post function data. I have asked this multiple times before but just been met with silence from Atlassian.

We are meant to migrate to account id or provide an API for Atlassian. At this stage I’m working on premise that if it is stored in Jira workflow then it is not my server so we don’t have to provide an API.

This migration API, will it let us search on username after the deadline. It’s possible that we will still have workflow post functions containing usernames after the deadline since there is no way to migrate them.

Thanks
Paul

19

Do you have any update on [ACJIRA-1692] - Ecosystem Jira?

Changelogs coming in webhooks (from to values) reference to usernames instead of account ids.

20

Hi Fernando

You can get the account id from the JWT Token.

Thanks
Paul