RFC-19: Deprecation of Confluence Cloud REST API v1 Endpoints

General Request for Comments (RFC)
, , ,
1

RFCs are a way for Atlassian to share what we’re working on with our valued developer community.

It’s a document for building shared understanding of a topic. It expresses a technical solution, but can also communicate how it should be built or even document standards. The most important aspect of an RFC is that a written specification facilitates feedback and drives consensus. It is not a tool for approving or committing to ideas, but more so a collaborative practice to shape an idea and to find serious flaws early.

Please respect our community guidelines: keep it welcoming and safe by commenting on the idea not the people (especially the author); keep it tidy by keeping on topic; empower the community by keeping comments constructive. Thanks!

Summary of Project:

Driven by unpredictable behavior, difficulty in optimization, and incompatibility with granular OAuth 2.0 scopes, a set of endpoints from the Confluence REST API V1 are being deprecated. These issues impact developers and users of apps that rely on the API for data fetching and manipulation in Confluence Cloud. The new Confluence REST API V2 aims to solve these problems by offering endpoint specialization, cursor-based pagination, and improved compatibility with granular OAuth 2.0 scopes.

The deprecated endpoints can be referenced here and here. Public access for the endpoints will be removed on Jan 1, 2024, except for the Content Property endpoints & the “Update inline task given global ID” endpoint, for which public access will be removed on Feb 1, 2024. This means any apps still using these V1 APIs will no longer have access and will need to move to the newer V2 endpoints.

Timeline for the RFC:

  • Publish: Aug 7 2023
  • Discuss: ~Aug 21 2023
  • Resolve: ~Sep 4 2023

Problem

The problem we are addressing is the limitations and performance issues with the previous Confluence REST API V1, for which a subset of endpoints are being deprecated. These issues impact developers who build and use apps that rely on the API to fetch and manipulate data in Confluence Cloud. The impact of this problem includes unpredictable behavior, difficulty in optimization, and incompatibility with granular OAuth 2.0 scopes.

It is essential to begin migration as the deprecation period for the REST API V1 has started, and public access to these endpoints will be removed on Jan 1, 2024 and Feb 1, 2024 (see above). By solving this problem, we can improve the performance, predictability, and ease of use of the API, ultimately enhancing the experience for developers and users alike.

The V1 API faced challenges such as.

  1. The lack of endpoint specialization in V1 makes it difficult to predict behavior and optimize performance. Moreover, it doesn’t fully support the granular OAuth 2.0 scopes that we’re striving to implement.
  2. The use of expansions and the expand query parameter in V1 complicates the process of fetching specific data types from a single endpoint. This, in turn, leads to increased request latency and challenges in optimization.
  3. V1’s offset-based pagination has shown some drawbacks, such as missing data when adding or deleting items during pagination and worse latency for high-offset requests.

The affected endpoints are:

Proposed Solution

Our current proposed solution is the Confluence REST API V2, which offers several improvements over the deprecated REST API V1. These improvements include:

  • Endpoint specialization: Separate endpoints for each data type, such as pages, blog posts, comments, attachments, custom content, labels, spaces, and much more.
    • This allows for better predictability and compatibility with granular OAuth 2.0 scopes.
  • Cursor pagination: Instead of the offset-based pagination present in REST API V1, REST API V2 uses cursor-based pagination, providing better latency, and preventing missing data issues.
  • Improved performance and ease of use for app developers and users.

When we originally posted our deprecation announcements, the community pointed out many gaps. We reviewed over 70 responses on our original announcement, and over 50 additional topics related to gaps. We made sure that we have addressed all the critical gaps identified therein. Many of these additions were noted in this changelog and in this changelog.

The only set of endpoints which has not yet released (but is nearing release) is:

  • Get content properties for (for all entities of specified type)

App owners and developers using Confluence Cloud APIs should plan to migrate to the REST API V2 endpoints before the above dates. If you have any concerns or difficulties migrating, or if you encounter capability gaps or unexpected behavior, please let us know by responding to the community post. We will create and link corresponding CONFCLOUD tickets to capture and track the progress of migration blockers.

Asks

We kindly ask for your valuable feedback on any gaps between REST API V1 and V2, as well as any potential blockers that you might face during the migration process. Your insights will help us improve the REST API V2 and ensure a smooth transition for all developers and users.

Thank you in advance for your collaboration and support in shaping the future of the Confluence REST API!

9 Likes
2

Hasn’t this been abandoned?

Putting on my solution partner hat, if Atlassian would proactively create a per app ticket with a list of all endpoints that each vendor is still using based on access logs, it feels like there would be less likelihood of an app vendor overlooking something and there being outages come deprecation time.

Also, Jan 1 may be a suboptimal day for deprecations to start happening as customers, app vendors, and Atlassian staff will likely have vacations in progress. Maybe push it a week or so?

10 Likes
3

You’ll probably want to push it at least 2 weeks because of Europeans. We like our vacations…

12 Likes
4

It’s also the middle of the southern hemisphere summer holidays.

2 Likes
5

Quick feature gap that comes to mind for one of our internal apps: the API for creating comments on custom content objects.

In v1 the create content API allowed creating (footer) comments on pages, blogposts and custom content by just setting the parent content ID accordingly.
In v2 you can only pass a pageId or a blogPostId, no custom content ID so far. Same for the endpoints to fetch footer comments on custom content.

4 Likes
6

Thank you for requesting our input on this. It is appreciated :+1:

I do have a some points:

  • My biggest concern is that differentiating between pages and blogs, which are both content types that accept macros from our apps, is going to create unnecessary complexity in our apps. We have to code the apps to use the appropriate APIs to basically just get some fields that are common between them. So it shifts complexity from you to us. Plus the migration work that would have to be done. If we could just get an endpoint that would just return some common metadata between the two please? Stuff like title, space, version, body? (almost all of our Confluence apps are macros, so this is my main point)
  • Basically what Boris already said. It would be hugely helpful if you could somehow let us know which deprecated endpoints our apps are using. There could be a bunch of factors why we would miss our apps using these endpoints, so as a vendor who’s responsible for a decent amount of apps, this would be a great help. Ideally this would be updated on a weekly or so basis as the deprecation period progresses so we can make sure we’re not missing anything. Ideally, this could be used as well for other API deprecations.
  • This does not affect us I think, but I could see some people being put off by the cursor change which does not provide the total number of items to be returned. This makes paginated lists (such as the Dynamic Table Atlaskit component) terrible to use from a UX perspective because you never know how many pages there are remaining.
  • I think the GraphQL APIs have some great potential to be replacing the V1 REST APIs as well, but they do not support some of the features we would require. I think the big one I bumped into recently was its inability to handle unpublished pages / page edits, but I’m not 100% sure about this. Also it would be nice if the GraphQL API docs would contain the OAuth2 scopes required for a certain field.

Thank you for your time and consideration!
Cheers,
Tobi from resolution

5 Likes
7

Some things we have struggled with so far when switching away from the deprecated endpoints:

  • No way to get favourited spaces in v2 API
    • v1 API: /wiki/rest/api/space?favourite=true
  • No simple way of getting labels used in a space in v2 API
    • v1 API: /wiki/rest/api/space/{spaceKey}/content?expand=metadata.labels
    • In v2 API I would have to: Get pages & blogposts on a space → Then get labels separately for each page & blogpost, turning the one v1 API call in to huge amount of v2 API calls.
  • Get pages from space method of REST API v2 missing depth parameter
  • No way to get page authors in v2 API
    • Confluence API v2: How to get page authors information? - #4 by SimonKliewer
    • We use the logged in users permissions to make our requests against Atlassian APIs (ACT_AS_USER user impersonation), this ensures that we are not exposing data to users that don’t have the rights to view it. The suggested solution is to use our apps permissions to fetch this information instead of the users because the user might not have the rights to call the user search API. Wouldn’t that mean that we are circumventing Atlassian set permissions?
4 Likes
8

Thanks to @SimonKliewer and team for all of their hard work! Here are some points to consider:

1. Honoring Deprecation Timelines

I would like to suggest an adjustment of the deprecation dates to respect Atlassian’s six-month deprecation policies:

The post mentions that the sunsetting of V.1 content property endpoints will be delayed one further month (to Feb 1). If I read between the lines, the post implies that this is being done because the V.2 “Get content properties for (all entities of specified type)” APIs are not yet ready. I assume the goal is to sunset APIs only after the replacement has been available for the six month deprecation period.

I think this misses the subtlety that the V.1 APIs being replaced by the indicated V.2 content property endpoints are not, in fact, the V.1 “get content property” APIs. Rather, it is the deprecation of the expansions on general content fetches that are forcing developers to use those new V.2 APIs.

For example, I can currently fetch a bunch of entities along with their content properties in V.1 through GET /rest/api/content/XYZ?expand=metadata.properties.foobar. This no longer exists in V.2, so to avoid doubling the number of requests, we fetch all of the content first, collate the IDs, then batch up a request to receive multiple content properties at once through the as-of-yet unreleased V.2 content property APIs.

If you sunset the general GET /rest/api/content API in January but those content property APIs are not released until late this month, we have four months rather than six to implement the replacement. Perhaps it would make sense to standardize all of the API sunsetting and implement everything all at once on March 1? This also helps with the vacation issue previously mentioned.

2. Dogfooding

The post indicates that Atlassian will be removing public access to the V.1 endpoints, implying that Atlassian may still use those endpoints internally. Out of curiosity, are any projects within Atlassian dogfooding this work, and has it uncovered any gaps?

3. Whiteboards

Atlassian has rolled out an entirely new content type (whiteboards) that supports only the V.1 API.

How is this being integrated into the V.2 API work, and do we have a timeline? How can developers be assured of having reasonable time (and a reasonable deprecation window) to support this feature?

Even if whiteboards cannot be directly interacted with via the API, I believe they are now officially part of the content tree, so applications relying on only the V.2 API will need some way to deal with pages or other content types that have whiteboard ancestors or children.

If whiteboards are only supported in V.1 now and the V.2 replacement is not out until (say) October, this gives developers very little time to implement changes, and perhaps this too points to moving the sunset dates.

9 Likes
9

This is a really important point that has felt lost to me during this entire notification and enactment of the deprecation period. Despite being extended, the deprecation period clock has been ticking since the deprecations were announced, rather than once the replacement APIs were deemed both complete and stable.

5 Likes
10

Dear @SimonKliewer ,

thank you for your additional post.

Honestly I have no idea which of the comments/concerns raised by the community have been picked-up by Atlassian and if yes when it will be implemented. Wouldn’t it be possible to list all of them somewhere, give them a status and an expected time until they will be implemented?

Thank you

Andreas

1 Like
11

Would it be possible for Atlassian to add a response header indicating that a deprecated API has been used? This would make it much easier to detect where we are still using deprecated APIs when running our automated tests, and would lower the risk of surprise when the switch is eventually flipped to disable the deprecations.

This would be a huge benefit for our code base given that we share code between DC and Cloud and could instrument our tests to warn anytime such a header was observed.

7 Likes
12

For us the blocker to migrate to spaces REST API V2 is that we cannot get space labels / categories.
See Will there be a way to get space labels/categories in V2 API?

Kind regards,
Raimonds

2 Likes
13

Hi @SimonKliewer ,
Thanks for the RFC.
I must say that the indicated deprecation period is problematic. We tried to use some v2 endpoints, but not all replacements for v1 endpoints were ready. At this moment I’m lost with what is already working and what not, and we don’t have the capacity to try to migrate every couple of weeks.
The 6 month deprecation period should start once the v2 endpoints are stable.
Just last month the v2 format changed from integers for ids to the intended strings due to an Atlassian release error.

6 Likes
14

Why remove access to deprecated endpoints? The v2 endpoints are already versioned which is literally the point of versioning APIs.

And second other points in here regarding notification. Given that you’re blanket deprecating dozens of endpoints all at once, you should actively inform us developers now as to which of our apps are using deprecated v1 APIs.

1 Like
15

Second the comment from @nathanwaters. Why is Atlassian removing an API that works to replace it with an API that forces rework for all of your partners?

The v2 API as is will SLOW DOWN our existing apps because without expansions we are forced to make multiple API calls in series, with additional latency, authentication, error handling for each.

Simple things, like fetching a page, current version history and 1 named property can be done today in 1 round trip and return a response to the customer user. V2 API will force 3+ round trips. If the connect app services are running on a different continent than the customers ATL cloud instance we are adding 500ms+ seconds to every user response.

Forcing unnecessary rework by providing less functionality and lower application performance is not serving the Atlassian customers or partners best interests. Recommendation: Build a better v2 API that is compelling because of function and performance, and people will move to it.

Sincerely
Chris

6 Likes
16

@SimonKliewer Are you planning a replacement for the childTypes.* expansions? This is currently a blocker for us because we are showing a page tree in our app (similar to the tree in the Confluence side bar) and we need to know if a page has children without fetching the children to show the correct UI.

3 Likes
17

Thanks all for your feedback so far!

For the ideas brought up:

  • Atlassian providing some sort of tooling for vendors to be able to request a snapshot of their apps and corresponding deprecated endpoint usage → this is a great idea, we will investigate this idea and present our findings in the “Resolve” phase.
  • Adding a header to these endpoints to help developers understand when a deprecated endpoint has been used → Same as above!

For another theme that has been noted:

  • We acknowledge that in certain scenarios, the number of requests required from apps to fetch the same amount of content from V1 to V2 will increase. We are moving forward with this tradeoff for the reasons outlined in our blogpost here. However, we are confident this is a step in the right direction, and want to work with the community to make this transition as smooth and performant as possible. For example, the upcoming content property endpoints are a response to this concern → a new set of endpoints that will help cut down on the amount of calls required.

For other feedback:

@thomas2 Thank you for your input! This CONFCLOUD can be tracked.

@tobitheo Thank you for the feedback! With V2, we are sticking with more strongly typed endpoints, and along with this comes the distinction between pages and blogposts. This is a case where the count of calls with need to increase.

@MiikaMkiniemi Thank you for the comment! Re: favourited spaces filters - this CONFCLOUD can be tracked. Re: space labels - these CONFCLOUDs (labels on space, labels on space content) can be tracked. Re: Depth parameter - this is now addressed! Re: Page authors → Thank you for the explanation here! I am still working with an internal team to determine the best path forward here, and will let you know the consensus we come to!

@scott.dudley Thanks for the feedback here! I am currently working with the corresponding internal teams to get a more extensive update, but in the meantime I have created this CONFCLOUD for you to track its progress.

@andreas1 Thanks for the comment! Aside from the CONFCLOUDs linked above, here are the other two CONFCLOUDs we have open as valid migration blockers: CONFCLOUD-76295, CONFCLOUD-76347. These 2 tickets combined with the tickets linked earlier in the message round out all current migration blockers.

@raimonds.simanovskis Hi - the 2 linked tickets above for space labels can be tracked! Thanks for noting this.

@klaussner Thanks for the question! In the current state, you will need to use the child endpoints to check for child pages. In the future, we hope to introduce a set of endpoints that will return metadata of all children/descendants of a piece of content, regardless of their type, similar to how our ancestors endpoint works. But in the meantime, the recommended way forward is to use the child endpoints.

1 Like
18

Blog post and your reply don’t address why the need to remove access to v1 API.

It’s clearly not an immediate security issue. Your team needs a legitimate reason for such a large breaking change that will force 10s of 1000s of cumulative hours of unnecessary rework on developers.

Every competent tech company versions their API in order to allow the evolution of it while maintaining backwards compatibility and preventing breaking changes.

While Atlassian somehow can’t go from v1 to v2 without breaking every god damn Confluence app on the Marketplace, companies like Stripe version their API based on a timestamped date:

When a breaking change is introduced to the Stripe API, a new dated version is released. To avoid breaking your code, we don’t change your version until you’re ready to upgrade.

Whereas what you guys are doing here is rug-pulling developers, breaking their code and forcing countless hours of unnecessary rework to migrate to an inferior v2 API. This is precisely the type of non-technical bureaucratic middle-manager decision that Atlassian’s internal “flattening” was intended to stamp out.

The team behind the Jira API seem to be more concerned about developer experience considering both v2 and v3 APIs are still active and accessible. WTF is going on with the Confluence team?

How to encourage devs to migrate to v2 without f^^king them:

You keep v1 active, mark it as deprecated in the docs, stop adding new features and deprioritize support. You then add new features to v2, promote its benchmarked performance improvements and prioritize support.

If maintaining v1 becomes too much of an overhead, say five years from now, and there’s only a handful of vendors still using it, then it makes sense to do a forced migration.

–

Given the shitty siloed endpoints of the v2 API, is the GraphQL API at parity and not on the chopping block?

With the literal 100s of wasted hours you’re forcing on me, I’d like to build on a stable API that isn’t going to be rug-pulled in the future. There’s no point building with this new v2 API if you’re just going to kill access to it when v3 arrives.

3 Likes
19

Hi @SimonKliewer ,
You haven’t answered how the deprecation date of the v1 API fits with the v2 API not being ready.
In your answer you write:

We don’t know when these “upcoming” endpoints will be delivered, but we hear the existing endpoints are deprecated.
Deliver first, then deprecate!

5 Likes
20

@nathanwaters Incidentally, the GraphQL API is not available to Connect apps and I have not seen any feedback indicating that this is likely to change for the foreseeable future, so it is not an option for the majority of active apps.

3 Likes