Deprecating many Confluence v1 APIs that have v2 equivalents

Confluence Development Confluence Cloud Announcements
, , ,
1

What is changing?

Hello everyone,

Confluence Cloud is announcing the start of a deprecation period for a number of v1 API endpoints, please see the changelog: https://developer.atlassian.com/cloud/confluence/changelog/#CHANGE-864 for full details.

We intend to remove public access to these endpoints following the 6 month deprecation period, which means any apps still using v1 will no longer have access and will need to move to the new, v2 API endpoints.

Why is it changing?

We are deprecating in favor of v2 endpoints that provide a couple of key benefits over v1, specifically:

  • Better performance, especially for bulk endpoints! Moving to cursor-based pagination has provided significant performance gains over v1’s offset-based pagination, especially at high offsets. This should be a big win for apps that go through all content on Confluence.
  • More granular endpoints, that allow you to be specific about the type of data being used. Not only does this result in more clear documentation and functionality, but will help prospective app consumers feel more confident about the type of data apps will have access to.

What do I need to do?

If you are the owner of a Confluence Cloud app, or are otherwise using the Confluence Cloud APIs, please ensure you are not using any endpoints marked for deprecation. If you are, plan to migrate to v2 API endpoints before September 1st 2023. If you have any concerns or difficulties migrating, please respond to this post. We do not intend to remove functionality, so if you are unable to migrate due to a capability gap or unexpected behavior, please let us know!

By when do I need to do it?

The six-month deprecation period for these changes begins on March 1st 2023. Public access to these endpoints will be removed on September 1st 2023.

2 Likes
2

Hi @AlexWilliams

Thanks for posting this update. I understand that one of the major goals of the new API is speed improvements.

In the post where the v2 API was announced, it mentions that the v2 REST endpoints do not support expansions (by design), and “API consumers who require expansions should consider using the Confluence GraphQL API”.

The Confluence GraphQL API docs say that “fields marked with Beta can change without notice”. Essentially all of the useful Confluence GraphQL fields are marked as beta (ie. pages, blogposts, spaces).

Will Atlassian move the GraphQL APIs out of beta before going through with the deprecation of these REST v1 APIs? I have admittedly not yet even looked to see if the GraphQL API would support the expansions that we rely on from the v1 APIs, because this is irrelevant if the GraphQL API is still beta (we cannot ship with it).

We (and presumably other API clients) rely significantly on v1 API expansions, and although the promised extra speed of the v2 API calls may lead to faster individual calls, I suspect that this will lead to an overall slower user experience since we now need to replace one API call with multiple roundtrips.

20 Likes
3

This would have been a great opportunity for Atlassian to create an RFC before categorically announcing the deprecation period, would it not?

12 Likes
4

Hi @AlexWilliams ,

Thanks for the update. I’m a huge fan of the v2 APIs and we do already use them in production.

However, we still need the v1 API available, as the v2 APIs are not completely functional for Forge apps yet. I’ve outlined the issue we’ve encountered in [FRGE-1147] - Ecosystem Jira.

The TL;DR is that v2 relies on headers for pagination and Forge swallows most headers, including the Link header.

It would be nice to have a commitment that the v1 APIs will not be deprecated before the aforementioned issue has been fixed.

Best regards,
Oliver

4 Likes
5

Hi @AlexWilliams

Thanks for this – I’m moving things as quickly as I can.

I hear talk of the “Atlassian Platform” which suggests that all products have been planned out and designed together. e.g. all products use the same editor.

Jira has it’s APIs at:

  • https://<site-url>/rest/api/2/<resource-name>
  • https://<site-url>/rest/api/3/<resource-name>

Confluence at:

  • https://<site-url>/wiki/rest/api/<resource-name>
  • https://<site-url>/wiki/api/v2/<resource-name> (note lack of /rest & v2 rather than 2 in the URL here)

Once we’ve been weaned off of the unnumbered Confluence v1 APIs and they are turned off, will you be moving to something more consistent?

Perhaps switching to Jira style naming for consistency?

  • https://<site-url>/wiki/rest/api/2/<resource-name>
  • https://<site-url>/wiki/rest/api/3/<resource-name>
2 Likes
6

We also use expansions a lot. Can any body from Atlassian explain how we can move to the new APIs without being equivalent.

Thanks

6 Likes
7

This is a significant change and should be handled carefully. Are we really sure that API v2 is equivalent? I have several doubts about this.

We have apps that heavily use these APIs, which will mean rewriting much of them. Also, this is not the first time Atlassian has brutally announced the deprecation of an API, without providing a truly equivalent solution.

6 Likes
8

I’m a little concerned about the timeline of this deprecation. There are a number of features that our app uses for which I couldn’t find equivalents in the v2 REST or GraphQL APIs:

  • Get content:
    • Finding out if pages have children (without fetching the children).
  • Create content:
    • Creating a page as a child of an existing page.
  • Get space(s):
    • Getting the space icon. (GraphQL only)
    • Finding out if the requesting user is a space administrator. (GraphQL only)
    • Getting a space by key.
  • Get content for space:
    • depth=root parameter, which returns only the first level of pages in a space.
  • Get space property:
    • Getting a space property by key.

Is there a public roadmap for the v2 and GraphQL APIs? That would be very helpful for planning our migration to the new endpoints.

9 Likes
9

To be clear, for anyone who has an app that runs against both Confluence Cloud and DC and uses substantial API surface area, this deprecation is actually saying that we need to migrate from 1 API to 4:

  • v1 for both Confluence Cloud and DC where no new v2 exists
  • v2 for Confluence Cloud where v1 is being deprecated in Cloud
  • v1 for DC where v1 is being deprecated in Cloud
  • GraphQL in Cloud where v2 lacks expand parity

And that’s assuming that v2+GraphQL actually covers all scenarios handled by v1, which itself will take some time to properly evaluate. 6 months seems very aggressive to me.

7 Likes
10

Further to my earlier comment, I see that GraphQL is apparently not even available to Connect apps (!), making whatever GraphQL supports a moot point for the vast majority of existing apps. The last update I saw (Nov 22) said that there was no timeline to make GraphQL available to Connect.

There is also a lot of good commentary and criticism on the earlier thread where the beta v2 API was originally announced.

@TylerBrown mentioned there in October:

While we work towards moving away from expansions where possible, we would love to hear more of your feedback on how you are using expansions, the pain points removing them would cause, and anything we can do to make the transition easier. Once we know these we can work to make sure our V2 APIs are providing the appropriate solution.

I would be interested in closing the loop on this comment and understanding how these have been (or will be) addressed!

Another relevant comment is this:

As you all mention, removing expansions will cause the number of network requests to increase. This is a problem we are aware of, but we are okay with it for a number of reasons.

I think the concern here is that, even if Atlassian is OK with the additional network traffic, it seems likely to cause substantial overall performance penalties to us as vendors, so I think a lot of us are less “OK” with it than Atlassian might be.

Before Atlassian takes away the v1 API, I would like to suggest that GraphQL should:

(1) be made available to Connect apps,
(2) be made functionally equivalent to the v1 APIs that are being removed, and
(3) taken out of beta.

I imagine that it is undesirable to maintain two versions of the REST API at the same time. But please do not take away features that our apps rely on without providing equivalent replacements.

13 Likes
11

I looked at two simple APIs used by an existing app. As far as I can tell, neither has an equivalent in the v2 API, and even if they did, the overall execution is still 2x slower.

Example 1. Attachment Information

For the following API call to fetch attachment information:

244 ms - /rest/api/content/<attachmentid>?expand=container,version,history.previousVersion

This becomes the following three requests:

1- 229 ms - /api/v2/attachments/<attachmentid>
2- 220 ms - /api/v2/pages/<pageid>
3- 247 ms - /api/v2/attachments/<attachmentid>/versions

We can probably call #2 and #3 in parallel, so the total time is 229+max(220,247) = 476 ms (96% slower).

But even worse than the speed problem is that we do not have any way to construct API call #2. The response from call #1 to fetch the attachment data does not tell us the contentId or contentType of the container, so we do not know which pageId to fetch, or even whether the container is a page or blog, so we do not know which endpoint to use.

This problem extends beyond just the attachment API call: any app that is unlucky enough to have persisted contentIds must now also record the contentType along with the contentId, so that we can use the correct API endpoint to access the content. Apps who are doing that will presumably need to backfill that data before the v1 API goes away.

But now apps are storing denormalized data, which seems to create further problems. When a customer converts a page to a blog post, the contentId seems to remain the same, but it is now a different content type and we can no longer access it with the same endpoint. What method can apps use to keep track of that?

Example 2. Space Permissions

For the following API call to fetch space information:

267 ms - /rest/api/space/<spacekey>?expand=permissions,permissions.subjects.group

This becomes more than one API call:

212 ms - /api/v2/spaces?keys=<spacekey>
??? ms - ???

Although this is 20% faster, we do not have any permissions data at all, and I am unable to see a non-deprecated API that produces the equivalent. Where do we get the space permissions data now?

==

In summary, the thread title is “Deprecating many Confluence v1 APIs that have v2 equivalents”, but I think the premise (that the v2 API provides equivalents to everything being deprecated) is mistaken. I strongly urge Atlassian to reconsider the deprecation until the v2 API and/or GraphQL is more mature.

17 Likes
12

Thank you @scott.dudley for pointing out that getting Space Permissions is a dead end - I’m afraid there are more.

Another example which results in a significant increase in processing time and requests towards Confluence Cloud (in former times I learned to use expands to limit the number of calls…): in one of our apps we need our custom content from all ancestors of a page. This means we must perform one call to get the page and its parent, another call to get the custom content and then continue with the next parent. In case of 10 ancestors this means 20 calls instead of one. Is this what you wanted Atlassian?

1 Like
13

The proactive approach here would be to monitor incoming API requests and send periodic email notifications to vendors saying “X app is using these deprecated APIs, here’s how to migrate each of them”.

1 Like
14

@AlexWilliams We have a similar use case in several of our apps. Did you consider adding something like a /page/{id}/ancestors endpoint so we can at least get all of those in one call?

Another minor thing is that v2 APIs require us know before the call of what type a certain content is, because there is no generic /content endpoint anymore. This also involves extra calls because we need to guess: 1. page, 2. blogpost, 3. custom, etc.

4 Likes
15

I am bemused why on v2 API the only supported body-format values are storage and atlas_doc_format. We use the view (HTML) format and so now have to resort to a GraphQL request to get this?

We tried migrating to GraphQL but found no way to get all pages in a space, so need to use the REST API for at least that.

With the v1 REST API we could get all pages and their content in a space in one request.

1 Like
16

Dear @AlexWilliams

another dead end might be getting the likes (replacement for /rest/api/content/{id}?expand=metadata.likes.users). I did not find an alternative in v2 (/api/v2/pages/{id}/properties does not return the likes/reactions).

Andreas

1 Like
17

I think this is an interesting development and it remains to be tested how this will impact the performance of our app. From experiments I know that expansions have a significant impact on the performance of requests in the existing API. Different parts of our app require different expansions, but it would be very complex to implement the API calls in a way that each component only loads the expansions that it needs, so we have a fixed set of expansions for pages, blogposts, custom content, etc that we use throughout our app, causing us to sometimes fetch data that we don’t actually need in that moment. The new API will force us to treat each expansion as a separate object that we need to fetch with a separate request. The advantage is that it will make us fetch only the data that we need in a particular moment, which could improve performance. The disadvantage is that there will be situations where requests need to be made as a waterfall, as the parameters of each request depend on the result of the previous request, which will drastically decrease performance.

I haven’t looked at the API in detail yet, so I don’t know yet if there are endpoints for each expansion that we currently load. What I do notice though is that the endpoints reference all content types by ID. As far as I’m aware (and I would be very happy to learn that this has changed), the way to reference content (in a way that will be detected as incoming links and that will survive space export/import) in Confluence is the space key + page title for pages, the space key + content title + posting day for blogposts, the space key + page title + file name for attachments (even though there can be duplicates), and for custom content there is no way to reference them unfortunately (this would be a real game changer for several of our apps). It would be nice if the v2 API endpoints supported these identifiers, so that we don’t have to look up the content IDs first in a separate request (to the convert body API).

4 Likes
18

I shared our use case in more detail on the original blog post announcing the v2 endpoins:

I think that our team is fortunate in the way that all the data we use - at least in the most important use cases - is already available on the v2 endpoints. And I can also see many positive things about the change, for example the cursor based pagination.

My main concerns are about performance - apart from the significant amount of work that will be needed for migrating away from the deprecated endpoints.

One use case that our app has, is to get all pages in a space (or a page tree) and expand labels, children, and content properties. Doing that with the v1 endpoints and expansions is not always very fast, but it’s not very slow either. Typically getting a response page that has the required data for 200 pages takes something like 5 to 10 seconds.

With the v2 API we will first need to get a response page with basic information about 200 Confluence pages, and then additionally do 3 REST calls for each of those. So we will need to do 600 (+1) REST requests to get the same data that we got with one v1 request.

If v2 API has around 200ms response time for each of those 600 requests, it would take 600 * 0,2 seconds = two minutes to get all the responses if we would run the requests one after each other.

So we obviously need to do lot more requests concurrently than we’re doing now. To get to the 5-10 seconds time that our customers expect using the v2 endpoints, about 15-30 request would need to be run in parallel all the time.

Additionally, we’re using several threads for handling different spaces / page trees in parallel.

So to get actual performance benefits we would probably need to have at least 100 REST requests to v2 APIs in-flight at the same time.

–

But maybe this is not a problem with the v2 APIs. If that’s the case, then this change can be great for our use case, especially if it also brings stability benefits. We have seen also cases where the v1 endpoints get very slow even though that’s not very common with the simple expansions that we use.

4 Likes
19

And here is the next dead end (without applying any workarounds): in some of our apps we store the attachment id as reference to an attachment. For usability reasons we need to show the user the page holding this attachment also later when e.g editing a macro. The /api/v2/attachments/{id} however does not return the container and extracting it of the webui feels kind of wrong.

Of course we could use /rest/api/content/search?cql=id={id}&expand=container for this but I’m asking myself how long this method will work.

@AlexWilliams sorry for being honest but I have the feeling that Atlassian has not acted prudently with this deprecation.

1 Like
20

Hey Scott,

Thanks for your feedback here!

We’ll be moving the Confluence GraphQL fields out of beta shortly, in their current form, so they’ll be an alternative for Forge/3LO apps looking for expansion-like behavior. Unfortunately, since Tyler’s last post, I don’t have any further updates to communicate on the Connect → GraphQL integration being unblocked.

With regards to expansions, and some of the feedback within the previous post and this post, we’re looking at expanding the filters available in our bulk endpoints to support bulk ids of other, related entities. This way, you can cut down on the number of network calls necessary in these cases. We’re also looking at treating some related entities similarly to v1’s expansions, where they can be optionally included, if the entity is not distinct enough to live on its own (page body and space description are existing examples) and will not cause significant performance penalties when fetched. However, we’re still following the principle from Tyler’s post that we need to keep separate content types’ data separate.

To your examples, we’ll add container id/type info to the attachment by id endpoint, and are looking to provide a bulk-supporting id → content type method to allow apps that have stored data this way to migrate. For space permissions, we’re evaluating how to best offer this data.