Broken Java API in Confluence 8 EAP: ContentSearch, getById(), etc

There is also another method that has been replaced without workarounds. The method ScheduledJobManager.disable accepts a ScheduledJobKey in Confluence version 7.10 and is replaced in Confluence 8.0 with JobId.

1 Like

Very nice. Don’t forget to like or create your own post, so these problems bubble up and community managers are incentivised to write a tutorial.

3 Likes

Hi All,

Replying to the above.

From the first block

Classes such as SiteSearchPermissionsSearchFilter have been removed, so we can’t apply user permissions when searching (which is a security breach awaiting to happen)

All the classes the implement the SearchFilter interface are still available in 7.20. They will only be removed in Confluence 8.0 as all SearchFilter will be replaced with SearchQuery. You should be able to find equivalent SearchQuery classes in Confluence 8.0. In the particular case of the SiteSearchPermissionsSearchFilter you can use the SiteSearchPermissionsQuery (available since 7.20) instead. Please refer to Searching using the v2 search API for examples

Last but not least, they change method signatures without notice!

** In 7.19: ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit)

** In 7.20: ContentSearch(@NonNull SearchQuery query, SearchSort sort, int startOffset, int limit),

** The ‘searchFilter’ argument was simply removed, no deprecation, it’s a public API class, so we can’t write code for 7.20 which works in 7.19, and we can’t write code for 7.19 that works in 7.20.

The signature ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit) was only deprecated since 7.20 and removed from 8.0. The interface SearchFilter and all of the classes that implement this interface was deprecated in 7.20 and removed from 8.0 in preparation for the future upgrade of Lucene library. This is required because as of Lucene 6 all of the Filter classes have been removed.

You can find additional information on our deprecated code paths removed in 8.0 page.

To your questions in the second block

How to perform a CQL search

CQL search can be done via CQLSearchService . You can find examples in the JavaDoc.

  • How to simply get a page by ID, a page version by ID and a space by key, and hopefully a page by title
  • How to modify the body of a page and save it as a new version (since getById() is deprecated, I suspect it is impossible)

Here are some example of the usage of ContentService:

Finding a page by page ID

contentService.find()
                .withStatus(ContentStatus.CURRENT)
                .withId(contentId)
                .fetch()
                .orElseThrow(notFound("No content found with id : " + contentId));

Finding a page by page version by ID

contentService.find(new Expansion("version"))
				.withIdAndVersion(page.getContentId(), page.getVersion())
				.fetchOrNull();

Note: in this example, we need to load “version” attribute in order to use withIdAndVersion
Finding a Space with Space key

spaceService.find()
                    .withKeys(spaceKey)
                    .fetch()

Finding a page by page title and Space

contentService.find(Expansions.of(VERSION, HISTORY, STATUS, SPACE).toArray()) // Load extra attributes: VERSION, HISTORY, STATUS, SPACE
                .withSpace(Space.build().key("SPACE_KEY"))
                .withTitle("Some page title")
                .withType(ContentType.PAGE) // This indicates which content type to be loaded: PAGE, BLOG_POST...
                .withStatus(CURRENT);

Update a Content’s body by ID

Content foundContent = contentService.find()
                .withStatus(ContentStatus.CURRENT)
                .withId(contentId)
                .fetch()
                .orElseThrow(notFound("No content found with id : " + contentId));
Content toBeUpdatedContent = Content.builder(foundContent)
	.body("new body", ContentRepresentation.STORAGE);
contentService.update(foundContent);

All of this using using genuine ContentEntityObject.java (Page, Blog, Comment, Space.java) and not a shell object like those returned by the search. If necessary, show us how to return shell objects into complete objects.

We’re attempting to remove dependancies on Hibernate objects. The ContentEntityObject is a Hibernate object. The service APIs expose similar capabilities, so we’d recommend using that API instead of depending on the ContentEntityObject previously used.

Hopefully this meets your exacting requirements.

Dutifully,
James Ponting
Engineering Manager - Confluence Data Center

5 Likes

Thanks for your response, but I think this does not help us at all. We are required to released apps which are compatible within the supported range of Atlassian, which means our apps have to be released with a compatibility range between Confluence 7.10 (EOL date: 15 Dec 2022) and including Confluence 8. As of right now, it wouldn’t be possible to release a single version of an app to cover this range. For example, if we now decide to use the new ContentSearch constructor introduced in 7.20, our apps cannot be used in Confluence <7.20.
How should we deal with this situation?

Best regards

Awesome, thank you James:

  • The code excerpts will help us, thank you very much (I haven’t tested, but that was what I needed),
  • The code deprecation-and-removal in 2 versions, while Confluence LTS spans from 7.13 to 7.19 to 7.20 to 8.0, is incompatible with the necessities of Data Center, which is to provide enterprise-grade experience with 2-year span of compatibility for plugins, to enterprise-grade customers. We can use Reflection for once, but it slows down Confluence; We can use different APIs for Confluence 7.13, 7.19, 7.20 and 8.0 and JDK 8 and 11, but it will impact the customers experience if we have to publish apps which are only compatible with Confluence 8.0. Unless you can convince all customers to use Confluence 8.0, oh I would be glad :wink: (yay JDK 11 for all!).

More seriously, may I suggest to loop back with the team to check whether it is possible for @PublicAPI classes to enforce signature-level compatibility for 2 years. We know it’s a hard job, we love you when you’re successful at it, and it’s sometimes just a matter of programming adaptors between old signatures and newer ones.

Best regards,
Adrien Ragot

@aragot It’s true that reflection slows down Confluence, but I think there is another way.

For starters, it’s not too difficult to create different internal components to be automatically injected depending on whether Confluence 7 or 8 is in use (see my code snippet on CONFSERVER-79630 for an example).

From there, you can use reflection to access the appropriate APIs, as you wrote. But using reflection is not actually forced by the runtime, because (with a few exceptions) it’s largely a compiler restriction. This is because your compiler can’t “see” the signature of the method you are trying to call, since it’s not compiling against that version of the dependency.

I believe it is possible to use a Maven multi-module project, with a submodule referencing a different version of Confluence, allowing you to write code within that module that can call the APIs directly.

I wasn’t able to figure out how to get the multi-module build to work within the context of the plugin SDK, but I admittedly did not spend much time on it. I vaguely recall that one of the Adaptavist folks (maybe @jmort ?) published an example many years ago of doing this?

1 Like

Hi All,

@MatthiasClasen

Thanks for your response, but I think this does not help us at all. We are required to released apps which are compatible within the supported range of Atlassian, which means our apps have to be released with a compatibility range between Confluence 7.10 (EOL date: 15 Dec 2022) and including Confluence 8. As of right now, it wouldn’t be possible to release a single version of an app to cover this range. For example, if we now decide to use the new ContentSearch constructor introduced in 7.20, our apps cannot be used in Confluence <7.20.
How should we deal with this situation?

Whilst we do our utmost to minimise the impact on our market partners, the simple answer is there are times where we need to make breaking changes to our APIs. In this case, we are working on upgrading Lucene to more modern versions, and they’ve made significant breaking changes in the library. Many of these changes we can’t reasonably write compatibility layers for because the changes are so dramatic.

As part of this work to upgrade Lucene, we also isolated it to ensure we have a public API that we can work to keep stable so that future upgrades to Lucene don’t break your plugins. I defer to you on the best approach for you and your team, but needing to maintain two seperate code branches is a possibility for a time. Equally, you can review @scott.dudley’s comment above.

@aragot

Awesome, thank you James:

  • The code excerpts will help us, thank you very much (I haven’t tested, but that was what I needed),
  • The code deprecation-and-removal in 2 versions, while Confluence LTS spans from 7.13 to 7.19 to 7.20 to 8.0, is incompatible with the necessities of Data Center, which is to provide enterprise-grade experience with 2-year span of compatibility for plugins, to enterprise-grade customers. We can use Reflection for once, but it slows down Confluence; We can use different APIs for Confluence 7.13, 7.19, 7.20 and 8.0 and JDK 8 and 11, but it will impact the customers experience if we have to publish apps which are only compatible with Confluence 8.0. Unless you can convince all customers to use Confluence 8.0, oh I would be glad :wink: (yay JDK 11 for all!).

More seriously, may I suggest to loop back with the team to check whether it is possible for @PublicAPI classes to enforce signature-level compatibility for 2 years. We know it’s a hard job, we love you when you’re successful at it, and it’s sometimes just a matter of programming adaptors between old signatures and newer ones.

Most of this follows the above to @MatthiasClasen.

We do our utmost to keep things static, but we need to be able to make changes, and the major version release is the line we’ve drawn in the sand to make these changes. I’m still trying to remove database attachment storage from the product, but am not doing so because we do our utmost not to make breaking changes outside of major versions. I have the code near ready to go, but missed the code freeze by days. The team already do an enormous amount of work to try and make things easier for our vendor partners.

I still understand we only deprecated the existing signature of ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit) in 7.20. Either way, the complete removal of this functionality from the underlying Lucene libraries means that we can’t support it in 8.0. It also means we can’t write simple adapters in these cases. We need the Lucene library upgrade, and the work we’ve done here should mean that as we upgrade Lucene to latest, you shouldn’t notice any additional impact.

@scott.dudley

@aragot It’s true that reflection slows down Confluence, but I think there is another way.

For starters, it’s not too difficult to create different internal components to be automatically injected depending on whether Confluence 7 or 8 is in use (see my code snippet on CONFSERVER-79630 for an example).

From there, you can use reflection to access the appropriate APIs, as you wrote. But using reflection is not actually forced by the runtime, because (with a few exceptions) it’s largely a compiler restriction. This is because your compiler can’t “see” the signature of the method you are trying to call, since it’s not compiling against that version of the dependency.

I believe it is possible to use a Maven multi-module project, with a submodule referencing a different version of Confluence, allowing you to write code within that module that can call the APIs directly.

I wasn’t able to figure out how to get the multi-module build to work within the context of the plugin SDK, but I admittedly did not spend much time on it. I vaguely recall that one of the Adaptavist folks (maybe @jmort ?) published an example many years ago of doing this?

Thanks for sharing. Hopefully that’ll help.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

1 Like

Thanks for this detailed response. There is one thing that would have helped us in enabling the Confluence 8 compatibility: until Confluence 8, we have been able to use Atlassian’s implementation of SearchFilters easily via SearchFilter.getInstance(). However, when switching to the SearchQuery implementation for the ContentSearch, it is now the case that the factories behind it are marked as @Internal, meaning we are currently cloning code from the Confluence source code, which must always be kept up to date.

“You should be able to find equivalent SearchQuery classes in Confluence 8.0.”

I tried using SpacePermissionQueryFactory instead of SpacePermissionsSearchFilter, but:
“NoSuchBeanDefinitionException: No qualifying bean of type ‘com.atlassian.confluence.search.v2.SpacePermissionQueryFactory’ available”.

How do I get a SpacePermissionQuery to use?

The class SpacePermissionQuery itself is @Internal, as are SpacePermissionFilteredQueryFactory and SpacePermissionQueryManager.

Ah, my bad: I was just missing a component import for the SpacePermissionQueryFactory

1 Like

Hi @MatthiasClasen,

This was by design so we can update the underlying libraries without breaking things you rely upon.

We have a long way to go in upgrading Lucene, so this was a necessity.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

Hi @aragot

Can you please list the entire code. I’ve tried

BooleanQuery.andQuery(query, new SiteSearchPermissionsFilteredQueryFactory().create())

It compiles but I’m getting
java.lang.NoClassDefFoundError: com/atlassian/confluence/impl/search/v2/SiteSearchPermissionsFilteredQueryFactory
at runtime

Thanks

1 Like

Hi @ashraf.teleb85 ,
You can inject a Spring bean with the name siteSearchPermissionsQueryFactory and call the create() method. SiteSearchPermissionsFilteredQueryFactory is an internal class and is not available to plugins.

2 Likes

We are also struggling with the ContentSearch problem in multiple apps. We had already discovered this with the several milestone releases, but somehow were still hopeful there would be some kind of backwards compatibility provided in the end. But now there is not.

I don’t quite understand the strategy of making the API deprecated only in 7.20, and then to fully remove it in 8.0. This doesn’t leave any room for backwards compatibility. We are now also considering implementing a reflection hack just for the ContentSearch constructor. I don’t think that this was supposed to be incentivized by the removal of this API component with such a small deprecation frame.

1 Like

Hi @fgrund,
we have had similar problems with our apps and decided to write our own library that itself uses Confluence in data version 7.20.2 and thus has both implementations (Confluence 7.x and 8.x). In the library we then make the U which Confluence version is running or which API is available and then use the API available in each case. I can strongly recommend to look into the source code of Confluence 7.20.2 as a template of changes that need to be made in your apps.
We are planning to make this library open source asap.

5 Likes

How do you use ContentService to retrieve a draft page. As ContentEntityManagerInternal is no longer available?

Below code is not working as expected

contentService.find()
    .withAnyStatus()
    .withId(ContentId.of(contentEntityId))
    .fetchOrNull();
1 Like

Hey @MatthiasClasen , please keep us posted on the progress of this if there is any. Some of us would be more than happy to contribute to the effort.

1 Like

Hy @MatthiasClasen, are there any concrete plans if and when you want to publish your library open source?

1 Like

Jumping on the train here, since we are also running into the problem with the ContentSearch and specifically the permission filter.

This is really poorly communicated and it is certainly not helping that the one available code sample still shows the old (removed) API without even mentioning that this does not work in 8.x upwards.

That said - we also have to create an app that works both on 7.x pre 20 and 8.x onward.

While we could use reflection to call the correct ContentSearch constructor, I can’t figure out how to create the correct permission filter dynamically, since the only way to get an instance of the SiteSearchPermissionsQueryFactory seems to be via a component import, and TBH I have no idea to do THIS dynamically (right now the code that actually executes the query isn’t even amenable to component injection, but I guess we could refactor that part).

Any ideas/solutions? We are certainly not going to recomponentize our whole app just to work around this mess; we’d rather just forego the permissions check completely (sorry guys).

Atlassian staff: If you break an API in that way, there NEEDS to be a clear migration guide and at the very least some samples that do work in both versions. This is really disappointing for a multi million dollar company.

Quick follow-up - would it be allowed to create to different versions of the same app that target 7.x and 8.x separately? Or would this mess up marketplace logic and/or not be permissible under some maintenance rules for confluence DC?