Broken Java API in Confluence 8 EAP: ContentSearch, getById(), etc

Confluence Development Confluence Data Center
1

At this time, it is impossible (or extremely, extremely badly documented) for apps to retrieve Pages, Content, Spaces by ID or perform a CQL search in Confluence 8.0 EAP with the official APIs.

  • PageManager.getById(), ContentEntityManager.getById() have been deprecated since Confluence 7.5,
  • PageManager.getPage(String spaceKey, String pageTitle) has been deprecated since Confluence 7.3,
  • Both redirect us to com.atlassian.confluence.api.service.content.ContentService.find(Expansion…) which is unusable/not documented (There is no valid Expansion argument which searches for content by ID or name, I’ve tried to guess which subclass of Expansion can do it, but there are a lot of classes without Javadocs),
  • The example provided in the JavaDoc of ContentService (contentService.find().withSpace("DEV").withType(ContentType.BLOG_POST).fetchMany(new SimplePageRequest(0,50)) is incorrect since it doesn’t compile - It expects a Space object as a parameter, not a space key, so should I always call spaceService.find().withKeys("DEV","PROD").fetchMany(new SimplePageRequest(0, 10) first, as shown in the Javadoc of SpaceService? It seems unnecessarily convoluted and basically inefficient, is the code of Confluence itself doing that? (There is no instance where the source of Confluence does that).
  • Classes such as SiteSearchPermissionsSearchFilter have been removed, so we can’t apply user permissions when searching (which is a security breach awaiting to happen),
  • Last but not least, they change method signatures without notice!
    ** In 7.19: ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit)
    ** In 7.20: ContentSearch(@NonNull SearchQuery query, SearchSort sort, int startOffset, int limit),
    ** The ‘searchFilter’ argument was simply removed, no deprecation, it’s a public API class, so we can’t write code for 7.20 which works in 7.19, and we can’t write code for 7.19 that works in 7.20.

It would be very nice, or very helpful for the continued compatibility of apps, if an Atlassian engineer could write us examples for the following usecases, which are essential for plugins to integrate with Confluence:

  • How to perform a CQL search,
  • How to simply get a page by ID, a page version by ID and a space by key, and hopefully a page by title,
  • How to modify the body of a page and save it as a new version (since getById() is deprecated, I suspect it is impossible),
  • All of this using using genuine ContentEntityObject.java (Page, Blog, Comment, Space.java) and not a shell object like those returned by the search. If necessary, show us how to return shell objects into complete objects.

Thank you very much,
Adrien Ragot

12 Likes
2

Concerning the SiteSearchPermissionsSearchFilter, I’ve succeeded to find the factory SiteSearchPermissionsQueryFactory, so the following code works:

    public List<Map<String, String>> findPages(String query) {
        try {
            SearchQuery searchV2Query = StringUtils.isNotBlank(query)
                ? new TextFieldQuery("title", query + "*", BooleanOperator.AND)
                : AllQuery.getInstance();
            searchV2Query = andQuery(searchV2Query,
                new ContentTypeQuery(Lists.newArrayList(
                    ContentTypeEnum.PAGE,
                    ContentTypeEnum.BLOG,
                    ContentTypeEnum.CUSTOM
                )));
            searchV2Query = andQuery(searchV2Query, siteSearchPermissionsQueryFactory.create());
            SearchSort sort = new RelevanceSort();
            ContentSearch search = new ContentSearch(searchV2Query, sort, 0, 20);
            SearchResults results = searchManager.search(search);

            List<Map<String, String>> result = Lists.newArrayList();
            for (SearchResult item : results.getAll()) {
                final Handle handle = item.getHandle();
                if (handle instanceof HibernateHandle) {
                    if (isCEO((HibernateHandle) handle)) {
                        long id = ((HibernateHandle) handle).getId();
                        String title = item.getDisplayTitle();
                        String type = item.getType();
                        String space = item.getSpaceKey();
                        result.add(mapOf(
                            "type", type,
                            "space", space,
                            "id", Objects.toString(id),
                            "label", title
                        ));
                    }
                }
            }
            return result;
        } catch (InvalidSearchException e) {
            throw new RuntimeException(e);
        }
    }
3 Likes
3

Confluence 7.20 seems to be the only version in which ContentSearch has a constructor with and without SearchFilter. Older versions only have the constructor with the SearchFilter, Confluence 8 RC1 only has the constructor without it.

I wonder how it is possible to build an app that is compatible with Confluence 7.10 (the oldest Confluence version that has not reached end of life) and Confluence 8…

3 Likes
4

I tried copying ContentSearch (and its superclass AbstractSearch) from Confluence 8, but AbstractSearch implements the interface ISearch that has a Method SearchFilter getSearchFilter() in Confluence 7.10-7.20, but not in Confluence 8.

1 Like
5

There is also another method that has been replaced without workarounds. The method ScheduledJobManager.disable accepts a ScheduledJobKey in Confluence version 7.10 and is replaced in Confluence 8.0 with JobId.

1 Like
6

Very nice. Don’t forget to like or create your own post, so these problems bubble up and community managers are incentivised to write a tutorial.

3 Likes
8

Hi All,

Replying to the above.

From the first block

Classes such as SiteSearchPermissionsSearchFilter have been removed, so we can’t apply user permissions when searching (which is a security breach awaiting to happen)

All the classes the implement the SearchFilter interface are still available in 7.20. They will only be removed in Confluence 8.0 as all SearchFilter will be replaced with SearchQuery. You should be able to find equivalent SearchQuery classes in Confluence 8.0. In the particular case of the SiteSearchPermissionsSearchFilter you can use the SiteSearchPermissionsQuery (available since 7.20) instead. Please refer to Searching using the v2 search API for examples

Last but not least, they change method signatures without notice!

** In 7.19: ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit)

** In 7.20: ContentSearch(@NonNull SearchQuery query, SearchSort sort, int startOffset, int limit),

** The ‘searchFilter’ argument was simply removed, no deprecation, it’s a public API class, so we can’t write code for 7.20 which works in 7.19, and we can’t write code for 7.19 that works in 7.20.

The signature ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit) was only deprecated since 7.20 and removed from 8.0. The interface SearchFilter and all of the classes that implement this interface was deprecated in 7.20 and removed from 8.0 in preparation for the future upgrade of Lucene library. This is required because as of Lucene 6 all of the Filter classes have been removed.

You can find additional information on our deprecated code paths removed in 8.0 page.

To your questions in the second block

How to perform a CQL search

CQL search can be done via CQLSearchService . You can find examples in the JavaDoc.

  • How to simply get a page by ID, a page version by ID and a space by key, and hopefully a page by title
  • How to modify the body of a page and save it as a new version (since getById() is deprecated, I suspect it is impossible)

Here are some example of the usage of ContentService:

Finding a page by page ID

contentService.find()
                .withStatus(ContentStatus.CURRENT)
                .withId(contentId)
                .fetch()
                .orElseThrow(notFound("No content found with id : " + contentId));

Finding a page by page version by ID

contentService.find(new Expansion("version"))
				.withIdAndVersion(page.getContentId(), page.getVersion())
				.fetchOrNull();

Note: in this example, we need to load “version” attribute in order to use withIdAndVersion
Finding a Space with Space key

spaceService.find()
                    .withKeys(spaceKey)
                    .fetch()

Finding a page by page title and Space

contentService.find(Expansions.of(VERSION, HISTORY, STATUS, SPACE).toArray()) // Load extra attributes: VERSION, HISTORY, STATUS, SPACE
                .withSpace(Space.build().key("SPACE_KEY"))
                .withTitle("Some page title")
                .withType(ContentType.PAGE) // This indicates which content type to be loaded: PAGE, BLOG_POST...
                .withStatus(CURRENT);

Update a Content’s body by ID

Content foundContent = contentService.find()
                .withStatus(ContentStatus.CURRENT)
                .withId(contentId)
                .fetch()
                .orElseThrow(notFound("No content found with id : " + contentId));
Content toBeUpdatedContent = Content.builder(foundContent)
	.body("new body", ContentRepresentation.STORAGE);
contentService.update(foundContent);

All of this using using genuine ContentEntityObject.java (Page, Blog, Comment, Space.java) and not a shell object like those returned by the search. If necessary, show us how to return shell objects into complete objects.

We’re attempting to remove dependancies on Hibernate objects. The ContentEntityObject is a Hibernate object. The service APIs expose similar capabilities, so we’d recommend using that API instead of depending on the ContentEntityObject previously used.

Hopefully this meets your exacting requirements.

Dutifully,
James Ponting
Engineering Manager - Confluence Data Center

5 Likes
9

Thanks for your response, but I think this does not help us at all. We are required to released apps which are compatible within the supported range of Atlassian, which means our apps have to be released with a compatibility range between Confluence 7.10 (EOL date: 15 Dec 2022) and including Confluence 8. As of right now, it wouldn’t be possible to release a single version of an app to cover this range. For example, if we now decide to use the new ContentSearch constructor introduced in 7.20, our apps cannot be used in Confluence <7.20.
How should we deal with this situation?

Best regards

10

Awesome, thank you James:

  • The code excerpts will help us, thank you very much (I haven’t tested, but that was what I needed),
  • The code deprecation-and-removal in 2 versions, while Confluence LTS spans from 7.13 to 7.19 to 7.20 to 8.0, is incompatible with the necessities of Data Center, which is to provide enterprise-grade experience with 2-year span of compatibility for plugins, to enterprise-grade customers. We can use Reflection for once, but it slows down Confluence; We can use different APIs for Confluence 7.13, 7.19, 7.20 and 8.0 and JDK 8 and 11, but it will impact the customers experience if we have to publish apps which are only compatible with Confluence 8.0. Unless you can convince all customers to use Confluence 8.0, oh I would be glad :wink: (yay JDK 11 for all!).

More seriously, may I suggest to loop back with the team to check whether it is possible for @PublicAPI classes to enforce signature-level compatibility for 2 years. We know it’s a hard job, we love you when you’re successful at it, and it’s sometimes just a matter of programming adaptors between old signatures and newer ones.

Best regards,
Adrien Ragot

11

@aragot It’s true that reflection slows down Confluence, but I think there is another way.

For starters, it’s not too difficult to create different internal components to be automatically injected depending on whether Confluence 7 or 8 is in use (see my code snippet on CONFSERVER-79630 for an example).

From there, you can use reflection to access the appropriate APIs, as you wrote. But using reflection is not actually forced by the runtime, because (with a few exceptions) it’s largely a compiler restriction. This is because your compiler can’t “see” the signature of the method you are trying to call, since it’s not compiling against that version of the dependency.

I believe it is possible to use a Maven multi-module project, with a submodule referencing a different version of Confluence, allowing you to write code within that module that can call the APIs directly.

I wasn’t able to figure out how to get the multi-module build to work within the context of the plugin SDK, but I admittedly did not spend much time on it. I vaguely recall that one of the Adaptavist folks (maybe @jmort ?) published an example many years ago of doing this?

1 Like
12

Hi All,

@MatthiasClasen

Thanks for your response, but I think this does not help us at all. We are required to released apps which are compatible within the supported range of Atlassian, which means our apps have to be released with a compatibility range between Confluence 7.10 (EOL date: 15 Dec 2022) and including Confluence 8. As of right now, it wouldn’t be possible to release a single version of an app to cover this range. For example, if we now decide to use the new ContentSearch constructor introduced in 7.20, our apps cannot be used in Confluence <7.20.
How should we deal with this situation?

Whilst we do our utmost to minimise the impact on our market partners, the simple answer is there are times where we need to make breaking changes to our APIs. In this case, we are working on upgrading Lucene to more modern versions, and they’ve made significant breaking changes in the library. Many of these changes we can’t reasonably write compatibility layers for because the changes are so dramatic.

As part of this work to upgrade Lucene, we also isolated it to ensure we have a public API that we can work to keep stable so that future upgrades to Lucene don’t break your plugins. I defer to you on the best approach for you and your team, but needing to maintain two seperate code branches is a possibility for a time. Equally, you can review @scott.dudley’s comment above.

@aragot

Awesome, thank you James:

  • The code excerpts will help us, thank you very much (I haven’t tested, but that was what I needed),
  • The code deprecation-and-removal in 2 versions, while Confluence LTS spans from 7.13 to 7.19 to 7.20 to 8.0, is incompatible with the necessities of Data Center, which is to provide enterprise-grade experience with 2-year span of compatibility for plugins, to enterprise-grade customers. We can use Reflection for once, but it slows down Confluence; We can use different APIs for Confluence 7.13, 7.19, 7.20 and 8.0 and JDK 8 and 11, but it will impact the customers experience if we have to publish apps which are only compatible with Confluence 8.0. Unless you can convince all customers to use Confluence 8.0, oh I would be glad :wink: (yay JDK 11 for all!).

More seriously, may I suggest to loop back with the team to check whether it is possible for @PublicAPI classes to enforce signature-level compatibility for 2 years. We know it’s a hard job, we love you when you’re successful at it, and it’s sometimes just a matter of programming adaptors between old signatures and newer ones.

Most of this follows the above to @MatthiasClasen.

We do our utmost to keep things static, but we need to be able to make changes, and the major version release is the line we’ve drawn in the sand to make these changes. I’m still trying to remove database attachment storage from the product, but am not doing so because we do our utmost not to make breaking changes outside of major versions. I have the code near ready to go, but missed the code freeze by days. The team already do an enormous amount of work to try and make things easier for our vendor partners.

I still understand we only deprecated the existing signature of ContentSearch(SearchQuery query, SearchSort sort, SearchFilter searchFilter, int startOffset, int limit) in 7.20. Either way, the complete removal of this functionality from the underlying Lucene libraries means that we can’t support it in 8.0. It also means we can’t write simple adapters in these cases. We need the Lucene library upgrade, and the work we’ve done here should mean that as we upgrade Lucene to latest, you shouldn’t notice any additional impact.

@scott.dudley

@aragot It’s true that reflection slows down Confluence, but I think there is another way.

For starters, it’s not too difficult to create different internal components to be automatically injected depending on whether Confluence 7 or 8 is in use (see my code snippet on CONFSERVER-79630 for an example).

From there, you can use reflection to access the appropriate APIs, as you wrote. But using reflection is not actually forced by the runtime, because (with a few exceptions) it’s largely a compiler restriction. This is because your compiler can’t “see” the signature of the method you are trying to call, since it’s not compiling against that version of the dependency.

I believe it is possible to use a Maven multi-module project, with a submodule referencing a different version of Confluence, allowing you to write code within that module that can call the APIs directly.

I wasn’t able to figure out how to get the multi-module build to work within the context of the plugin SDK, but I admittedly did not spend much time on it. I vaguely recall that one of the Adaptavist folks (maybe @jmort ?) published an example many years ago of doing this?

Thanks for sharing. Hopefully that’ll help.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

1 Like
13

Thanks for this detailed response. There is one thing that would have helped us in enabling the Confluence 8 compatibility: until Confluence 8, we have been able to use Atlassian’s implementation of SearchFilters easily via SearchFilter.getInstance(). However, when switching to the SearchQuery implementation for the ContentSearch, it is now the case that the factories behind it are marked as @Internal, meaning we are currently cloning code from the Confluence source code, which must always be kept up to date.

14

“You should be able to find equivalent SearchQuery classes in Confluence 8.0.”

I tried using SpacePermissionQueryFactory instead of SpacePermissionsSearchFilter, but:
“NoSuchBeanDefinitionException: No qualifying bean of type ‘com.atlassian.confluence.search.v2.SpacePermissionQueryFactory’ available”.

How do I get a SpacePermissionQuery to use?

The class SpacePermissionQuery itself is @Internal, as are SpacePermissionFilteredQueryFactory and SpacePermissionQueryManager.

15

Ah, my bad: I was just missing a component import for the SpacePermissionQueryFactory…

1 Like
16

Hi @MatthiasClasen,

This was by design so we can update the underlying libraries without breaking things you rely upon.

We have a long way to go in upgrading Lucene, so this was a necessity.

Thanks,
James Ponting
Engineering Manager - Confluence Data Center

17

Hi @aragot

Can you please list the entire code. I’ve tried

BooleanQuery.andQuery(query, new SiteSearchPermissionsFilteredQueryFactory().create())

It compiles but I’m getting
java.lang.NoClassDefFoundError: com/atlassian/confluence/impl/search/v2/SiteSearchPermissionsFilteredQueryFactory
at runtime

Thanks

1 Like
18

Hi @ashraf.teleb85 ,
You can inject a Spring bean with the name siteSearchPermissionsQueryFactory and call the create() method. SiteSearchPermissionsFilteredQueryFactory is an internal class and is not available to plugins.

2 Likes
19

We are also struggling with the ContentSearch problem in multiple apps. We had already discovered this with the several milestone releases, but somehow were still hopeful there would be some kind of backwards compatibility provided in the end. But now there is not.

I don’t quite understand the strategy of making the API deprecated only in 7.20, and then to fully remove it in 8.0. This doesn’t leave any room for backwards compatibility. We are now also considering implementing a reflection hack just for the ContentSearch constructor. I don’t think that this was supposed to be incentivized by the removal of this API component with such a small deprecation frame.

1 Like
20

Hi @fgrund,
we have had similar problems with our apps and decided to write our own library that itself uses Confluence in data version 7.20.2 and thus has both implementations (Confluence 7.x and 8.x). In the library we then make the U which Confluence version is running or which API is available and then use the API available in each case. I can strongly recommend to look into the source code of Confluence 7.20.2 as a template of changes that need to be made in your apps.
We are planning to make this library open source asap.

5 Likes
21

How do you use ContentService to retrieve a draft page. As ContentEntityManagerInternal is no longer available?

Below code is not working as expected

contentService.find()
    .withAnyStatus()
    .withId(ContentId.of(contentEntityId))
    .fetchOrNull();
1 Like