Preparing for Confluence 9.0 - EAP out now

Kusal · May 8, 2024, 1:08am

Has there been any consideration to automatically allowlisting all public methods defined in an app’s <velocity-context-item> modules?

Hi @scott.dudley

This seems sensible - let me get back to you on this.

If it’s not possible to allowlist automatically, does Atlassian already have any tool that reads a .class and outputs the method signatures in a compatible format?

Unfortunately not as we do not want to encourage the blind allowlisting of every method in a class. Currently, the logging will provide the methods needing allowlisting in a compatible format.

By enabling the system property, then running some acceptance tests and otherwise using your plugin, all necessary methods should be logged in a compatible format. You can then filter and dedupe your log lines to retrieve a list of currently invoked methods.

Now that the velocity method allowlist is enabled is it time for Confluence’s base classes like ActionSupport to annotate their accessor methods (e.g., getActionErrors, etc.)? If we use those inherited methods in our velocity templates should we add them to our velocity-allowlist?

Hi @turehoefner_appfire

All getters on Action classes should be allowlisted automatically (including #getActionErrors()). If you find there are methods defined in Confluence that should be allowlisted and are not currently so, please feel free to call them out and I will have them evaluated for addition.

I feel like I probably shouldn’t put inherited methods like that into my velocity-allowlist because I’d expect the base class to annotate them as velocity property accessors, but I don’t see annotations on them (@ParameterSafe or @StrutsParameter).

The OGNL allowlist (and its annotations) are entirely independent to the newly introduced Velocity allowlist. The OGNL allowlist primarily concerns request parameter injection and has no impact on which methods can be invoked from Velocity templates.

The Velocity allowlist does not currently have an accompanying annotation, and allowlisting must be done through the velocity-allowlist descriptor.

I tracked this down and I found that most Velocity logging is now turned off at the system level.

Thank you @scott.dudley for calling this out. This was indeed an oversight and will be rectified in the next EAP.

AndreasEbert · May 8, 2024, 11:28am

@Kusal, fyi with the current EAP version the page $baseUrl/plugins/servlet/upm/osgi is broken, i.e. the list of apps is not shown. That is a helpful page for inspecting OSGi details. Would be helpful if that page would work again.

adam.labus · May 8, 2024, 11:37am

Hi @NikhilJain

any news regarding missing artifacts?

Cheers
Adam

Kusal · May 8, 2024, 12:01pm

fyi with the current EAP version the page $baseUrl/plugins/servlet/upm/osgi is broken

Hi @AndreasEbert

This has been rectified in the latest milestone and will also be available in next week’s EAP.

any news regarding missing artifacts?

@adam.labus This has been addressed above

scott.dudley · May 8, 2024, 1:21pm

HI @Kusal - thanks for the upcoming fixes to the logging and for consideration of the auto-allowlisting of context items.

Although searching through the logs after running acceptance tests is smart, I still can’t guarantee that I have 100% code coverage of all Velocity templates with that set of tests, so to be safe, I need to review all of the impacted modules manually anyway.

I think the use case here is not to allowlist everything, but at least to correctly generate a matching list of methods and parameter types that we can choose from, and then optionally insert into the allowlist if warranted.

For example, when going through a velocity-context-item module, I already had to work through two dozen methods that needed to be available, and fish out the fully-qualified types for the multiple parameters for each. This is labor intensive and prone to error. (Even if copy and pasting, one needs to fetch the class name from one place, then copy and paste the package name separately from the import, also make sure that you have exactly one space between types, and so on.) Then take that effort and multiply by a thousand plugins.

I am not sure if the existing allowlist framework already does this, but it would also be fantastic if it could throw an error in the logs if one specifies a method name that cannot be resolved.

In the end, something like this would be really helpful:

# java -jar print-parameter-list.jar target/classes/com/mycompany/MyVelocityContextItem.class

com.mycompany.velocity#sayHello(java.lang.String)
com.mycompany.velocity#doHello(com.atlassian.confluence.core.ContentEntityObject java.lang.Long)
...

For reference, I see that the parameter type string is being constructed as follows:

   String toMethodStr(Method method) {
        String methodName = method.getName();
        return methodName + "(" + Arrays.stream(method.getParameterTypes()).map(Class::getName).collect(Collectors.joining(" ")) + ")";
    }

Kusal · May 8, 2024, 1:41pm

I am not sure if the existing allowlist framework already does this, but it would also be fantastic if it could throw an error in the logs if one specifies a method name that cannot be resolved.

It will only throw errors on unresolvable classes, unfortunately not method syntax as of yet.

In the end, something like this would be really helpful:

This Java function should achieve what you’re after, it can be called from a unit test:

public static String toAllowlistEntries(Class<?> clazz) {
    return Arrays.stream(clazz.getMethods())
            .filter(method -> !Modifier.isAbstract(method.getModifiers()))
            .map(method -> method.getDeclaringClass().getName() + "#" +
                    method.getName() + "(" + Arrays.stream(method.getParameterTypes()).map(Class::getName).collect(joining(" ")) + ")")
            .map(str -> "<method>" + str + "</method>")
            .collect(joining("\n"));
}

AndreasEbert · May 8, 2024, 1:44pm

Thanks.

Another problem:
UPDATE: Wait, there’s something weird with dependency resolution going on. Might be just on my end. Looking into it…
UPDATE 2: Apparently org.apache.velocity:velocity:pom:1.6.4-atlassian-34-m07 cannot be found. That might abort the dependency-resolution process, so that subsequent deps are not even tried to be found. I see in https://packages.atlassian.com/artifactory/maven-external/org/apache/velocity/velocity/ that indeed 1.6.4-atlassian-34-m07 is not there, but 1.6.4-atlassian-34-m08 is. Is that intentional?
(Starting with 9.0.0-m72 class com.atlassian.confluence.util.GeneralUtil is not available anymore (was still available in 9.0.0-m57). Is that intentional? That class was not deprecated, and I don’t see it being listed in Deprecated code paths removed in 9.0 | Confluence Data Center 8.9 | Atlassian Documentation, so I’m confused by it suddenly being gone. Can you look into that, please?)

DirkBrutscher · May 12, 2024, 7:04am

I also see the problem of maintaining backwards compatibility. I get the first requests to update my app because it is no longer compatible with Confluence 7.x (according to the Marketplace), then they have to manually select an older version in the historical versions. This is bad, 99% of all users will not even make an upgrade request and simply assume that the app is not available for their older Confluence version. Isn’t there a better way?

One way may be to give us an option e.g. in the “atlassian-plugin-marketing.xml” to select an overall compatibility, which leads to a meaningful compatibility info in the header of the marketplace (c.f. screenshot: I have version going back to 6.14.0 compatibility not only 8.5.4). Then, if a user with an older version of Confluence starts an download the information in the “atlassian-plugin-marketing.xml” determins which “historical” version should be delivered to the user.

MichaelAndreacchio · May 13, 2024, 10:55pm

Hi Developer Community,

I’m happy to announce that we’ve completed approximately 95% of the breaking changes targeting Confluence 9.0 and have shipped these in our most recent EAP available for download. This includes the full adoption of Platform 7, RESTv2, compatibility with Java 17 at minimum, code removals, refactors, and many more changes.

We encourage our marketplace partners to take a look at this early preview and provide any feedback you may have either in this thread or via our feedback channel. As we draw nearer to the release of Confluence 9, your early feedback is more valuable now than ever to ensure we can address it before we complete our remaining milestones. We’re hugely thankful for the feedback in this thread, which has guided us towards making adjustments in our release, so hearing about the progress in your compatibility journey will benefit the wider ecosystem.

Thanks - looking forward to your feedback!

Kusal · May 14, 2024, 7:16am

1.6.4-atlassian-34-m07 is not there, but 1.6.4-atlassian-34-m08 is. Is that intentional?

This has been addressed above - it is resolved with the latest EAP

clouless · May 14, 2024, 7:31am

I am getting this error with m81:

java.lang.IllegalStateException: Failed to transform class with name com.atlassian.confluence.renderer.radeox.macros.MacroUtils. Reason: cannot find org.radeox.macro.parameter.MacroParameter
Caused by: javassist.CannotCompileException: cannot find org.radeox.macro.parameter.MacroParameter
Caused by: javassist.NotFoundException: org.radeox.macro.parameter.MacroParameter

I am mocking import com.atlassian.confluence.renderer.radeox.macros.MacroUtils;
in my tests and that has worked before for years. Have you made some more packages private?

Please make the classes available, thanks.

Workaround for now - with this my build succeeds

Create the three missing interfaces in src/test/java

src/test/java/org/radeox/macro/parameter/MacroParameter.java

package org.radeox.macro.parameter;

import org.radeox.api.engine.context.RenderContext;

public interface MacroParameter {
    void setParams(String var1);
    String getContent();
    void setContent(String var1);
    int getLength();
    String get(String var1, int var2);
    String get(String var1);
    String get(int var1);
    void setStart(int var1);
    void setEnd(int var1);
    int getStart();
    int getEnd();
    void setContentStart(int var1);
    void setContentEnd(int var1);
    int getContentStart();
    int getContentEnd();
    RenderContext getContext();
}

src/test/java/org/radeox/api/engine/RenderEngine.java

package org.radeox.api.engine;

import java.io.IOException;
import java.io.Writer;
import org.radeox.api.engine.context.RenderContext;

public interface RenderEngine {
    String getName();
    String render(String var1, RenderContext var2);
    void render(Writer var1, String var2, RenderContext var3) throws IOException;
}

src/test/java/org/radeox/api/engine/context/RenderContext.java

package org.radeox.api.engine.context;

import java.util.Map;
import org.radeox.api.engine.RenderEngine;

public interface RenderContext {
    RenderEngine getRenderEngine();
    void setRenderEngine(RenderEngine var1);
    Object get(String var1);
    void set(String var1, Object var2);
    Map getParameters();
    void setParameters(Map var1);
    void setCacheable(boolean var1);
    void commitCache();
    boolean isCacheable();
}

KhaNguyen2 · May 14, 2024, 9:15am

I need to test my server to cloud migration with Confluence 9, but I run into this error when using CCMA:

URL: http://mac-studio.local:1990/confluence/admin/migration.action
CCMA 3.9.10
Confluence 9.0.0-m81

BhavinDoshi · May 14, 2024, 9:58am

@KhaNguyen2 CCMA currently does not support Confluence 9, and we are in the process of making it compatible with Confluence 9. I hope this helps!

KhaNguyen2 · May 14, 2024, 10:11am

got it. Do you have an ETA on when it will be available?

GorkaGalloBustinza · May 14, 2024, 2:45pm

Hi,

Do you have any estimate of how many milestones are left and when the beta will be available?

Gorka.

andreas1 · May 14, 2024, 7:09pm

Dear all,

m81 (Linux bin) does not work at all on our test system. Lots of UI pieces are broken:

The start screen only shows the bar on the top and "Error rendering WebPanel (com.atlassian.confluence.plugins.confluence-dashboard:dashboard-server-side-soy/DashboardTemplates.App.serverSideLayout): Could not find custom mapper jackson2soy for class class com.atlassian.confluence.api.model.web.WebItemView "
Adding pages is not possible - the UI shows no page title and you can’t type anything.
Editing pages is not possible - only the toolbar and a broken footer are shown.
…

Any ideas?

Thank you

Andreas

ashraf.teleb85 · May 14, 2024, 8:14pm

Hi,

Is this endpoint removed

http://localhost:1990/confluence/pages/editpage.action?pageId=884737

I’m getting this error:

Your request could not be processed because a required security token was not present in the request. You may need to re-submit the form or reload the page.

We build this URL and use window.open in JS to open a page in edit mode

Thanks

rlander · May 14, 2024, 8:27pm

You need to pass an XSRF token now for this endpoint: Enable XSRF protection in your app

ashraf.teleb85 · May 14, 2024, 10:18pm

Thanks for your reply.
How can I pass the XSRF token? This is a Confluence endpoint (the page editor) and I’m opening it from another page using JS (via window.open)

rlander · May 14, 2024, 10:25pm

See: Enable XSRF protection in your app

You’ll need to supply the token via atl_token query parameter when building the URL.