Problems with gadgets when run atlas-run using quickreload

david.gonzalez · January 30, 2020, 11:14am

Hi all,

We have a plugin with gadgets and when we start Jira with atlas-run or atlas-debug using quickreload, all our gadgets even the atlassian’s gadgets don’t load. But when we install the plugin into another instance of Jira server, the plugin load all gadgets without problems.

In the atlassian-jira.log we observed this trace when we access to the dashboard page:

2020-01-30 11:59:35,902 http-nio-8080-exec-9 WARN admin 719x2149x5 1ropmpa 0:0:0:0:0:0:0:1 /secure/Dashboard.jspa [c.a.v.htmlsafe.introspection.AnnotationBoxingUberspect] Velocity template accessing deprecated method com.atlassian.jira.plugin.webfragment.model.JiraHelper#getProjectObject - getEncodedBodyFromContent[line 1, column 10]
2020-01-30 11:59:35,902 http-nio-8080-exec-9 WARN admin 719x2149x5 1ropmpa 0:0:0:0:0:0:0:1 /secure/Dashboard.jspa [c.a.v.htmlsafe.introspection.AnnotationBoxingUberspect] Velocity template accessing deprecated method com.atlassian.jira.plugin.webfragment.model.JiraHelper#getProjectObject - getEncodedBodyFromContent[line 1, column 40]
2020-01-30 11:59:44,245 indexing-stats-0 INFO      [c.a.j.issue.index.IndexingStatsManager] [indexing-stats] Top 10 addIndex total (noFieldIndexers: 36, sum: 230ms): [{field: timespent, addIndex: {sum/allSum:49,8%, sum:114ms, avg:114,6ms, max:114ms, count:1}}, {field: subtasks, addIndex: {sum/allSum:17,6%, sum:40ms, avg:40,5ms, max:40ms, count:1}}, {field: watcher, addIndex: {sum/allSum:15,2%, sum:35ms, avg:35,1ms, max:35ms, count:1}}, {field: issuekey, addIndex: {sum/allSum:6,1%, sum:13ms, avg:13,9ms, max:13ms, count:1}}, {field: voter, addIndex: {sum/allSum:6,0%, sum:13ms, avg:13,9ms, max:13ms, count:1}}, {field: attachment, addIndex: {sum/allSum:3,7%, sum:8ms, avg:8,4ms, max:8ms, count:1}}, {field: votes, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: parent, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: issueid, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: progress, addIndex: {sum/allSum:0,2%, sum:0ms, avg:0,5ms, max:0ms, count:1}}]
2020-01-30 11:59:44,253 indexing-stats-0 INFO      [c.a.j.issue.index.IndexingStatsManager] [indexing-stats] Top 10 addIndex snapshot (noFieldIndexers: 36, sum: 230ms): [{field: timespent, addIndex: {sum/allSum:49,8%, sum:114ms, avg:114,6ms, max:114ms, count:1}}, {field: subtasks, addIndex: {sum/allSum:17,6%, sum:40ms, avg:40,5ms, max:40ms, count:1}}, {field: watcher, addIndex: {sum/allSum:15,2%, sum:35ms, avg:35,1ms, max:35ms, count:1}}, {field: issuekey, addIndex: {sum/allSum:6,1%, sum:13ms, avg:13,9ms, max:13ms, count:1}}, {field: voter, addIndex: {sum/allSum:6,0%, sum:13ms, avg:13,9ms, max:13ms, count:1}}, {field: attachment, addIndex: {sum/allSum:3,7%, sum:8ms, avg:8,4ms, max:8ms, count:1}}, {field: votes, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: parent, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: issueid, addIndex: {sum/allSum:0,3%, sum:0ms, avg:0,7ms, max:0ms, count:1}}, {field: progress, addIndex: {sum/allSum:0,2%, sum:0ms, avg:0,5ms, max:0ms, count:1}}]
2020-01-30 11:59:50,353 http-nio-8080-exec-9 WARN admin 719x3189x1 1ropmpa 0:0:0:0:0:0:0:1 /rest/config/1.0/directoryitems/local.json [c.a.g.d.internal.impl.DirectoryImpl] Could not retrieve directory entries from application-provided gadget specs: plugin gadget 'Key{pluginKey='com.deiser.jira.profields', location='com/deiser/jira/profields/gadget/onefield/profields_gadget_onefield.xml'}' failed validation
2020-01-30 11:59:50,367 http-nio-8080-exec-7 WARN admin 719x3190x2 1ropmpa 0:0:0:0:0:0:0:1 /rest/config/1.0/directoryitems/external.json [c.a.g.d.internal.impl.DirectoryImpl] Could not retrieve directory entries from application-provided gadget specs: plugin gadget 'Key{pluginKey='com.deiser.jira.profields', location='com/deiser/jira/profields/gadget/onefield/profields_gadget_onefield.xml'}' failed validation

Also we are getting some trace like this:

2020-01-30 12:01:25,052 http-nio-8080-exec-12 ERROR anonymous 721x3346x3 1s2echs 127.0.0.1 /rest/gadgets/1.0/g/feed [c.a.p.r.c.error.jersey.ThrowableExceptionMapper] Uncaught exception thrown by REST service: plugin gadget 'Key{pluginKey='com.deiser.jira.profields', location='com/deiser/jira/profields/gadget/onefield/profields_gadget_onefield.xml'}' failed validation
com.atlassian.gadgets.GadgetParsingException: plugin gadget 'Key{pluginKey='com.deiser.jira.profields', location='com/deiser/jira/profields/gadget/onefield/profields_gadget_onefield.xml'}' failed validation
	at com.atlassian.gadgets.publisher.internal.impl.PublishedGadgetSpecStore$PluginGadgetSpecStore.validate(PublishedGadgetSpecStore.java:341)
	at com.atlassian.gadgets.publisher.internal.impl.PublishedGadgetSpecStore$PluginGadgetSpecStore.lambda$put$0(PublishedGadgetSpecStore.java:298)
	at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:167)
	at com.atlassian.gadgets.publisher.internal.impl.PublishedGadgetSpecStore$PluginGadgetSpecStore.lambda$toSpecs$1(PublishedGadgetSpecStore.java:311)
	at com.google.common.collect.Iterators$6.transform(Iterators.java:785)
	at com.google.common.collect.TransformedIterator.next(TransformedIterator.java:47)
	at com.google.common.collect.Collections2$FilteredCollection.size(Collections2.java:220)
	at com.google.common.collect.Collections2$TransformedCollection.size(Collections2.java:307)
	at java.util.AbstractCollection.toArray(AbstractCollection.java:136)
	at java.util.ArrayList.addAll(ArrayList.java:581)
	at com.atlassian.gadgets.publisher.internal.impl.GadgetSpecSyndicationImpl.addGadgetSpecEntries(GadgetSpecSyndicationImpl.java:86)
	at com.atlassian.gadgets.publisher.internal.impl.GadgetSpecSyndicationImpl.getFeed(GadgetSpecSyndicationImpl.java:72)
	at com.atlassian.gadgets.publisher.internal.rest.GadgetSpecSyndicationResource.get(GadgetSpecSyndicationResource.java:29)
	... 3 filtered
	at java.lang.reflect.Method.invoke(Method.java:498)
	... 18 filtered
	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter$JerseyOsgiServletContainer.doFilter(RestDelegatingServletFilter.java:159)
	... 1 filtered
	at com.atlassian.plugins.rest.module.RestDelegatingServletFilter.doFilter(RestDelegatingServletFilter.java:69)
	... 36 filtered
	at com.atlassian.jira.plugin.mobile.web.filter.MobileAppRequestFilter.doFilter(MobileAppRequestFilter.java:37)
	... 4 filtered
	at com.atlassian.jira.plugin.mobile.login.MobileLoginSuccessFilter.doFilter(MobileLoginSuccessFilter.java:54)
	... 3 filtered
	at com.atlassian.diagnostics.internal.platform.monitor.http.HttpRequestMonitoringFilter.doFilter(HttpRequestMonitoringFilter.java:55)
	... 8 filtered
	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
	... 24 filtered
	at com.atlassian.labs.httpservice.resource.ResourceFilter.doFilter(ResourceFilter.java:59)
	... 36 filtered
	at com.atlassian.jira.security.JiraSecurityFilter.lambda$doFilter$0(JiraSecurityFilter.java:66)
	... 1 filtered
	at com.atlassian.jira.security.JiraSecurityFilter.doFilter(JiraSecurityFilter.java:64)
	... 16 filtered
	at com.atlassian.plugins.rest.module.servlet.RestSeraphFilter.doFilter(RestSeraphFilter.java:37)
	... 19 filtered
	at com.atlassian.jira.servermetrics.CorrelationIdPopulatorFilter.doFilter(CorrelationIdPopulatorFilter.java:30)
	... 10 filtered
	at com.atlassian.web.servlet.plugin.request.RedirectInterceptingFilter.doFilter(RedirectInterceptingFilter.java:21)
	... 4 filtered
	at com.atlassian.web.servlet.plugin.LocationCleanerFilter.doFilter(LocationCleanerFilter.java:36)
	... 26 filtered
	at com.atlassian.jira.servermetrics.MetricsCollectorFilter.doFilter(MetricsCollectorFilter.java:25)
	... 23 filtered
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)

Also we tried with the atlassian.dev.mode set to false, but we get the same result.

Has anyone seen these same problems and can help us with this problem?

Thank you very much.
Best regards.

aswan · February 6, 2020, 1:49am

Hi David,

My team maintains the Gadgets framework for Server products.

Have you tried either:

(preferably) setting a breakpoint inside GadgetSpecValidatorImpl#isValid, to see why it returns false, or
(less informative) setting the log level of com.atlassian.gadgets.publisher.internal.impl.GadgetSpecValidatorImpl to DEBUG, to see the underlying parsing exception?

david.gonzalez · February 6, 2020, 10:13am

Thank you @aswan

When we set the level of the GadgetSpecValidator, We discover in the logs traces like:

2020-02-06 10:18:13,182+0100 http-nio-8080-exec-3 DEBUG admin 618x13053x1 14vcmoo 0:0:0:0:0:0:0:1 /rest/config/1.0/directoryitems/local.json [c.a.g.p.internal.impl.GadgetSpecValidatorImpl] couldn't parse gadget spec
org.xml.sax.SAXParseException; lineNumber: 48; columnNumber: 241; The string "--" is not permitted within comments.
    at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
    at com.atlassian.gadgets.publisher.internal.impl.GadgetSpecValidatorImpl.isValid(GadgetSpecValidatorImpl.java:40)
    at com.atlassian.gadgets.publisher.internal.impl.PublishedGadgetSpecStore$PluginGadgetSpecStore.validate(PublishedGadgetSpecStore.java:340)
    at com.atlassian.gadgets.publisher.internal.impl.PublishedGadgetSpecStore$PluginGadgetSpecStore.lambda$put$0(PublishedGadgetSpecStore.java:298)
......

In the xml of gadgets we have some comments with when we remove this comments the gadgets load perfectly. Do you have some comments into the xml of gadgets? Do you need to make different things for use gadgets into development or production?

Thank you so much!
Regards

aswan · February 6, 2020, 10:41pm

I wrote a quick unit test to verify the validation of comments. It turns out that the isValid method considers the following XML to be valid:

<foo><!-- a comment -->bar</foo>

So it would seem that for the trivial case at least, comments are supported. Is there something about your XML comments that makes them invalid? For example do they contain the sequence --, like it says in your error message? Maybe check line 48, column 241 of your XML?

david.gonzalez · February 7, 2020, 10:43am

Our XMLs for gadgets are like:

<?xml version="1.0" encoding="UTF-8" ?>
<Module>
   <ModulePrefs ...>
     ...
   </ModulePrefs>
   <!-- isConfigured is used by gadget to determine if standard config screen should be shown, which we don't want to happen -->
   ...
   <Content type="html" view="profile, canvas,default,projectTab,profileTab,issueTab">
       <!-- #requireResource("com.test.jira.foo:bar") -->
      
       <![CDATA[ 
              ...
       ]]>
   </Content>
</Module>

These comments are in lines 25 and 36 and the line 48 is empty.

It seems strange that if I remove these comments into XML of the gadget, it works perfectly.

aswan · February 9, 2020, 10:30pm

Well, my test validates that XML as correct, even with the comments in place (as long as I remove all the “…” where you’ve omitted some things), so there’s something else going on. Is there any chance you could send me your entire XML, to “aswan at atlassian dot com”?