In my Jira P2 app, I am using two ConcurrentHashMap variables (propertiesConfigurationMap and filteredPropertiesMap) and
1. propertiesConfigurationMap
Map<String, CustomClass> propertiesConfigurationMap =new ConcurrentHashMap<String, CustomClass>();
propertiesConfigurationMap.computeIfAbsent(fileNameKey, k -> Utilities.loadProperties(fileNameKey));
final PropertiesConfiguration propertiesConfig = propertiesConfigurationMap.get(fileNameKey);
2. filteredPropertiesMap
Map<String, Map<String, Properties>> filteredPropertiesMap = new ConcurrentHashMap<String, Map<String, Properties>>();
filteredPropertiesMap.computeIfAbsent(fileNameKey, k -> new ConcurrentHashMap<String, Properties>());
final Map<String, Properties> propertiesMapByPrefix = filteredPropertiesMap.get(fileNameKey);
propertiesMapByPrefix.computeIfAbsent(prefixKey, k -> Utilities.getFilteredProperties(propertiesConfig, prefixKey));
Properties properties = propertiesMapByPrefix.get(prefixKey);
3. I am implementing ClusterMessageConsumer and ClusterMessagingService interfaces as well
The maps are loaded based on properties file, which contains key value pairs.
Prior to moving to ConcurrentHashMap, we were using synchronized methods, where we encountered problems working on DataCentre instances.
This solution works perfectly except for the following:
The total app code (code shared above and remaining app code) is executed by daemon threads (starting with name - http-nio-8080-exec-xx).
Once the app code is executed, the Daemon thread goes back to WAITING state
At the end, strangely we noticed that a normal thread is created starting with name pool-xxx-thread-1 (where xxx is a two or three digit number). This thread is in WAITING state and never gets terminated. In fact, the thread is created after exiting from the app code. Prior to using ConcurrentHashMap, I never encountered such non terminating normal threads in WAITING state. Also, this behaviour is observed in both server and DC instances
I request help from the community and Atlassian folks to understand why JDK/Jira is creating the threads, as I see that my app code was executed using the daemon thread.
The format pool-N-thread-M is the default pattern for Java Executors framework, i.e. when we create a new thread pool for the executor service, by default threads are named like this; quoting from above link;
Returns a default thread factory used to create new threads. This factory creates all new threads used by an Executor in the same ThreadGroup. If there is a SecurityManager, it uses the group of System.getSecurityManager(), else the group of the thread invoking this defaultThreadFactory method. Each new thread is created as a non-daemon thread with priority set to the smaller of Thread.NORM_PRIORITY and the maximum priority permitted in the thread group. New threads have names accessible via Thread.getName() of pool-N-thread-M , where N is the sequence number of this factory, and M is the sequence number of the thread created by this factory.
Returns:
a thread factory
You may be using Executors yourself in your plugin or this may be a pool that Jira itself is creating. Two points to note are;
(1) the creation of the threads in this pool may be delayed until a job is submitted to this executor. Hence it may be the case that even if a thread pool is created for an ExecutorService, you may be missing threads for it in the threaddump if no work is submitted to this executor yet. This may explain sometimes not seeing Threads named like this
(2) these thread pools are also used by Jira components and some third-party apps. They are not inherently dangerous and don’t normally block JVM from exiting, even if the thread may be kept alive in WAITING state for an extended period to execute async tasks, hence they may show up in thread dumps.
Attaching a sample code snippet below to demonstrate the above points; it creates a pool and threaddump so that we can see such threads (this is pure Java, no Jira API involved).
At sum, seeing such threads in your thread dumps is not unexpected while running your app in Jira. Does this answer your question?
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadInfo;
import java.lang.management.ThreadMXBean;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import static java.lang.String.format;
class Scratch {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(5);
// point 1 above; if you comment below, no "pool-1-thread-1" is created
submitWorkItem(executor);
ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
System.out.println(threadDumpDescription());
// point 2 above; thread pools exit normally when terminating JVM
// if you comment below like though, JVM does not terminate the execution
System.exit(0);
}
private static void submitWorkItem(ExecutorService executor) {
executor.submit(new Runnable() {
@Override
public void run() {
System.out.println("here is some dummy async work");
}
});
}
public static String threadDumpDescription() {
StringBuilder threadDumpDescription = new StringBuilder();
ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
ThreadInfo[] threadInfos = threadMXBean.dumpAllThreads(true, true);
for (ThreadInfo threadInfo : threadInfos) {
threadDumpDescription.append(format("\"%s\"%n\tjava.lang.Thread.State: %s",
threadInfo.getThreadName(), threadInfo.getThreadState()));
for (StackTraceElement stackTraceElement : threadInfo.getStackTrace()) {
threadDumpDescription.append(System.lineSeparator()).append("\t\tat ").append(stackTraceElement);
}
threadDumpDescription.append(System.lineSeparator()).append(System.lineSeparator());
}
return threadDumpDescription.toString();
}
}
Curious to know, if the usage of ConcurrentHashMap objects will result in ExecutorService getting kicked off in the background. Because in my app, before using ConcurrentHashMap objects, I did not find the pool-1-thread-1 threads but noticing them now.
Hi @ravi, ConcurrentHashMap itself does not trigger Executors by itself. However, a thread showing up, maybe a side effect of your business logic;
If you’d recall point 1 above, depending on the configuration of threadpool, threads may not be created until work is submitted (in the example snippet above, we create a threadpool of 5 threads(Executors.newFixedThreadPool(5)), but not all the threads are created right away, i.e. if we comment out the line submitting work (submitWorkItem(executor)), the pool-1-thread-1 type thread is not created at all, it does not show up in the generated thread dump). So I guess it may be the case that your code is not creating an Executor thread pool itself, but maybe causing work to be submitted to an already existing Executor thread pool in Jira (most probably created by Jira itself), causing it to create the first thread of that thread pool.
One way to spot exactly what’s happening, i.e. which component is creating the Executor thread pool in the first place, might be to instrument the ExecutorService class with a bytecode instrumentation library like Byteman, or using a monitoring tool like Dynatrace which can do the same. Such instrumentation that can give us a thread dump whenever someone calls a method from ExecutorService class for example. But given you are not creating an ExecutorService yourself, this will eventually turn out to be created by Jira anyways.