Running code on a specific node in a Jira DC Cluster

Hi everyone,

I am trying to develop a feature for a Jira Data Center plugin which requires that a REST endpoint is handled by a specific node, in order to fetch some data that is local to each node and isn’t replicated.
Since this plugin will be installed on many clusters with different load balancer configurations, I can’t rely on a specific way to ask the LB to route the request to the correct node (e.g. HTTP headers).
What I’ve tried so far is using ClusterManager#findLiveNodes so that any node can check whether the target node is online and get its IP. Unfortunately the Node class doesn’t contain the port on which Jira is running, otherwise - assuming that nodes are able to communicate with each other - I could easily create an HTTP request to that specific node and just forward the original request and return the response from whichever node handled the client API call.

Is anyone aware of a way to do an RPC from a node to another?
If there’s no way, an hackish workaround could be storing in an Active Object the port on which each node is running and manually updating it, but if there were a cleaner way, that would be much better!

Thanks!

1 Like

I’m working on a similar problem at the moment. Our use case is that we have some data that is held in memory on each node, we hold it in memory in a rolling buffer as the write volume can be quite large, and only the most recent entries are deemed to be of value.

We want to avoid constantly writing the above data to disk/database/replicated cache, as we are concerned about performance implications.

To summarise, we need the ability to fetch some data from all nodes in the cluster on demand, from any given node that an admin may be interacting with from our UI.

As far as I can tell the only “sanctioned” way to do this is to broadcast a cluster message via ClusterMessagingService and then each node will write back its current buffer on receipt of the message (destination up in the air, could be DB/disk/replicated cache).

From what I know the support zip app does something similar to the approach mentioned above.

Of course the above process is asynchronous, and you need to handle various error states, which makes this quite complicated. Distributed systems are hard. We will probably end up with a polling solution due to the lack of async servlet support in Jira (as far as I know).

I don’t think you can guarantee direct HTTP communication between nodes will be possible in all cases, I don’t think Atlassian documents that firewall rules should allow this? I could be wrong.

TLDR is it’s hard, in general DC nodes act independently and synchronise via cache invalidation (as recommended here: Jira DC Cache Replication - dev tips & tricks), this case where we need to pull data from specific nodes is new to us.

2 Likes