Is there a way to debug the Rovo Agent prompt?
I’m currently working on a Rovo agent and sometime is just get a generic error message invoking the agent and the only option given is to resend the message to the agent.
In playing around with my action that the agent invokes I would think it has something to do with the size of the response of the action. All is working when I return 78 objects from the action, when I return 79 objects the agent response gets truncated to a random looking length, an when I return 80 objects the generic error pops up.
Going over the Rovo docs I didn’t see any debug option so was wondering if there is anyone out there that found a way to debug this behaviour.
Hi @markrekveld ,
I don’t think it’s feasible to debug a Rovo Agent, but it might be possible to create a set of test cases and use automation to trigger the Agent. For example, perhaps you can define a set of prompts that should each result in a certain response or at least keywords in each response. For each prompt, your test harness could create a new artefact (e.g. Confluence page or Jira issue) containing the prompt text and also add the label “rovo-test” to each of the created artefacts. A configured automation rule could then be used to detect the new artefact (with the label to filter out reacting to artefacts created outside of the test harness) and then the automation rule could trigger your agent with the text in the artefact and finally create a comment on the artefact containing the response from your Rovo Agent. Your test harness would then inspect the text in each response to check it is as expected.
Regards,
Dugald
Rate limiting or quote perhaps?
Thanks @dmorrow that does give me some ideas to start creating smaller prompts maybe add some additional returns to the user to get to know here it is in the prompt where the issue comes in.
I’m not sure if limits or quotes are blocking, because I can call the agent again with a limited build count included and it happily continues and gives me responses again. Then again it may be an interesting new way to look at quotes or limits with AI. Do you know @MattDoar1 where I can see the quotes for Rovo? I’ve been looking around in the admin sections but have not found any Rovo quotes/limits/etc page with details on this.
It almost makes me feel like I’m talking to my 2 year old daughter. We can be fully focused on builds a Lego Duplo house and then all of the sudden she is gone talking to a doll or looking out the window, to then come back a minute later not even acknowledging she stepped out for bit.
Rovo usage quota | Rovo | Atlassian Support is what I’m aware of
And yes, kids are like AI. There’s a video waiting to be made
I found that page as well.
Hopefully Atlassian will include an admin page like this once my standard instance gets free Rovo for all.