Rovo agent is not displaying the message returned from an action

Hi there,
I created a Rovo agent to trigger an action and returned a message.
Prompt:

You must list all the active rulesets for the space using the below steps:
        a. invoke the list-ruleset-action with the space key and space id of the space in which you are invoked. Inform the users of the action you are performing.
        b. After successfully executing the list-ruleset-action, display the returned message as it is. Do not add pre-written rules or fallback content.

action (forge function):

const result = "ruleset includes inclusive language";
        console.log(result)
        return result;

The action is invoked and runs smoothly( as per the logs). However, the Agent says the action was not invoked:

The action to list out the ruleset for the space was not successful. There was an error in executing the action, specifically a failure to invoke the action. Please try again later or check for any issues that might be causing this error. There were no links provided with this action.

There is also inconsistency. Sometimes the Agent says:

The action was executed successfully. There were no links provided with this action.

It never displays the returned message.

Hi Mayank,

The prompt needs some work:

  1. How is this agent where is it supposed to get space key and space id from?
  2. in case the above step fails (because at times AI hallucinates), what’s the fallback?
  3. after the list-ruleset-action is triggered, how is output from that function passed on the next step?

That’s why AI is inconsistent and confused. When it runs successfully, the output doesn’t reach the last step. Other times, it hallucinates in the first step.

Hope this helps.

Hi Anmol,
Thanks for your response.
The issue is not with the space key or the space id. The list-ruleset-action is triggered with the correct inputs every time.
The output of this action should be displayed to the user as is. For ex: “ruleset includes inclusive language”.
This is where I am stuck. The returned message does not show up in the chat, instead it just says the action was run successfully.
the last 3 lines of the function is:

const result = "ruleset includes inclusive language";
        console.log(result)
        return result;

I can see the message logged in the console, and then the same message is returned and should be displayed in the Rovo chat.

Hi @MayankJhaK15t ,

I haven’t been able to reproduce the problem you are seeing whereby the Agent log is inconsistent with the observed behaviour, but I have been able to create an Agent that reliably produces the behaviour I think you want. Here is my code:

manifest.yml

modules:
  rovo:agent:
    - key: ruleset-retriever-agent
      name: Retrieve Rulesets Agent
      description: An agent for testing Forge agent functionality
      prompt: >
        Sections of this prompt are separated with a '----' and labeled with a title in uppercase.  
        Subsections are separated with a '---' to help you understand how the information is related.

        ----
        DEFINITIONS

        A rule is a statement that users must follow.
        A ruleset is a set of rules.

        ----
        AGENT INSTRUCTIONS

        You are an Agent that is able to provide users with a ruleset that is applicable to a particular context.
        When a user is in a space context and requests the current rulesets, invoke the 
        action "retrieve-space-rulesets" to get the list of rulesets.
        The action will return a list of strings, each of which is a rule.
        After retrieving a list of rulesets, display the list to the user using the template "PRESENT_RULESET_TEMPLATE" 
        if at least one ruleset is in the list or present the template "NO_RULES_TEMPLATE" if no rulesets are in the list:

        ----
        TEMPLATES

        ---
        PRESENT_ONE_RULE_TEMPLATE

        * [rule]

        ---
        PRESENT_RULESET_TEMPLATE

        The applicable rulesets are as follows:
        * {PRESENT_ONE_RULE_TEMPLATE}

        ---
        NO_RULES_TEMPLATE

        There are no rules applicable to this space.
      conversationStarters:
        - Retrieve the rulesets applicable to this space.
        - Retrieve a maximum of 2 rulesets applicable to this space.
      actions:
        - retrieve-space-rulesets
  action:
    - key: retrieve-space-rulesets
      name: Retrieve the rulesets applicable to a given space.
      function: retrieveRulesets
      actionVerb: GET
      description: >
        This action retrieves the ruleset applicable to a given space. The ruleset is 
        retuned as an array of strings where each string represents a single rule.
      inputs:
        maxResults:
          title: The maximum number of results to return.
          type: string
          required: false
          description: |
            "The maximum number of results to return if specified by the user."
  function:
    - key: retrieveRulesets
      handler: index.retrieveRulesets
app:
  runtime:
    name: nodejs22.x
  id: ari:cloud:ecosystem::app/xxxxxxx <- run forge register to make this app yours

src/index.js

const emptyRuleset = []

const nonEmptyRuleset = [
  "Do not run with scissors",
  "Do not eat yellow snow",
  "Maximise the cow bells",
]

export const retrieveRulesets = (payload) => {
  console.log(`Retrieving rulesets for payload: ${JSON.stringify(payload, null, 2)}`);
  const maxResults = payload.maxResults ?? 10;
  const ruleset = Math.random() > 0.5 ? emptyRuleset : nonEmptyRuleset;
  const rulesetToReturn = ruleset.slice(0, maxResults);
  return rulesetToReturn;
}

Regards,
Dugald

Hi @dmorrow,
Thanks for the detailed solution.
I added the DEFINITIONS section to my manifest and that solved the issue.

1 Like