Custom Entities - Can not set an object in storage

Forge
,
1

Hi everyone,

I have defined a custom entity in my forge manifest file:

storage:
    entities:
      - name: 'exampleentity'
        attributes:
          issueId:
            type: 'string'
          content:
            type: 'string'
        indexes:
          - issueId
          - name: 'content'
            range:
              - 'content'

When I’m trying to set it like this:

        await storage
          .entity("exampleentity").set("10055-0", {issueId: "10055", content: "......."})

I get the following error:

ERROR   14:11:12.338  8a91f2028868b8d3  An unexpected error occurred
Error: An unexpected error occurred
    at Function.forErrorCode (webpack://jira-global-page-custom-ui/node_modules/@forge/storage/out/errors.js:35)
    at assertNoErrors (webpack://jira-global-page-custom-ui/node_modules/@forge/storage/out/global-storage.js:9)
    at GlobalStorage.mutation (webpack://jira-global-page-custom-ui/node_modules/@forge/storage/out/global-storage.js:122)
    at async GlobalStorage.setEntity (webpack://jira-global-page-custom-ui/node_modules/@forge/storage/out/global-storage.js:90)
    at async processIssue (webpack://jira-global-page-custom-ui/src/jira/processIssue.ts:130)
    at async run (webpack://jira-global-page-custom-ui/src/issueUpdatedTrigger.ts:13)
t=2023-07-02T14:11:12+0000 lvl=dbug msg="got response for request" id=c3fc941e6cee typ=proxy reqpath=/issueUpdatedTrigger
t=2023-07-02T14:11:12+0000 lvl=dbug msg="add entry" obj=storage entrysz=13999 sz=255448

It used to work a few days ago. There’s nothing more in the logs, which could hint what’s the problem. Any ideas of hints are appreciated!

2

This seems to be a bug in the Storage API. We are currently looking into it. We’ll get back to you once we have more details on this.

1 Like
3

@BasuDubey & @SunandanGokhroo I’m still getting this error. Have you been able to diagnose the problem?

4

Hi @GeoSystems thank you for pointing out this error.
Our initial exploration revealed that this is due to large key sizes being used for partition key and range key, which is exceeding the limits within our validation checks. It looks like in this particular case you are exceeding range key limit of 1024 bytes currently. Reducing the size to within the limits should help you mitigate this error, however you would need to leave some buffer(calculated dynamically) as we need to run some operations internally as well. We are actively working, on what are the largest reasonable sizes we can support and will be able to update you shortly, as well as improving error codes to better cover this case in the future.

2 Likes
5

Thank you @SunandanGokhroo - I have tested and it works when the content field is only a few characters. Hence:

// Works with small content:
await storage.entity("exampleentity").set("1", {issueId: "i1", content: "ABC"});
// Fails when content is long:
await storage.entity("exampleentity").set("2", {issueId: "i1", content: "ABC..........."});

In my application, I’m trying to index the content attribute, which can be up to 200KB string, which is within the limit of 240KB per entity object. I need to be able to search for entities by words or expressions contained in their content.

This will not work if the index key is limited to 1024bytes. Is there any chance this limit can be increased substantially?

6

Hi @GeoSystems happy to hear that this works. Let me look into your query and work with team to get back to you asap. Thank you so much!

7

Hi @GeoSystems we explored internally to find a way we can support you. However, as of now it does not look like there is scope for size increment to index key, since this is a database limitation for us. However I have recorded this as a potential use case and a pain point, and we will let you know if we find a workaround. Thank you!