Work around for the character limit when storing Forge Storage API

Forge
1

Hi all, I wanted to ask if there was any solution to get around the limit of characters that can be saved via forge.set('key', value), I had thought of saving them in chunk but it doesn’t work.

and another question, when i then take the data how can i merge the chunks according to you?

2

Hi @MauroRemondi,

Do you mean the storage.set(‘key’, value) ?

Can you tell me a little bit more about the size / type of data you’re planning to store?

Cheers,
Melissa

3

yeah sorry i mean storage.set(‘key’, value) and i have an array of objects and the error is
Content for key "value" was 325316 characters, but should be at most 131072 characters
my idea was to split the value in chunks loop the array of chunks and save the chunk with key: value{index of loop}

but another question that i have how can i get all the chunks?

4

Hi @MauroRemondi,

You’ll probably need to split your array into smaller arrays and store it that way. As far as I’m aware, there’s nothing implemented within the forge storage api to break data into smaller key/value pairs and track it automatically, so you will most likely need to implement this yourself within your app.

I hope this explains.
Cheers,
Mel

5

@mpaisley @AdamMoore I’ve just hit this limit too with a recent customer support request.

There is zero bloody mention of a character limit in the Forge storage API docs: https://developer.atlassian.com/platform/forge/platform-quotas-and-limits/#storage-limits

There’s a 128KB key-value chunk limit which I have coded for. Is this a bug?

–

Edit: so 128 * 1024 = 131072. I think my chunking function was encoding incorrectly.

Atlassian should be moving this stupid complexity server-side and offering something like Cloudflare’s 25MB key-value storage limit.

Here’s my function for any other masochists trying to build on this platform:

const chunkToLimit = (arr, kbLimit = 128) => {
  let chunk = []
  const result = []
  for (let i = 0; i < arr.length; i += 1) {
    const item = arr[i]
    const newSize = Buffer.byteLength(
      JSON.stringify(chunk.concat(item)),
      'utf-8'
    )
    if (newSize > kbLimit * 1024) {
      result.push(chunk)
      chunk = []
    }
    chunk.push(item)
  }
  if (chunk.length > 0) {
    result.push(chunk)
  }
  return result
}
3 Likes