A bug in REST API search endpoint pagination

Trello
1

Hello,

I’ve already reported this error on Atlassian Community here. The problem is that when you call the same GET path of GET /1/search many times, you get different cards in an indeterministic way. I think the @[Iain Dooley] code better explains the problem:

He took the following code:

ran it in BenkoBo, and it produced this output:

Duplicate id: 5b835034317af003201e402e found at element: 0 on page: 3
Duplicate id: 5e8e61127645162f34fb7117 found at element: 1 on page: 4
Duplicate id: 5f220468e5b7934721039e6f found at element: 2 on page: 4
Duplicate id: 5de455c22a56df2818f268ee found at element: 1 on page: 5
Duplicate id: 5db1f1d0e3f8ba12a48d473e found at element: 3 on page: 6
Duplicate id: 5f600723e9900966f0545438 found at element: 6 on page: 6
Duplicate id: 5f1bf11e06d25e0e4ebf0f50 found at element: 1 on page: 7
Duplicate id: 5f5e2f809d71f74155537b72 found at element: 0 on page: 8
Duplicate id: 5f54edfca0a42d4537646e5c found at element: 3 on page: 9
Duplicate id: 5f5ee943bb328f63d84083a8 found at element: 0 on page: 10
Duplicate id: 5f5e25151681c93399fbd1b0 found at element: 0 on page: 13
Duplicate id: 5f49d6728c97a3310e230e7a found at element: 2 on page: 14
Duplicate id: 5ea8dc9aa44a2d79555658f7 found at element: 1 on page: 21
Duplicate id: 5ec449c0c3f3f565d6a9e240 found at element: 2 on page: 21
Duplicate id: 5e7a754ca5fe0e72ce7964dd found at element: 7 on page: 24
Duplicate id: 5ecf1cb14218bd11fa966865 found at element: 2 on page: 25
Duplicate id: 5ec73d919b49488bafbb90ae found at element: 3 on page: 25
Duplicate id: 5ed59ec331be002c6d6f3dec found at element: 4 on page: 25
Duplicate id: 5efd51196034fe7a2486691e found at element: 6 on page: 25
Duplicate id: 5f17d9d030c76241e0361e69 found at element: 0 on page: 28
Duplicate id: 5db8f3e223b9986e6de0e04e found at element: 1 on page: 28
Duplicate id: 5d4111403627560a47eb75f2 found at element: 2 on page: 28
Duplicate id: 5d6c864e5a030266b880893e found at element: 3 on page: 28
Duplicate id: 5cf35c4e78ea7a37b9fc1663 found at element: 4 on page: 28
Duplicate id: 5da3b2a0d807646e6a786ed1 found at element: 5 on page: 28
2

Do you have a public board that you can reproduce this one? I’ve not been able to reproduce on a board I’ve been playing with.

How active is the board you’re querying? And how new is it?

Under the hood, the /1/search endpoint passes everything directly to an ElasticSearch instance. ElasticSearch takes some time to index cards/boards if there is a lot of activity on the board. It sounds like it is likely the case that ElasticSearch is adding new cards to the index as you’re querying it.

In general, /1/search's primary use is intended to be finding a specific string. The /1/search endpoint is what is used to back search in the Trello clients:
image

If you’re trying to get the entire state of a board (as opposed to trying to find a specific string in the content of cards/boards), then you should query the board’s resource directly.

3

Yes, here is the board: Trello

Also, I’ve prepared a script that you can run to reproduce the issue
GitHub - Alberto42/test-trello-rest-api: Shows a bug in trello rest api. This script is using the board that I’ve sent you

4

Alright, I was able to reproduce consistently. Below is Javascript to reproduce, as well.

I’ve submitted this as a bug with the team responsible for this service. If if is prioritized and worked on, I will update this thread.

let go = async () => {
  let myKey = "";
  let myToken = "";

  let getCards = async (
    key,
    token,
    boardId = "5f71fd9dae37660676fd09b0",
    limit = 100,
    iterations = 4
  ) => {
    let toIterate = [...Array(iterations).keys()].map((x) => ++x);
    return Promise.all(
      toIterate.map((i) => {
        console.log(`On iteration: ${i}`);
        return fetch(
          `https://api.trello.com/1/search?query=is:open&cards_limit=${limit}&cards_page=${i}&idBoards=${boardId}&key=${key}&token=${token}`
        )
          .then((data) => data.json())
          .then((resp) => resp.cards.map((card) => card.id));
      })
    );
  };

  let cards = await getCards(myKey, myToken);
  cards = cards.flat();
  console.log(`Total fetched cards ${cards.length}`);
  let cardsSet = new Set(cards);
  p = cardsSet;
  console.log(`Total unique cards ${cardsSet.size}`);
};

go();