NextUrl Pagination - Return data until response = empty

[Mkallday]

1. Do I have to make 2 API calls to trigger a pagination flow?

2. How do I make it continuously call the nextUrl? Return all the data?

First Request

1.Tests Tab: Set Env Variable for nextUrl

Second Request

1. Tests Tab: Set Env Variable for nextUrl
2. Pre Run Tab: Run the First Request
   
   OR

Pre Run Tab: Both requests

[rangav]

@Mkallday no, you dont need 2 request for pagination to work.

• You can use the Tests Tab -> scripting and use custom script then you can run the same request until a desired result.
• See example here - https://github.com/rangav/thunder-client-support/blob/master/docs/filters.md#retry-request

[Mkallday]

Can you be a little more specific? I've been arguing with ChatGPT all day and I have yet to successfully paginate.

[Mkallday]

@rangav Could you please give me a little more direction? Claude, Cody, ChatGPT and Genie aren't producing correct responses. I'm using the paid version and I won't ask anymore specific questions this year

[rangav]

no problem you can email us regarding subscription confirmation, better not to share here for privacy/security.

Can you please share your account number or email used in below contact form
https://www.thunderclient.com/contact

[Mkallday]

Just did so! @rangav

[rangav]

• remove all the pre requests in Pre Run tab
• remove the existing Tests also
• You can use custom scripting to get this functionality and use Tests tab -> scripting

Modify the below code as per your requirements

let json = tc.response.json;
let nextUrl = json["@attributes"].next;

if(nextUrl)
{
    tc.setVar('next_url', nextUrl)
    console.log("running next request", nextUrl);

    await tc.retryRequest();
}
else
{
   console.log("got all the data");
}

[Ak-sky]

I tried but await tc.retryRequest(); kept on calling the first request in loop instead of calling the nextUrl

[Ak-sky]

and if I am sending custom requests using axios

let nextUrl = JSON.stringify(jsonData['next'])

if(nextUrl)
{
    tc.setVar('next_url', nextUrl)
    console.log("running next request", nextUrl);
    const response = await axios.get(nextUrl);
}
else
{
  console.log("got all the data");
}

I am getting below error

Error in Script: - connect ECONNREFUSED 127.0.0.1:80

Am I supposed to send token with axios? If yes how?

[Ak-sky]

Can someone please reply on this?

[rangav]

We will schedule call to resolve this

please share your availability using below form
https://www.thunderclient.com/contact

[Mkallday]

• remove all the pre requests in Pre Run tab
• remove the existing Tests also
• You can use custom scripting to get this functionality and use Tests tab -> scripting

Modify the below code as per your requirements

let json = tc.response.json;
let nextUrl = json["@attributes"].next;

if(nextUrl)
{
    tc.setVar('next_url', nextUrl)
    console.log("running next request", nextUrl);

    await tc.retryRequest();
}
else
{
   console.log("got all the data");
}

I STILL CAN'T PAGINATE.

• Previously, when I use your code above, it would "run" forever and eventually time out.
• Today, I watched the Terminal Output of this API call and this script running the same URL. It's not getting the NEXTURL.
• Claude + Cline have made dozens of variations, but NOTHING IS WORKING.

[Mkallday]

@rangav

[rangav]

I will verify and get back to you.