How Can I Paginate Through a REST API Using Cursor-Based PowerShell Requests?
Asked by
MapleHarbor42
I'm calling a REST API that returns a maximum of 25 items per request. Each response includes a cursor: the first response has only an `after` cursor, middle responses have both `before` and `after`, and the final response has only `before`. I want to keep requesting pages until the response no longer contains an `after` cursor, while collecting every item.
My current approach makes the first request, then enters a `do`/`while` loop and builds the next URL from `$response.result_info.cursors.after`. However, I keep getting an infinite loop or repeatedly requesting the wrong page. I suspect I'm not updating the response and cursor correctly after each iteration. What is the right way to structure this loop in PowerShell?
Answers
Be careful with the URL string. PowerShell does not expand variables inside ordinary single-quoted strings. Use a double-quoted string or the format operator, such as:
```powershell
$uri = "$baseUri&cursor=$($response.result_info.cursors.after)"
```
With single quotes, `$response` and the subexpression are generally passed literally, which can produce a request containing the text `$($response.result_info.cursors.after)` instead of the actual cursor.
Reply to this answer
Your original code uses `$response_cursor` for the new request, but the next loop iteration still reads the cursor from `$response`. Assign the new response back to the variable used to build the next URL, or always reference the cursor from `$response_cursor` and then update it consistently. Mixing those two variables is what prevents the cursor from advancing.
Reply to this answer
A debugger or breakpoint can help verify this quickly. Inspect the generated URI and the value of `cursors.after` on every iteration. Also confirm that the API expects the cursor as a query parameter exactly as shown; some APIs return a complete next-page URL or require URL encoding. Stop when `after` is null or empty, not merely when the response contains a cursor object.
Reply to this answer
Use a `do`/`until` or `do`/`while` loop with the current response as the source of the next cursor. Keep the base URL separate, update the response on every iteration, and add each page’s items to a collection. For example:
```powershell
$baseUri = 'https://api.domain.com/items?per_page=25'
$allItems = @()
$response = $null
do {
$uri = if ($response -and $response.result_info.cursors.after) {
"$baseUri&cursor=$($response.result_info.cursors.after)"
} else {
$baseUri
}
$response = Invoke-RestMethod -Uri $uri -Method Get -Headers $headers
$allItems += $response.result
} while ($response.result_info.cursors.after)
```
The important part is that `$response` is replaced with the newest response before the loop condition is checked. Otherwise, the same cursor can be used repeatedly.
Reply to this answer
Add your answer
Answers are reviewed before publication.
MapleHarbor42: That makes sense. I was rebuilding the URL from the original response instead of consistently using the response returned by the previous iteration.