I am trying to consume an API which returns structure below.

{
  data: [{...},{...},{...},{...},...],
  nextUrl: "url_goes_here";
}

The pages end when the nextUrl is null. I want to collect all the elements in data into one array while going through the paginated responses. I tried the following code segment.

const getUserData = async (url) => {
    const result = await fetch(url)
    .then(res => res.json())
    .then(res => {
        dataList= [...dataList, ...res.data];
        console.log(res.data)
        if (res.nextUrl !== null) {
          getUserData(res.nextUrl);
        } else {
          console.log("else ", dataList);
        }
    })
    .catch(err => {});
  return result;
}

I am trying to consume an API which returns structure below.

{
  data: [{...},{...},{...},{...},...],
  nextUrl: "url_goes_here";
}

The pages end when the nextUrl is null. I want to collect all the elements in data into one array while going through the paginated responses. I tried the following code segment.

const getUserData = async (url) => {
    const result = await fetch(url)
    .then(res => res.json())
    .then(res => {
        dataList= [...dataList, ...res.data];
        console.log(res.data)
        if (res.nextUrl !== null) {
          getUserData(res.nextUrl);
        } else {
          console.log("else ", dataList);
        }
    })
    .catch(err => {});
  return result;
}

The console.log can print the result. but I want to get all the data to get into a variable that can be used for further processing later.

• javascript
• asynchronous
• async-await

Share Improve this question Follow asked Feb 23, 2020 at 16:17 dhanushkacdhanushkac 55022 gold badges88 silver badges2626 bronze badges

Add a ment  | 

1 Answer 1

Sorted by: Reset to default 4

Your approach using recursion isn't bad at all, but you're not returning the chunk of data (there's no value returned by your second .then handler). (You're also falling into the fetch pitfall of not checking ok. It's a flaw IMHO in the fetch API design.)

There's no need for recursion though. I'd be tempted to just use a loop:

const getUserData = async (url) => {
    const result = [];
    while (url) {
        const response = await fetch(url);
        if (!response.ok) {
            throw new Error("HTTP error " + response.status);
        }
        const {data, nextUrl} = await response.json();
        result.push(...data);
        url = nextUrl;
    }
    return result;
};

But if you want to use recursion:

const getUserData = async (url) => {
    const response = await fetch(url);
    if (!response.ok) {
        throw new Error("HTTP error " + response.status);
    }
    const {data, nextUrl} = await response.json();
    if (nextUrl) {
        data.push(...await getUserData(nextUrl));
    }
    return data;
};

or

const getUserData = async (url, result = null) => {
    const response = await fetch(url);
    if (!response.ok) {
        throw new Error("HTTP error " + response.status);
    }
    const {data, nextUrl} = await response.json();
    if (!result) {
        result = data;
    } else {
        result.push(...data);
    }
    if (nextUrl) {
        result = await getUserData(nextUrl, result);
    }
    return result;
};