Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!
Get Started for Free

rbind_pages

Combine pages into a single data frame


Description

The rbind_pages function is used to combine a list of data frames into a single data frame. This is often needed when working with a JSON API that limits the amount of data per request. If we need more data than what fits in a single request, we need to perform multiple requests that each retrieve a fragment of data, not unlike pages in a book. In practice this is often implemented using a page parameter in the API. The rbind_pages function can be used to combine these pages back into a single dataset.

Usage

rbind_pages(pages)

Arguments

pages

a list of data frames, each representing a page of data

Details

The rbind_pages function generalizes base::rbind and plyr::rbind.fill with added support for nested data frames. Not each column has to be present in each of the individual data frames; missing columns will be filled up in NA values.

Examples

# Basic example
x <- data.frame(foo = rnorm(3), bar = c(TRUE, FALSE, TRUE))
y <- data.frame(foo = rnorm(2), col = c("blue", "red"))
rbind_pages(list(x, y))


baseurl <- "https://projects.propublica.org/nonprofits/api/v2/search.json"
pages <- list()
for(i in 0:20){
  mydata <- fromJSON(paste0(baseurl, "?order=revenue&sort_order=desc&page=", i))
  message("Retrieving page ", i)
  pages[[i+1]] <- mydata$organizations
}
organizations <- rbind_pages(pages)
nrow(organizations)
colnames(organizations)

jsonlite

A Simple and Robust JSON Parser and Generator for R

v1.7.2
MIT + file LICENSE
Authors
Jeroen Ooms [aut, cre] (<https://orcid.org/0000-0002-4035-0289>), Duncan Temple Lang [ctb], Lloyd Hilaiel [cph] (author of bundled libyajl)
Initial release

We don't support your browser anymore

Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.