HTTP client
Create and execute HTTP requests
an HttpResponse object
curl handles are re-used on the level of the connection object, that is,
each HttpClient object is separate from one another so as to better
separate connections.
If you don't pass in a curl handle to the handle parameter,
it gets created when a HTTP verb is called. Thus, if you try to get handle
after creating a HttpClient object only passing url parameter, handle
will be NULL. If you pass a curl handle to the handle parameter, then
you can get the handle from the HttpClient object. The response from a
http verb request does have the handle in the handle slot.
print()
print method for HttpClient objects
HttpClient$print(x, ...)
xself
...ignored
new()
Create a new HttpClient object
HttpClient$new( url, opts, proxies, auth, headers, handle, progress, hooks, verbose )
url(character) A url. One of url or handle required.
optsany curl options
proxiesa proxy() object
authan auth() object
headersnamed list of headers, see http-headers
handlea handle()
progressonly supports httr::progress(), see progress
hooksa named list, see hooks
verbosea special handler for verbose curl output,
accepts a function only. default is NULL. if used, verbose
and debugfunction curl options are ignored if passed to opts
on $new() and ignored if ... passed to a http method call
urls(character) one or more URLs
A new HttpClient object
get()
Make a GET request
HttpClient$get(path = NULL, query = list(), disk = NULL, stream = NULL, ...)
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
diska path to write to. if NULL (default), memory used.
See curl::curl_fetch_disk() for help.
streaman R function to determine how to stream data. if
NULL (default), memory used. See curl::curl_fetch_stream()
for help
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
post()
Make a POST request
HttpClient$post( path = NULL, query = list(), body = NULL, disk = NULL, stream = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
bodybody as an R list
diska path to write to. if NULL (default), memory used.
See curl::curl_fetch_disk() for help.
streaman R function to determine how to stream data. if
NULL (default), memory used. See curl::curl_fetch_stream()
for help
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
put()
Make a PUT request
HttpClient$put( path = NULL, query = list(), body = NULL, disk = NULL, stream = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
bodybody as an R list
diska path to write to. if NULL (default), memory used.
See curl::curl_fetch_disk() for help.
streaman R function to determine how to stream data. if
NULL (default), memory used. See curl::curl_fetch_stream()
for help
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
patch()
Make a PATCH request
HttpClient$patch( path = NULL, query = list(), body = NULL, disk = NULL, stream = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
bodybody as an R list
diska path to write to. if NULL (default), memory used.
See curl::curl_fetch_disk() for help.
streaman R function to determine how to stream data. if
NULL (default), memory used. See curl::curl_fetch_stream()
for help
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
delete()
Make a DELETE request
HttpClient$delete( path = NULL, query = list(), body = NULL, disk = NULL, stream = NULL, encode = "multipart", ... )
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
bodybody as an R list
diska path to write to. if NULL (default), memory used.
See curl::curl_fetch_disk() for help.
streaman R function to determine how to stream data. if
NULL (default), memory used. See curl::curl_fetch_stream()
for help
encodeone of form, multipart, json, or raw
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
head()
Make a HEAD request
HttpClient$head(path = NULL, query = list(), ...)
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
verb()
Use an arbitrary HTTP verb supported on this class Supported verbs: "get", "post", "put", "patch", "delete", "head". Also supports retry
HttpClient$verb(verb, ...)
verban HTTP verb supported on this class: "get", "post", "put", "patch", "delete", "head". Also supports retry.
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
\dontrun{
(x <- HttpClient$new(url = "https://httpbin.org"))
x$verb('get')
x$verb('GET')
x$verb('GET', query = list(foo = "bar"))
x$verb('retry', 'GET', path = "status/400")
}
retry()
Retry a request
HttpClient$retry( verb, ..., pause_base = 1, pause_cap = 60, pause_min = 1, times = 3, terminate_on = NULL, retry_only_on = NULL, onwait = NULL )
verban HTTP verb supported on this class: "get", "post", "put", "patch", "delete", "head". Also supports retry.
...For retry, the options to be passed on to the method
implementing the requested verb, including curl options. Otherwise,
curl options, only those in the acceptable set from curl::curl_options()
except the following: httpget, httppost, post, postfields, postfieldsize,
and customrequest
pause_base, pause_cap, pause_minbasis, maximum, and minimum for
calculating wait time for retry. Wait time is calculated according to the
exponential backoff with full jitter algorithm. Specifically, wait time is
chosen randomly between pause_min and the lesser of pause_base * 2 and
pause_cap, with pause_base doubling on each subsequent retry attempt.
Use pause_cap = Inf to not terminate retrying due to cap of wait time
reached.
timesthe maximum number of times to retry. Set to Inf to
not stop retrying due to exhausting the number of attempts.
terminate_on, retry_only_ona vector of HTTP status codes. For
terminate_on, the status codes for which to terminate retrying, and for
retry_only_on, the status codes for which to retry the request.
onwaita callback function if the request will be retried and a wait time is being applied. The function will be passed two parameters, the response object from the failed request, and the wait time in seconds. Note that the time spent in the function effectively adds to the wait time, so it should be kept simple.
Retries the request given by verb until successful
(HTTP response status < 400), or a condition for giving up is met.
Automatically recognizes Retry-After and X-RateLimit-Reset headers
in the response for rate-limited remote APIs.
\dontrun{
x <- HttpClient$new(url = "https://httpbin.org")
# retry, by default at most 3 times
(res_get <- x$retry("GET", path = "status/400"))
# retry, but not for 404 NOT FOUND
(res_get <- x$retry("GET", path = "status/404", terminate_on = c(404)))
# retry, but only for exceeding rate limit (note that e.g. Github uses 403)
(res_get <- x$retry("GET", path = "status/429", retry_only_on = c(403, 429)))
}
handle_pop()
reset your curl handle
HttpClient$handle_pop()
url_fetch()
get the URL that would be sent (i.e., before executing the request) the only things that change the URL are path and query parameters; body and any curl options don't change the URL
HttpClient$url_fetch(path = NULL, query = list())
pathURL path, appended to the base URL
queryquery terms, as a named list. any numeric values are
passed through format() to prevent larger numbers from being
scientifically formatted
URL (character)
x <- HttpClient$new(url = "https://httpbin.org")
x$url_fetch()
x$url_fetch('get')
x$url_fetch('post')
x$url_fetch('get', query = list(foo = "bar"))
clone()
The objects of this class are cloneable with this method.
HttpClient$clone(deep = FALSE)
deepWhether to make a deep clone.
A little quirk about crul is that because user agent string can
be passed as either a header or a curl option (both lead to a User-Agent
header being passed in the HTTP request), we return the user agent
string in the request_headers list of the response even if you
pass in a useragent string as a curl option. Note that whether you pass
in as a header like User-Agent or as a curl option like useragent,
it is returned as request_headers$User-Agent so at least accessing
it in the request headers is consistent.
## Not run:
# set your own handle
(h <- handle("https://httpbin.org"))
(x <- HttpClient$new(handle = h))
x$handle
x$url
(out <- x$get("get"))
x$handle
x$url
class(out)
out$handle
out$request_headers
out$response_headers
out$response_headers_all
# if you just pass a url, we create a handle for you
# this is how most people will use HttpClient
(x <- HttpClient$new(url = "https://httpbin.org"))
x$url
x$handle # is empty, it gets created when a HTTP verb is called
(r1 <- x$get('get'))
x$url
x$handle
r1$url
r1$handle
r1$content
r1$response_headers
r1$parse()
(res_get2 <- x$get('get', query = list(hello = "world")))
res_get2$parse()
library("jsonlite")
jsonlite::fromJSON(res_get2$parse())
# post request
(res_post <- x$post('post', body = list(hello = "world")))
## empty body request
x$post('post')
# put request
(res_put <- x$put('put'))
# delete request
(res_delete <- x$delete('delete'))
# patch request
(res_patch <- x$patch('patch'))
# head request
(res_head <- x$head())
# query params are URL encoded for you, so DO NOT do it yourself
## if you url encode yourself, it gets double encoded, and that's bad
(x <- HttpClient$new(url = "https://httpbin.org"))
res <- x$get("get", query = list(a = 'hello world'))
# access intermediate headers in response_headers_all
x <- HttpClient$new("https://doi.org/10.1007/978-3-642-40455-9_52-1")
bb <- x$get()
bb$response_headers_all
## End(Not run)
## ------------------------------------------------
## Method `HttpClient$verb`
## ------------------------------------------------
## Not run:
(x <- HttpClient$new(url = "https://httpbin.org"))
x$verb('get')
x$verb('GET')
x$verb('GET', query = list(foo = "bar"))
x$verb('retry', 'GET', path = "status/400")
## End(Not run)
## ------------------------------------------------
## Method `HttpClient$retry`
## ------------------------------------------------
## Not run:
x <- HttpClient$new(url = "https://httpbin.org")
# retry, by default at most 3 times
(res_get <- x$retry("GET", path = "status/400"))
# retry, but not for 404 NOT FOUND
(res_get <- x$retry("GET", path = "status/404", terminate_on = c(404)))
# retry, but only for exceeding rate limit (note that e.g. Github uses 403)
(res_get <- x$retry("GET", path = "status/429", retry_only_on = c(403, 429)))
## End(Not run)
## ------------------------------------------------
## Method `HttpClient$url_fetch`
## ------------------------------------------------
x <- HttpClient$new(url = "https://httpbin.org")
x$url_fetch()
x$url_fetch('get')
x$url_fetch('post')
x$url_fetch('get', query = list(foo = "bar"))Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.