sparklyr: copy_to.spark_connection – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

copy_to.spark_connection

Copy an R Data Frame to Spark

Description

Copy an R data.frame to Spark, and return a reference to the generated Spark DataFrame as a tbl_spark. The returned object will act as a dplyr-compatible interface to the underlying Spark table.

Usage

## S3 method for class 'spark_connection'
copy_to(
  dest,
  df,
  name = spark_table_name(substitute(df)),
  overwrite = FALSE,
  memory = TRUE,
  repartition = 0L,
  ...
)

Arguments

`dest`	A `spark_connection`.
`df`	An R `data.frame`.
`name`	The name to assign to the copied table in Spark.
`overwrite`	Boolean; overwrite a pre-existing table with the name `name` if one already exists?
`memory`	Boolean; should the table be cached into memory?
`repartition`	The number of partitions to use when distributing the table across the Spark cluster. The default (0) can be used to avoid partitioning.
`...`	Optional arguments; currently unused.

Value

A tbl_spark, representing a dplyr-compatible interface to a Spark DataFrame.

sparklyr

R Interface to Apache Spark

v1.6.2

Apache License 2.0 | file LICENSE

Authors

Javier Luraschi [aut], Kevin Kuo [aut] (<https://orcid.org/0000-0001-7803-7901>), Kevin Ushey [aut], JJ Allaire [aut], Samuel Macedo [ctb], Hossein Falaki [aut], Lu Wang [aut], Andy Zhang [aut], Yitao Li [aut, cre] (<https://orcid.org/0000-0002-1261-905X>), Jozef Hajnala [ctb], Maciej Szymkiewicz [ctb] (<https://orcid.org/0000-0003-1469-9396>), Wil Davis [ctb], RStudio [cph], The Apache Software Foundation [aut, cph]

Initial release