Kubernetes Configuration
Convenience function to initialize a Kubernetes configuration instead
of spark_config(), exposes common properties to set in Kubernetes
clusters.
spark_config_kubernetes(
master,
version = "2.3.2",
image = "spark:sparklyr",
driver = random_string("sparklyr-"),
account = "spark",
jars = "local:///opt/sparklyr",
forward = TRUE,
executors = NULL,
conf = NULL,
timeout = 120,
ports = c(8880, 8881, 4040),
fix_config = identical(.Platform$OS.type, "windows"),
...
)master |
Kubernetes url to connect to, found by running |
version |
The version of Spark being used. |
image |
Container image to use to launch Spark and sparklyr. Also known
as |
driver |
Name of the driver pod. If not set, the driver pod name is set
to "sparklyr" suffixed by id to avoid name conflicts. Also known as
|
account |
Service account that is used when running the driver pod. The driver
pod uses this service account when requesting executor pods from the API
server. Also known as |
jars |
Path to the sparklyr jars; either, a local path inside the container
image with the sparklyr jars copied when the image was created or, a path
accesible by the container where the sparklyr jars were copied. You can find
a path to the sparklyr jars by running |
forward |
Should ports used in sparklyr be forwarded automatically through Kubernetes?
Default to |
executors |
Number of executors to request while connecting. |
conf |
A named list of additional entries to add to |
timeout |
Total seconds to wait before giving up on connection. |
ports |
Ports to forward using kubectl. |
fix_config |
Should the spark-defaults.conf get fixed? |
... |
Additional parameters, currently not in use. |
Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.