Retrieves or sets the auto broadcast join threshold
Configures the maximum size in bytes for a table that will be broadcast to all worker nodes when performing a join. By setting this value to -1 broadcasting can be disabled. Note that currently statistics are only supported for Hive Metastore tables where the command 'ANALYZE TABLE <tableName> COMPUTE STATISTICS noscan' has been run, and file-based data source tables where the statistics are computed directly on the files of data.
spark_auto_broadcast_join_threshold(sc, threshold = NULL)
sc |
A |
threshold |
Maximum size in bytes for a table that will be broadcast to all worker nodes
when performing a join. Defaults to |
Other Spark runtime configuration:
spark_adaptive_query_execution(),
spark_advisory_shuffle_partition_size(),
spark_coalesce_initial_num_partitions(),
spark_coalesce_min_num_partitions(),
spark_coalesce_shuffle_partitions(),
spark_session_config()
Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.