Enable parallelization on batch systems
This class is used to parameterize scheduler options on managed high-performance computing clusters.
BatchJobsParam(workers, cleanup = TRUE,
work.dir = getwd(), stop.on.error = TRUE, seed = NULL,
resources = NULL, conffile = NULL, cluster.functions = NULL,
progressbar = TRUE, jobname = "BPJOB",
reg.pars=list(seed=seed, work.dir=work.dir),
conf.pars=list(conffile=conffile, cluster.functions=cluster.functions),
submit.pars=list(resources=resources),
...)workers |
|
Number of workers to divide tasks
(e.g., elements in the first argument of bplapply)
between. On Multicore and SSH backends, this defaults to all
available nodes. On managed (e.g., slurm, SGE) clusters
workers defaults to NA, meaning that the number of
workers equals the number of tasks. See argument
n.chunks in chunk and
submitJobs for more information.
cleanup |
|
BatchJobs creates temporary
directories in the work.dir. If cleanup is set to
TRUE (default), the directories are removed from the file
systems automatically. Set this to FALSE whenever it
might become necessary to utilize any special functionality
provided by BatchJobs. To retrieve the registry, call
loadRegistry on the temporary
directory.
work.dir |
|
Directory to store temporary
files. Note that this must be shared across computational nodes
if you use a distributed computing backend. Default ist the
current working directory of R, see
getwd. Ignored when reg.pars is
provided.
stop.on.error |
|
Stop all jobs as soon as one
jobs fails (stop.on.error == TRUE) or wait for all jobs
to terminate. Default is TRUE.
seed |
|
Set an initial seed for the RNG. See
makeRegistry for more information.
Default is NULL where a random seed is chosen upon
initialization. Ignored when reg.pars is provided.
resources |
|
List of job specific resources
passed to submitJobs. Default is
NULL where the resources defined in the configuration are
used. Ignored when submit.pars is provided.
conffile |
|
URI to a custom BatchJobs
configuration file used for execution. Default is NULL
which relies on BatchJobs to handle configuration files. Ignored
when conf.pars is provided.
cluster.functions |
ClusterFunctions |
Specify a specific
cluster backend using on of the constructors provided by
BatchJobs, see
ClusterFunctions.
Default is NULL where the default cluster functions
defined in the configuration are used. Ignored when
conf.pars is provided.
progressbar |
|
Suppress the progress bar used in BatchJobs and be less verbose.
Default is FALSE.
jobname |
|
Job name that is prepended to the output log and result files. Default is "BPJOB".
reg.pars |
|
List of parameters passed to BatchJobs::makeRegistry(). When
present, user-supplied arguments seed and work.dir
to BatchJobsParam are ignored.
conf.pars |
|
List of parameters passed to BatchJobs::setConfig(). When
present, user-supplied arguments conffile,
cluster.functions to BatchJobsParam are ignored.
submit.pars |
|
List of parameters passed to BatchJobs::submitJobs. When
present, user-supplied argument resources to
BatchJobsParam is ignored. submitJobs parameters
reg, id cannot be set.
... |
Addition arguments, currently not handled. |
Return an object with specified values. The object may be saved to disk or reused within a session.
The following generics are implemented and perform as documented on
the corresponding help page: bpworkers,
bpnworkers, bpstart,
bpstop, bpisup, bpbackend,
bpbackend<-
Michel Lang, mailto:michellang@gmail.com
getClass("BiocParallelParam") for additional parameter classes.
register for registering parameter classes for use in parallel
evaluation.
p <- BatchJobsParam(progressbar=FALSE)
bplapply(1:10, sqrt, BPPARAM=p)
## Not run:
## see vignette for additional explanation
funs <- makeClusterFunctionsSLURM("~/slurm.tmpl")
param <- BatchJobsParam(4, cluster.functions=funs)
register(param)
bplapply(1:10, function(i) sqrt)
## End(Not run)Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.