BatchJobs: batchReduceResults – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

batchReduceResults

Reduces results via a binary function and adds jobs for this to a registry.

Description

Each jobs reduces a certain number of results on one slave. You can then submit these jobs to the batch system. Later, you can do a final reduction with reduceResults on the master.

Usage

batchReduceResults(reg, reg2, fun, ids, part = NA_character_, init,
  block.size, more.args = list())

Arguments

`reg`	[`Registry`] Registry whose results should be reduced by `fun`.
`reg2`	[`Registry`] Empty registry that should store the job for the mapping.
`fun`	[`function(aggr, job, res, ...)`] Function to reduce results with.
`ids`	[`integer`] Ids of jobs whose results should be reduced with `fun`. Default is all jobs.
`part`	[`character`] Only useful for multiple result files, then defines which result file part(s) should be loaded. `NA` means all parts are loaded, which is the default.
`init`	[any] Initial object for reducing.
`block.size`	[`integer(1)`] Number of results reduced in one job.
`more.args`	[`list`] A list of other arguments passed to `fun`. Default is empty list.

Value

Vector of type integer with job ids.

Examples

# generating example results:
reg1 = makeRegistry(id = "BatchJobsExample1", file.dir = tempfile(), seed = 123)
f = function(x) x^2
batchMap(reg1, f, 1:20)
submitJobs(reg1)
waitForJobs(reg1)

# define function to reduce on slave, we want to sum the squares
myreduce = function(aggr, job, res) aggr + res

# sum 5 results on each slave process, i.e. 4 jobs
reg2 = makeRegistry(id = "BatchJobsExample2", file.dir = tempfile(), seed = 123)
batchReduceResults(reg1, reg2, fun = myreduce, init = 0, block.size = 5)
submitJobs(reg2)
waitForJobs(reg2)

# now reduce one final time on master
reduceResults(reg2, fun = myreduce)

BatchJobs

Batch Computing with R

v1.8

BSD_2_clause + file LICENSE

Authors

Bernd Bischl <bernd_bischl@gmx.net>, Michel Lang <michellang@gmail.com>, Henrik Bengtsson <henrikb@braju.com>

Initial release