Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!
Get Started for Free

batchReduceResults

Reduces results via a binary function and adds jobs for this to a registry.


Description

Each jobs reduces a certain number of results on one slave. You can then submit these jobs to the batch system. Later, you can do a final reduction with reduceResults on the master.

Usage

batchReduceResults(reg, reg2, fun, ids, part = NA_character_, init,
  block.size, more.args = list())

Arguments

reg

[Registry]
Registry whose results should be reduced by fun.

reg2

[Registry]
Empty registry that should store the job for the mapping.

fun

[function(aggr, job, res, ...)]
Function to reduce results with.

ids

[integer]
Ids of jobs whose results should be reduced with fun. Default is all jobs.

part

[character] Only useful for multiple result files, then defines which result file part(s) should be loaded. NA means all parts are loaded, which is the default.

init

[any]
Initial object for reducing.

block.size

[integer(1)]
Number of results reduced in one job.

more.args

[list]
A list of other arguments passed to fun. Default is empty list.

Value

Vector of type integer with job ids.

Examples

# generating example results:
reg1 = makeRegistry(id = "BatchJobsExample1", file.dir = tempfile(), seed = 123)
f = function(x) x^2
batchMap(reg1, f, 1:20)
submitJobs(reg1)
waitForJobs(reg1)

# define function to reduce on slave, we want to sum the squares
myreduce = function(aggr, job, res) aggr + res

# sum 5 results on each slave process, i.e. 4 jobs
reg2 = makeRegistry(id = "BatchJobsExample2", file.dir = tempfile(), seed = 123)
batchReduceResults(reg1, reg2, fun = myreduce, init = 0, block.size = 5)
submitJobs(reg2)
waitForJobs(reg2)

# now reduce one final time on master
reduceResults(reg2, fun = myreduce)

BatchJobs

Batch Computing with R

v1.8
BSD_2_clause + file LICENSE
Authors
Bernd Bischl <bernd_bischl@gmx.net>, Michel Lang <michellang@gmail.com>, Henrik Bengtsson <henrikb@braju.com>
Initial release

We don't support your browser anymore

Please choose more modern alternatives, such as Google Chrome or Mozilla Firefox.