secr: Parallel – R documentation

Become an expert in R — Interactive courses, Cheat Sheets, certificates and more!

Parallel

Multi-core Processing

Description

From version 4.0 secr uses multi-threading in C++ (package RcppParallel, Allaire et al. 2019) to speed likelihood evaluation and hence model fitting in secr.fit. Detection histories are distributed over threads. Setting ncores = NULL in functions with multi-threading uses the existing value from the environment variable RCPP_PARALLEL_NUM_THREADS (see setNumThreads).

These functions use multi-threading and call setNumThreads internally:

Function

autoini

confint.secr

derivedSystematic

esa

fxi.secr and related functions

pdot

region.N

score.test

secr.fit

NOTE: The mechanism for setting the number of threads changed between versions 4.1.0 and 4.2.0. The default number of cores is now capped at 2 to meet CRAN requirements. Setting ncores = NULL previously specified one less than the maximum number of cores.

Earlier versions of secr made more limited use of multiple cores (CPUs) through the package parallel. This mechanism is still available in the functions listed here, but the speed gains are often small or even negative. Set ncores > 1 in the function call to use multiple cores.

Function	Unit	Benefit	Notes
`ip.secr`	replicate	large
`par.secr.fit`	model	none
`par.derived`	fitted model	none
`par.region.N`	fitted model	none

‘Unit’ refers to the unit of work sent to each worker process. As a guide, a ‘large’ benefit means >60% reduction in process time with 4 CPUs.

parallel offers several different mechanisms, bringing together the functionality of multicore and snow. The mechanism used by secr is the simplest available, and is expected to work across all operating systems. Technically, it relies on Rscript and communication between the master and worker processes is via sockets. As stated in the parallel documentation "Users of Windows and Mac OS X may expect pop-up dialog boxes from the firewall asking if an R process should accept incoming connections". You may possibly get warnings from R about closing unused connections. These can safely be ignored.

Use parallel::detectCores() to get an idea of how many cores are available on your machine; this may (in Windows) include virtual cores over and above the number of physical cores. See RShowDoc("parallel", package = "parallel") in core R for explanation.

In secr.fit the output component ‘proctime’ misrepresents the elapsed processing time when multiple cores are used.

Warning

It appears that multicore operations in secr using parallel may fail if the packages snow and snowfall are also loaded. The error message is obscure:

“Error in UseMethod("sendData") : no applicable method for 'sendData' applied to an object of class "SOCK0node"”

References

Allaire, J. J., Francois, R., Ushey, K., Vandenbrouck, G., Geelnard, M. and Intel (2019) RcppParallel: Parallel Programming Tools for 'Rcpp'. R package version 4.4.4. https://CRAN.R-project.org/package=RcppParallel.

Examples

## Not run: 

sessionInfo()
# R version 3.6.1 (2019-07-05)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows 7 x64 (build 7601) Service Pack 1
# quad-core i7 CPU, 16 Gb RAM
# ...

## benefit from multi-threading in secr.fit

for (i in 1:8) 
     print(system.time(secr.fit(ovenCH, buffer = 400, trace = FALSE, ncores = i)))
     
#   user  system elapsed 
#  54.25    0.17   54.43 
#   user  system elapsed 
#  29.48    0.16   20.30 
#   user  system elapsed 
#  43.34    0.17   25.04 
#   user  system elapsed 
#  43.92    0.14   22.95 
#   user  system elapsed 
#  46.16    0.22   22.70 
#   user  system elapsed 
#  31.59    0.17   15.19 
#   user  system elapsed 
#  45.58    0.12   21.93 
#   user  system elapsed 
#  37.43    0.15   18.43 

## and for simulation...
 
for (i in 1:8)
     print(system.time(sim.secr(secrdemo.0, nsim = 20, tracelevel = 0, ncores = i)))
     
#   user  system elapsed 
# 160.68    0.78  161.93 
#   user  system elapsed 
# 158.48    0.56   89.42 
#   user  system elapsed 
# 158.54    0.43   65.05 
#   user  system elapsed 
# 165.00    0.39   55.06 
#   user  system elapsed 
# 171.38    0.53   48.03 
#   user  system elapsed 
# 191.03    0.47   48.14 
#   user  system elapsed 
# 184.46    0.52   43.42 
#   user  system elapsed 
# 193.07    0.56   42.34 

for (i in 1:8) 
    print(system.time(ip.secr (captdata, trace = FALSE, ncores = i)))

#   user  system elapsed 
# 121.88    0.08  122.27 
#   user  system elapsed 
#   0.54    0.42   72.85 
#   user  system elapsed 
#   0.55    0.76   55.91 
#   user  system elapsed 
#   0.91    0.77   47.65 
#   user  system elapsed 
#   1.21    0.81   44.83 
#   user  system elapsed 
#   1.42    1.23   43.21 
#   user  system elapsed 
#   1.18    1.98   42.46 
#   user  system elapsed 
#   1.81    1.81   42.54  
 
for (i in 1:8)
    print(system.time(LLsurface(secrdemo.0, ncores = i)))

# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   26.59    0.14   26.80 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   25.74    0.13   17.73 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   26.08    0.18   15.12 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   26.93    0.19   13.82 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   28.32    0.06   13.52 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   29.40    0.16   13.05 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   29.76    0.22   12.92 
# Evaluating log likelihood across grid of 121 points...
#    user  system elapsed 
#   29.10    0.20   12.81 


## End(Not run)

secr

Spatially Explicit Capture-Recapture

v4.4.1

GPL (>= 2)

Authors

Murray Efford

Initial release

2021-05-01