代码之家 › 专栏 › 技术社区 › mastefan

运行覆盖时的光栅群集错误

r-raster parallel-processing r

mastefan · 技术社区 · 7 年前

我正在尝试编写一个函数,以便在光栅上运行几个并行的单元计算我希望使用光栅包的内置并行处理代码来加快处理速度。

我得到一个错误:“get(name,envir=envir)错误:找不到对象'mn'此错误发生在下面的“错误”标记处,函数正在该标记处尝试运行raster::overlay。

我看到这个错误来自get函数,并指出mn对象不是所有集群都可以访问的。

如何编写此代码以使对象处于适当的环境中谢谢。

下面是我的代码的基本版本:

my_func <- function(file = NULL, na.rm = TRUE, ncores = 2){
    # begin parallel processing
    raster::beginCluster(ncores, type='SOCK')

    # load data from file as raster brick
    data <- raster::brick(file)

    # calculate the standard deviation of the brick
    sd <- raster::clusterR(x = data, fun = function(data){ raster::calc(x = data, fun = sd, na.rm = na.rm) })

    # calculate mean - for layer i, mean is the mean of all layers but i
    set <- 1:raster::nbands(data) # marker to pass to for loop
    mn <- data # creates raster object to be filled with results
    for(i in set){
      subset <- set[-i]
      wrk <- raster::stack(x = data, bands = subset) # make stack of all but i
      mn_i <- raster::clusterR(x = wrk, fun = function(wrk){ raster::calc(x = wrk, fun = mean, na.rm = na.rm) }) # calculate mean of stack
      mn[[i]] <- mn_i # set values of appropriate band with results
    }

    # calculate z score
    ##### ERROR HERE #####
    z <- raster::clusterR(x = data, fun = function(data, mn){ raster::overlay(x = data, y = mn, fun = function(data, mn){ data - mn }, na.rm = na.rm }, export = mn)

    # normalize z score
    ##### I assume the error would also occur here #####
    z <- raster::clusterR(x = data, fun = function(z, sd){ raster::overlay(x = z, y = sd, fun = function(z, sd){ z / sd }, na.rm = na.rm) }, export = sd)

    # end parallel processing
    raster::endCluster()


    # return result
    return(result)

    }

2 回复 | 直到 7 年前

aldo_tapia 7 年前

你在设置 x 和 y 在里面 overlay() 但是使用 data , mn , z 和 sd 作为参数。使用:

z <- raster::clusterR(x = data, fun = function(data, mn){ raster::overlay(x = data, y = mn, fun = function(x, y){ x - y }, na.rm = na.rm }, export = mn)

z <- raster::clusterR(x = data, fun = function(z, sd){ raster::overlay(x = z, y = sd, fun = function(x, y){ x / y }, na.rm = na.rm) }, export = sd)

提议 :(始终提供示例数据)

library(raster)

set.seed(123)

r <- raster()
r[] <- 1:ncell(r)

data <- r
mn <- setValues(r , rnorm(ncell(r)))
data[1:1000] <- NA # to force some NA testing

sd <- r*2

fun1 <- function(x,y){x - y}
fun2 <- function(x,y){x / y}

beginCluster()
z <- clusterR(stack(data,mn), overlay, arg = list(fun = fun1))
z <- clusterR(stack(z,sd), overlay, arg = list(fun = fun2))
endCluster()

plot(z)

mastefan 7 年前

要使用多波段光栅,可以将cluster函数滚动到for循环中。

    # example data
    data("Rlogo")
    data <- raster::brick(Rlogo)

    # calculate sd
    sd <- raster::calc(data, sd)

    # calculate mean (mean of layer x is mean of all layers but x)
    mn <- data
    set <- 1:nlayers(data)
    for (i in set) {
      subset <- set[-i]
      wrk <- raster::stack(x = data, bands = subset)
      mn_i <- raster::calc(x = wrk, fun = mean)
      mn[[i]] <- mn_i
    }

    # calculate z score
    z <- data
    set <- 1:nlayers(data)
    fun1 <- function(x,y){x-y}
    raster::beginCluster()
    for (i in set){
      wrk <- raster::stack(data[[i]], mn[[i]])
      z_i <- clusterR(x = wrk, fun = raster::overlay, arg = list(fun = fun1))
      z[[i]] <- z_i
      rm(z_i, wrk)
    }

    # normalize z score
    fun2 <- function(x,y){x/y}
    for(i in set){
      wrk <- raster::stack(z[[i]], sd)
      z_i <- clusterR(x = wrk, fun = raster::overlay, arg = list(fun = fun2))
      z[[i]] <- z_i
      rm(z_i, wrk)
    }
    raster::endCluster()