代码之家  ›  专栏  ›  技术社区  ›  dwiz

R中具有焦函数和parlappy的光栅并行处理列表

  •  1
  • dwiz  · 技术社区  · 7 年前

    我试图并行处理光栅列表,并使用Parlappy执行焦点函数。我想我误解了一些至关重要的事情。代码运行了,但看起来它没有在我的驱动器上正确地写出焦点函数。看起来它在列表中的第一个光栅上执行了两次密度_函数。。。。刚接触并行处理世界,想知道是否有任何关于如何处理这个问题的建议?请注意,当我使用lappy运行Density_函数和list时,它是有效的。如何并行处理?

    `# Density function
    Density_Function <- function (raster_layer){
                        weight <- focalWeight(raster_layer,90,type = "circle")                        
                        raster_name <- names(raster_layer)
                        short_name <- substr(raster_name,1,4)
                        half_output <- "X:/Path"
                        full_output <- paste0(half_output,short_name,"_90m.tif")
                        focal(raster_layer, weight, fun=sum, full_output, na.rm=TRUE, pad=TRUE, NAonly=FALSE, overwrite=TRUE)         
                        }
    
    #Bring in raster data and create list
    roads_raster <-raster('X:/roads.tif')
    pipe_raster <-raster('X:/pipes.tif')
    raster_list <- list(roads_raster,pipe_raster) ` 
    
    #Activate cluster
    no_cores <- detectCores() - 1
    cl <- makeCluster(no_cores)
    
    #Apply function
    parLapply(cl = cl, x = raster_list, fun = Density_Function)
    
    #Close cluster
    stopCluster(cl)
    
    1 回复  |  直到 7 年前
        1
  •  1
  •   dwiz    7 年前

    我采取了不同的方法,但最终得到了我想要的。我没有使用parlappy,而是使用foreach循环遍历光栅列表,并行执行密度函数。

    这个博客真的很有帮助: http://www.gis-blog.com/increasing-the-speed-of-raster-processing-with-r-part-23-parallelisation/

    library(doParallel)   
    library(foreach)
    
    #Density function, 1km circular radius
    Density_Function_1000 <- function (raster_layer){
      raster_name <- names(raster_layer)
      short_name <- substr(raster_name,1,4)
      weight <- focalWeight(raster_layer,1000,type = "circle")
      half_output <- "X:/Path"
      full_output <- paste0(half_output,short_name,"_1km.tif")
      focal(raster_layer, weight, fun=sum, full_output, na.rm=TRUE, pad=TRUE, NAonly=FALSE, overwrite=TRUE)
      }
    
    #Define how many cores you want to use
    UseCores <- detectCores() -1
    #Register CoreCluster
    cl <- makeCluster(UseCores)
    registerDoParallel(cl)
    
    #Create my list of rasters
    raster_list <- list(roads_raster, cuts_raster, wells_raster, seis_raster, pipes_raster, fires_raster)
    
    #Use foreach loop and %dopar% command to execute my density function in parallel 
    foreach(i = raster_list) %dopar% {
      library(raster)
      Density_Function_1000(i)
      }
    
    #end cluster
    stopCluster(cl)