代码之家  ›  专栏  ›  技术社区  ›  Roger Allen

如何让每个CPU核心可变访问Vec的一部分?[副本]

  •  1
  • Roger Allen  · 技术社区  · 8 年前

    我有一个令人尴尬的并行图形渲染代码,我想在我的CPU内核上运行。我编写了一个测试用例(计算出的函数毫无意义),以探索如何将其并行化。我想使用std Rust对此进行编码,以便了解如何使用 std::thread 。但是,我不知道如何给每个线程分配一部分帧缓冲区。我将在下面列出完整的测试用例代码,但我将首先尝试将其分解。

    顺序形式非常简单:

    let mut buffer0 = vec![vec![0i32; WIDTH]; HEIGHT];
    for j in 0..HEIGHT {
        for i in 0..WIDTH {
            buffer0[j][i] = compute(i as i32,j as i32);
        }
    }
    

    我认为制作一个大小相同的缓冲区会有所帮助,但会被重新安排为3D&先按核心索引。这是相同的计算,只是对数据进行重新排序以显示加工。

    let mut buffer1 = vec![vec![vec![0i32; WIDTH]; y_per_core]; num_logical_cores];
    for c in 0..num_logical_cores {
        for y in 0..y_per_core {
            let j = y*num_logical_cores + c;
            if j >= HEIGHT {
                break;
            }
            for i in 0..WIDTH {
                buffer1[c][y][i] = compute(i as i32,j as i32)
            }
        }
    }
    

    但是,当我试图将代码的内部部分放入闭包时&创建一个线程,我会得到有关缓冲区的错误;生命周期。我基本上不知道该怎么办&可能需要一些指导。我想要 per_core_buffer 暂时参考中的数据 buffer2 属于该核心&允许写入,同步所有线程&然后阅读 缓冲区2 之后这可能吗?

    let mut buffer2 = vec![vec![vec![0i32; WIDTH]; y_per_core]; num_logical_cores];
    let mut handles = Vec::new();
    for c in 0..num_logical_cores {
        let per_core_buffer = &mut buffer2[c]; // <<< lifetime error
        let handle = thread::spawn(move || {
            for y in 0..y_per_core {
                let j = y*num_logical_cores + c;
                if j >= HEIGHT {
                    break;
                }
                for i in 0..WIDTH {
                    per_core_buffer[y][i] = compute(i as i32,j as i32)
                }
            }
        });
        handles.push(handle)
    }
    for handle in handles {
        handle.join().unwrap();
    }
    

    错误是(&我不明白:

    error[E0597]: `buffer2` does not live long enough
      --> src/main.rs:50:36
       |
    50 |         let per_core_buffer = &mut buffer2[c]; // <<< lifetime error
       |                                    ^^^^^^^ borrowed value does not live long enough
    ...
    88 | }
       | - borrowed value only lives until here
       |
       = note: borrowed value must be valid for the static lifetime...
    

    完整的测试用例是:

    extern crate num_cpus;
    use std::time::Instant;
    use std::thread;
    
    fn compute(x: i32, y: i32) -> i32 {
        (x*y) % (x+y+10000)
    }
    
    fn main() {
        let num_logical_cores = num_cpus::get();
        const WIDTH: usize = 40000;
        const HEIGHT: usize = 10000;
        let y_per_core = HEIGHT/num_logical_cores + 1;
    
        // ------------------------------------------------------------
        // Serial Calculation...
        let mut buffer0 = vec![vec![0i32; WIDTH]; HEIGHT];
        let start0 = Instant::now();
        for j in 0..HEIGHT {
            for i in 0..WIDTH {
                buffer0[j][i] = compute(i as i32,j as i32);
            }
        }
        let dur0 = start0.elapsed();
    
        // ------------------------------------------------------------
        // On the way to Parallel Calculation...
        // Reorder the data buffer to be 3D with one 2D region per core.
        let mut buffer1 = vec![vec![vec![0i32; WIDTH]; y_per_core]; num_logical_cores];
        let start1 = Instant::now();
        for c in 0..num_logical_cores {
            for y in 0..y_per_core {
                let j = y*num_logical_cores + c;
                if j >= HEIGHT {
                    break;
                }
                for i in 0..WIDTH {
                    buffer1[c][y][i] = compute(i as i32,j as i32)
                }
            }
        }
        let dur1 = start1.elapsed();
    
        // ------------------------------------------------------------
        // Actual Parallel Calculation...
        let mut buffer2 = vec![vec![vec![0i32; WIDTH]; y_per_core]; num_logical_cores];
        let mut handles = Vec::new();
        let start2 = Instant::now();
        for c in 0..num_logical_cores {
            let per_core_buffer = &mut buffer2[c]; // <<< lifetime error
            let handle = thread::spawn(move || {
                for y in 0..y_per_core {
                    let j = y*num_logical_cores + c;
                    if j >= HEIGHT {
                        break;
                    }
                    for i in 0..WIDTH {
                        per_core_buffer[y][i] = compute(i as i32,j as i32)
                    }
                }
            });
            handles.push(handle)
        }
        for handle in handles {
            handle.join().unwrap();
        }
        let dur2 = start2.elapsed();
    
        println!("Runtime: Serial={0:.3}ms, AlmostParallel={1:.3}ms, Parallel={2:.3}ms",
                 1000.*dur0.as_secs() as f64 + 1e-6*(dur0.subsec_nanos() as f64),
                 1000.*dur1.as_secs() as f64 + 1e-6*(dur1.subsec_nanos() as f64),
                 1000.*dur2.as_secs() as f64 + 1e-6*(dur2.subsec_nanos() as f64));
    
        // Sanity check
        for j in 0..HEIGHT {
            let c = j % num_logical_cores;
            let y = j / num_logical_cores;
            for i in 0..WIDTH {
                if buffer0[j][i] != buffer1[c][y][i] {
                    println!("wtf1? {0} {1} {2} {3}",i,j,buffer0[j][i],buffer1[c][y][i])
                }
                if buffer0[j][i] != buffer2[c][y][i] {
                    println!("wtf2? {0} {1} {2} {3}",i,j,buffer0[j][i],buffer2[c][y][i])
                }
            }
        }
    
    }
    
    1 回复  |  直到 8 年前
        1
  •  0
  •   Roger Allen    8 年前

    感谢@Shepmaster的指点和澄清,这对于生锈来说不是一个容易的问题,我需要考虑板条箱来找到合理的解决方案。我才刚开始生锈,所以我真的不清楚。

    我喜欢控制线程数量的能力 scoped_threadpool 给,所以我同意了。直接从上面翻译我的代码,我尝试使用4D缓冲区,核心作为最重要的索引,但由于3D向量没有实现 Copy 特质它实现的事实 复制 让我担心性能,但我回到了最初的问题,并更直接地实现了它&通过将每一行设为线程,找到了合理的加速。复制每一行不会带来很大的内存开销。

    对我有效的代码是:

    let mut buffer2 = vec![vec![0i32; WIDTH]; HEIGHT];
    let mut pool = Pool::new(num_logical_cores as u32);
    pool.scoped(|scope| {
        let mut y = 0;
        for e in &mut buffer2 {
            scope.execute(move || {
                for x in 0..WIDTH {
                    (*e)[x] = compute(x as i32,y as i32);
                }
            });
            y += 1;
        }
    });
    

    对于400000x4000测试用例,在6核、12线程i7-8700K上,此操作连续运行3.2秒;481ms并行——合理的加速。

    编辑:我继续思考这个问题,并在推特上得到了Rustlang的建议,我应该考虑 rayon 。我将代码转换为 人造丝 并通过以下代码获得了类似的加速。

    let mut buffer2 = vec![vec![0i32; WIDTH]; HEIGHT];
    buffer2
        .par_iter_mut()
        .enumerate()
        .map(|(y,e): (usize, &mut Vec<i32>)| {
            for x in 0..WIDTH {
                (*e)[x] = compute(x as i32,y as i32);
            }
        })
        .collect::<Vec<_>>();
    
    推荐文章