numpy阵列变换的性能改进

你好像在试着摇一扇窗 (2, 3) 在二维阵列中。这与 convolution of image 这可以通过 np.lib.stride_tricks 以一种非常有效的方式。

a = np.transpose([Xd, Yd, Zd])
window = (2, 3)
view_shape = (len(a) - window[0] + 1,) + window # (4,2,3) if len(a) == 5
sub_matrix = np.lib.stride_tricks.as_strided(a, shape = view_shape, strides = (a.itemsize,) + a.strides)
>>> sub_matrix
array([[[0. , 0. , 0. ],
        [0. , 0. , 1.5]],

       [[0. , 0. , 1.5],
        [1. , 0. , 1.5]],

       [[1. , 0. , 1.5],
        [1. , 2.5, 1.5]],

       [[1. , 2.5, 1.5],
        [0.5, 2.5, 1.5]]])

请注意 np.lib.u技巧 对任何其他方法都很有效。

一般的建议是,当您想要提高速度时,通常应该尽量避免堆栈和连接,因为这通常意味着要多次复制相同的数据。

不管怎样,这里是我应该怎么做的,稍微长一点的代码,但不会做比需要更多的工作

n = len(Xd)
segments = np.empty((n-1, 2, 3))

segments[:,0,0] = Xd[:-1]
segments[:,1,0] = Xd[1:]

segments[:,0,1] = Yd[:-1]
segments[:,1,1] = Yd[1:]

segments[:,0,2] = Zd[:-1]
segments[:,1,2] = Zd[1:]

[编辑]-以下内容是为了科学/娱乐,请勿复制

所以我试着看看我是否能从中挤出更多的表演 @mathfux

a = np.empty(3*n)
a[:n]    = Xd
a[n:n+n] = Yd
a[n+n:]  = Zd

interface = dict(a.__array_interface__)
interface['shape'] = (n-1, 2, 3)
interface['strides'] = (a.itemsize, a.itemsize, n*a.itemsize)
segments= np.array(np.lib.stride_tricks.DummyArray(interface, base=a), copy=False)

在我的机器上,速度明显更快(根据输入的大小高达30%)。收益的部分原因是建造了 a 跳过支票 as_strided

下面是一些在更大的阵列上进行的计时测试,这使得差别更为明显。

import numpy as np
from timeit import timeit

# original
def f1(x, y, z):
    points = np.stack([x, y, z], axis=1).reshape(-1, 1, 3)
    return np.concatenate([points[:-1], points[1:]], axis = 1)

# preallocating and then assigning
def f2(x, y, z):
    segments = np.empty((len(x)-1, 2, 3))

    segments[:,0,0] = x[:-1]
    segments[:,1,0] = x[1:]

    segments[:,0,1] = y[:-1]
    segments[:,1,1] = y[1:]

    segments[:,0,2] = z[:-1]
    segments[:,1,2] = z[1:]
    return segments

# stacking, but in one go
def f3(x, y, z):
    segments = np.stack([x[:-1], y[:-1], z[:-1], x[1:], y[1:],z[1:]], axis=1)
    return segments.reshape(-1, 2, 3)

# list comparison
def f4(x, y, z):
    z_ = [i for i in zip(x,y,z)]
    return [[[z_[i]],[z_[i+1]]] for i in range(len(z_)-1)]

#np.lib.stride_tricks approach
def f5(x, y, z):
    a = np.transpose([x, y, z])
    window = (2, 3)
    view_shape = (len(a) - window[0] + 1,) + window # (4,2,3) if len(a) == 5
    return np.lib.stride_tricks.as_strided(a, shape = view_shape, strides = (a.itemsize,) + a.strides)
    

ntime = 5000 #number of test runs
nxd = 500    #array length

Xd = np.random.randn(nxd)
Yd = np.random.randn(nxd)
Zd = np.random.randn(nxd)

print(timeit(lambda: f1(Xd, Yd, Zd), number=ntime))
#0.11369249999999999

print(timeit(lambda: f2(Xd, Yd, Zd), number=ntime))
#0.0480651

print(timeit(lambda: f3(Xd, Yd, Zd), number=ntime))
#0.10202380000000003

print(timeit(lambda: f4(Xd, Yd, Zd), number=ntime))
#1.8407391

print(timeit(lambda: f5(Xd, Yd, Zd), number=ntime))
#0.09132560000000023
    
ntime = 50     #number of test runs
nxd = 500000   #array length

Xd = np.random.randn(nxd)
Yd = np.random.randn(nxd)
Zd = np.random.randn(nxd)

print(timeit(lambda: f1(Xd, Yd, Zd), number=ntime))
#1.7519548999999999

print(timeit(lambda: f2(Xd, Yd, Zd), number=ntime))
#1.504727

print(timeit(lambda: f3(Xd, Yd, Zd), number=ntime))
#1.5010566

print(timeit(lambda: f4(Xd, Yd, Zd), number=ntime))
#22.6208157

print(timeit(lambda: f5(Xd, Yd, Zd), number=ntime))
#0.46465339999999955

如您所见,@Miguel的方法就是这样:预先分配数组,然后分配是最有效的方法。即使您以更智能的方式(如在f3()中)堆叠它们,它仍然比f2()慢。但是当数组长度大幅增加时,没有什么比f5()更好的了。

我发现这比@Miguel的代码快。

z = [i for i in zip(Xd,Yd,Zd)]
segments = [[[z[i]],[z[i+1]]] for i in range(len(z)-1)]