我一直认为python列表理解并没有隐式地利用多处理和阅读堆栈上的问题(例如。
this one
)也给我同样的印象。然而,以下是我的小实验:
import numpy as np
import time
# some arbitrary data
n = 1000
p = 5
X = np.block([[np.eye(p)], [np.zeros((n-p, p))]])
y = np.sum(X, axis=1) + np.random.normal(0, 1, (n, ))
n_loop = 100000
# run linear regression using direct matrix algebra
def in_sample_error_algebra(X, y):
beta_hat = np.linalg.inv(X.transpose()@X)@(X.transpose()@y)
y_hat = X@beta_hat
error = metrics.mean_squared_error(y, y_hat)
return error
start = time.time()
errors = [in_sample_error_algebra(X, y) for _ in range(n_loop)]
print('run time =', round(time.time() - start, 2), 'seconds')
当这段代码运行时,我的CPU的所有6个(物理)内核都达到了100%
更神奇的是,当我从列表理解改为for循环时,同样的事情发生了。我想是因为
.append
start = time.time()
errors = []
for _ in range(n_loop):
errors.append(in_sample_error_algebra(X, y))
print('run time =', round(time.time() - start, 2), 'seconds')
运行时间=21.29秒
有什么理论吗?
Python 3.7.2, numpy 1.15.4
