代码之家 › 专栏 › 技术社区 › Jiadong

如何加速数百万对象的python实例初始化?

instance performance python

Jiadong · 技术社区 · 6 年前

我定义了一条巨蟒 class 命名 Edge 如下:

class Edge:
    def __init__(self):
        self.node1 = 0
        self.node2 = 0
        self.weight = 0

现在我必须创建大约10^6到10^7个边缘实例,使用:

edges= []
for (i,j,w) in ijw:
    edge = Edge()
    edge.node1 = i
    edge.node2 = j
    edge.weight = w
    edges.append(edge)

我在桌面上花了大约2秒钟。有更快的方法吗?

2 回复 | 直到 6 年前

Martijn Pieters 6 年前

你做不到 许多的 更快,但我肯定会用 __slots__ 以节省内存分配。还可以在创建实例时传递属性值:

class Edge:
    __slots__ = ('node1', 'node2', 'weight')
    def __init__(self, node1=0, node2=0, weight=0):
        self.node1 = node1
        self.node2 = node2
        self.weight = weight

更新了 __init__ 您可以使用列表理解:

edges = [Edge(*args) for args in ijw]

这些技术加在一起可以节省大量时间来创建对象,大约将所需时间减半。

比较创建100万个对象;设置:

>>> from random import randrange
>>> ijw = [(randrange(100), randrange(100), randrange(1000)) for _ in range(10 ** 6)]
>>> class OrigEdge:
...     def __init__(self):
...         self.node1 = 0
...         self.node2 = 0
...         self.weight = 0
...
>>> origloop = '''\
... edges= []
... for (i,j,w) in ijw:
...     edge = Edge()
...     edge.node1 = i
...     edge.node2 = j
...     edge.weight = w
...     edges.append(edge)
... '''
>>> class SlotsEdge:
...     __slots__ = ('node1', 'node2', 'weight')
...     def __init__(self, node1=0, node2=0, weight=0):
...         self.node1 = node1
...         self.node2 = node2
...         self.weight = weight
...
>>> listcomploop = '''[Edge(*args) for args in ijw]'''

时间安排:

>>> from timeit import Timer
>>> count, total = Timer(origloop, 'from __main__ import OrigEdge as Edge, ijw').autorange()
>>> (total / count) * 1000 # milliseconds
722.1121070033405
>>> count, total = Timer(listcomploop, 'from __main__ import SlotsEdge as Edge, ijw').autorange()
>>> (total / count) * 1000 # milliseconds
386.6706900007557

快了近2倍。

将随机输入列表增加到10^7项,时间差保持:

>>> ijw = [(randrange(100), randrange(100), randrange(1000)) for _ in range(10 ** 7)]
>>> count, total = Timer(origloop, 'from __main__ import OrigEdge as Edge, ijw').autorange()
>>> (total / count)
7.183759553998243
>>> count, total = Timer(listcomploop, 'from __main__ import SlotsEdge as Edge, ijw').autorange()
>>> (total / count)
3.8709938440006226

Hannes Ovrén 6 年前

另一种选择是跳过 Edge 类并通过表或邻接矩阵实现边。

例如。

A = create_adjacency_graph(ijw)  # Implement to return a IxJ (sparse?) matrix of weights
edge_a_weight = A[3, 56]
edge_b_weight = A[670, 1023]
# etc...

不过,这确实消除了一些灵活性,但在创建和使用方面应该相当快。