代码之家 › 专栏 › 技术社区 › Brad Solomon

存储临时Python引用的不安全C派生

cython

1

Brad Solomon · 技术社区 · 7 年前

考虑以下设计的Cython函数来加入字符串列表:

# cython: language_level=3
cpdef test_join():
    """ ["abc", "def", "ghi"] -> "abcdefghi" """
    cdef:
        list lines = ["abc", "def", "ghi"]
        char* out = ""
        char* line = ""
        int i
    for i in range(len(lines)):
        line = lines[i]
        out = out + line
    return out

它将无法编译,出现以下错误:

我想这与 line char* 并不断地重新分配。我看到了一个问题的答案 similar question

如何修改上述函数以按预期编译和返回?

更广泛地说,我想更好地理解这个错误。犯罪 37e4a20 有一点解释:

采取行动 煤焦*

:为了进一步简化,似乎是分配导致了问题:

cpdef int will_succeed():
    cdef char* a = b"hello"
    cdef char* b = b" world"
    print(a + b)  # no new assignment
    return 1

cpdef will_fail():
    cdef char* a = b"hello"
    cdef char* b = b" world"
    a = a + b  # won't compile
    return a

我想可能有一种更合适的方法来处理来自 string.pxd string.h ,但我在C内存管理和效率方面相当薄弱:

from libc.string cimport strcat, strcpy

cpdef use_strcat():
    cdef char out[1024]
    strcpy(out, b"")

    cdef char* a = b"hello"
    cdef char* b = b" world"

    strcat(out, a)
    strcat(out, b)
    return out

1 回复 | 直到 7 年前

1

3

Brad Solomon 7 年前

我认为问题出在你身上

out = out + line

Cython没有定义运算符 + 对于C字符串。相反,它将它们转换为Python字符串,并连接这些字符串:

tmp1 = str(out)
tmp2 = str(line)
tmp3 = tmp1 + tmp2
out = get_c_string_from(tmp3)

out 因此,只要 tmp3 被摧毁(立即被摧毁)。

我会避免使用 strcat not very efficient for repeated uses . 而是跟踪当前字符串长度,并自己复制数据。假设您有一个未知的长度,您可能希望分配字符串 malloc

from libc.stdlib cimport free, malloc, realloc
from libc.string cimport memcpy

from cython import Py_ssize_t

cdef char         *line
cdef Py_ssize_t   i
cdef Py_ssize_t   length = 0
cdef Py_ssize_t   incrlength
cdef char         *out = <char *>malloc(1)  # Reallocate as needed

try:
    out[0] = b'\x00' # keep C-strings null-terminated
    for i in range(len(lines)):
        line = lines[i]
        incrlength = len(line)
        out = <char *>realloc(out, length + incrlength + 1)
        memcpy(out + length, line, incrlength)
        length += incrlength
        out[length] = '\x00'  # keep C-strings null-terminated
    return out  # autoconversion back to a Python string

finally:
   free(out)

这是我认为你应该做的事情的大致轮廓,并没有经过真正的测试。