具有未来的Dask计算子图

1. 使用dask构建延迟dask图。袋子( def fakejob )
2. 从1开始计算图形,并将其保存到parquet(去掉这一部分,只是一个动机)

from dask.distributed import Client

client = Client(processes=True)

def fakejob(path):
    return (
        dask.bag
        .read_text(path)
        .to_dataframe()
    )

futures = client.map(fakejob, [input_path1, input_path2])

问题是我一直得到: AssertionError: daemonic processes are not allowed to have children

我试过跟随 this link

from dask.distributed import Client

client = Client(processes=True)

def fakejob(path):
    with dask.set_options(get=client.get):
        return (
            dask.bag
            .read_text(path)
            .to_dataframe()
        )

futures = client.map(fakejob, [input_path1, input_path2])

关于如何做到这一点,有什么线索吗?

干杯