代码之家  ›  专栏  ›  技术社区  ›  sniperd Ali Ahmed

快速映射/修改Python命令的大列表中的值?

  •  4
  • sniperd Ali Ahmed  · 技术社区  · 7 年前

    我有一些代码,我想加快速度。也许我的想法是对的,但每当我问StackOverflow时,总会有人知道一个聪明的小技巧“使用map!”“试试这个lambda”,或者“导入itetools”,我希望有人能帮上忙。这是我关心的代码部分:

    #slowest part from here....
    for row_dict in json_data:
        row_dict_clean = {}
        for key, value in row_dict.items():
            value_clean = get_cleantext(value)
            row_dict_clean[key] = value_clean
        json_data_clean.append(row_dict_clean)
        total += 1
    #to here...
    

    list 包含字典,我需要运行每个 value 通过一个小清洁工。最后我得到了一份很好的清理字典的清单。任何聪明的 iterate

    def get_json_data_clean(json_data):
        json_data_clean = []
        total = 0
        #slowest part from here....
        for row_dict in json_data:
            row_dict_clean = {}
            for key, value in row_dict.items():
                value_clean = get_cleantext(value)
                row_dict_clean[key] = value_clean
            json_data_clean.append(row_dict_clean)
            total += 1
        #to here...
        return json_data_clean
    
    def get_cleantext(value):
        #do complex cleaning stuffs on the string, I can't change what this does
        value = value.replace("bad", "good")
        return value
    
    json_data = [
        {"key1":"some bad",
         "key2":"bad things",
         "key3":"extra bad"},
        {"key1":"more bad stuff",
         "key2":"wow, so much bad",
         "key3":"who dis?"},
        # a few million more dictionaries
        {"key1":"so much bad stuff",
         "key2":"the bad",
         "key3":"the more bad"},
    ]
    
    json_data_clean = get_json_data_clean(json_data)
    print(json_data_clean)
    

    每当我在脑袋里套上一个小铃铛,也许有更好的办法。感谢您的帮助!

    1 回复  |  直到 7 年前
        1
  •  2
  •   Evgeny    7 年前

    一定要问聪明人 https://codereview.stackexchange.com/ ,但作为一个快速修复方法,您可以 map() 你对字典列表的转换功能如下:

    def clean_text(value: str)-> str:
        # ...
        return value.replace("bad", "good")
    
    def clean_dict(d: dict):
        return {k:clean_text(v) for k,v in d.items()}
    
    
    json_data = [
        {"key1":"some bad",
         "key2":"bad things",
         "key3":"extra bad"},
        {"key1":"more bad stuff",
         "key2":"wow, so much bad",
         "key3":"who dis?"},
        # a few million more dictionaries
        {"key1":"so much bad stuff",
         "key2":"the bad",
         "key3":"the more bad"},
    ]
    
    x = list(map(clean_dict, json_data))
    

    total 柜台,但好像从来没有离开过 get_json_data_clean() 无论如何。

    不知道为什么@Daniel Gale提议 filter()