代码之家 › 专栏 › 技术社区 › pahool

在jq中从另一个json文件中减去一个json文件

subtraction jq object json arrays

pahool · 技术社区 · 8 年前

有没有办法比较jq中的两个json文件?具体来说,如果对象出现在另一个json文件中,我希望能够从一个json文件中删除它们。基本上,从一个文件中减去另一个文件。如果我可以推广这一点,以便为对象定义相等标准,那将是一个额外的收获,但这并不是严格必要的,它可以严格基于对象是相同的。

[
  {
    "name": "Cynthia",
    "surname": "Craig",
    "isActive": true,
    "balance": "$2,426.88"
  },
  {
    "name": "Elise",
    "surname": "Long",
    "isActive": false,
    "balance": "$1,892.72"
  },
  {
    "name": "Hyde",
    "surname": "Adkins",
    "isActive": true,
    "balance": "$1,769.34"
  },
  {
    "name": "Matthews",
    "surname": "Jefferson",
    "isActive": true,
    "balance": "$1,991.42"
  },
  {
    "name": "Kris",
    "surname": "Norris",
    "isActive": false,
    "balance": "$2,137.11"
  }
]

我还有第二个文件,看起来像这样:

[
  {
    "name": "Cynthia",
    "surname": "Craig"
  },
  {
    "name": "Kris",
    "surname": "Norris"
  }
]

我想从第一个文件中删除名称和姓氏字段与第二个文件的对象匹配的任何对象,因此结果应该如下所示:

[
  {
    "name": "Elise",
    "surname": "Long",
    "isActive": false,
    "balance": "$1,892.72"
  },
  {
    "name": "Hyde",
    "surname": "Adkins",
    "isActive": true,
    "balance": "$1,769.34"
  },
  {
    "name": "Matthews",
    "surname": "Jefferson",
    "isActive": true,
    "balance": "$1,991.42"
  }
]

3 回复 | 直到 8 年前

peak 8 年前

根据前两个目标,以下解决方案旨在通用、高效且尽可能简单。

对于泛型,假设$1和$2是两个这样($x | filter)就不会出现在地图($two | filter)中,其中 filter 是任意过滤器。(在本例中 {surname, name}

解决方案使用 INDEX/1 ,它是在正式1.5版本后添加到jq中的,因此我们首先复制它的定义:

def INDEX(stream; idx_expr):
  reduce stream as $row ({};
    .[$row|idx_expr|
      if type != "string" then tojson
      else .
      end] |= $row);
def INDEX(idx_expr): INDEX(.[]; idx_expr);

效率

为了提高效率,我们需要使用JSON对象作为字典; 对于字符串,对象被归一化。为此,我们定义 normalize 具体如下:

# Normalize the input with respect to the order of keys in objects
def normalize:
  . as $in
  | if type == "object" then reduce keys[] as $key
         ( {}; . + { ($key):  ($in[$key] | normalize) } ) 
    elif type == "array" then map( normalize )
    else .
    end;

def todict(filter):
  INDEX(filter| normalize | tojson);

解决方案

解决方案现在相当简单:

# select those items from the input stream for which 
# (normalize|tojson) is NOT in dict:
def MINUS(filter; $dict):
 select( $dict[filter | normalize | tojson] | not);

def difference($one; $two; filter):
  ($two | todict(filter)) as $dict
  | $one[] | MINUS( filter; $dict );

difference( $one; $two; {surname, name} )

调用

$ jq -n --argfile one one.json --argfile two two.json -f difference.jq

jq170727 8 年前

--argfile 和 project/1 从…起 pull/1062

def project(q):
    . as $in
  | reduce (q | if type == "object" then keys[] else .[] end) as $k (
      {}
      ; . + { ($k) : ($in[$k]) }
    )
;

  map(
    reduce $arg[] as $a (
        .
      ; select(project($a) != $a)
    )
    | values
  )

second.json ,中的数据 data.json filter.jq 你可以用

jq -M --argfile arg second.json -f filter.jq data.json

生产

[
  {
    "name": "Elise",
    "surname": "Long",
    "isActive": false,
    "balance": "$1,892.72"
  },
  {
    "name": "Hyde",
    "surname": "Adkins",
    "isActive": true,
    "balance": "$1,769.34"
  },
  {
    "name": "Matthews",
    "surname": "Jefferson",
    "isActive": true,
    "balance": "$1,991.42"
  }
]

可以替换表达式 select(project($a) != $a)

再考虑一下,我们可以消除 项目/1 通过使用 contains

  map(
    reduce $arg[] as $a (
        .
      ; select(.!=null and contains($a)==false)
    )
    | values
  )

这可以通过使用 any

map(select(any(.; contains($arg[]))==false))

它足够短,可以直接在命令行上使用:

jq -M --argfile arg second.json 'map(select(any(.; contains($arg[]))==false))' data.json

RomanPerekhrest 8 年前

jq公司 解决方案:

jq --slurpfile s f2.json '[ .[] | . as $o | if (reduce $s[0][] as $i
     ([]; . + [($o | contains($i))]) | any) then empty else $o end ]' f1.json

[
  {
    "name": "Elise",
    "surname": "Long",
    "isActive": false,
    "balance": "$1,892.72"
  },
  {
    "name": "Hyde",
    "surname": "Adkins",
    "isActive": true,
    "balance": "$1,769.34"
  },
  {
    "name": "Matthews",
    "surname": "Jefferson",
    "isActive": true,
    "balance": "$1,991.42"
  }
]