代码之家 › 专栏 › 技术社区 › Jeff

弹性搜索多词关键字标记化同义词分析

synonym elasticsearch

8

Jeff · 技术社区 · 11 年前

我正在尝试使用_analyze API获取关键字标记化的多单词同义词。然而,API返回的是单个单词同义词的预期结果,而不是多单词同义词。以下是我的设置和分析链:

curl -XPOST "http://localhost:9200/test" -d'
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "my_syn_filt": {
            "type": "synonym",
            "synonyms": [
              "foo bar, fooo bar", 
              "bazzz, baz"
            ]
          }
        },
        "analyzer": {
          "my_synonyms": {
            "filter": [
              "lowercase",
              "my_syn_filt"
            ],
            "tokenizer": "keyword"
          }
        }
      }
    }
  }
}'

现在使用分析API进行测试(_A):

curl 'localhost:9200/test/_analyze?analyzer=my_synonyms&text=baz'

该调用返回我所期望的结果(同样的结果也返回给“bazzz”):

{
  "tokens": [
    {
      "position": 1,
      "type": "SYNONYM",
      "end_offset": 3,
      "start_offset": 0,
      "token": "bazzz"
    },
    {
      "position": 1,
      "type": "SYNONYM",
      "end_offset": 3,
      "start_offset": 0,
      "token": "baz"
    }
  ]
}

现在,当我尝试使用多单词同义词文本进行相同的调用时,API只返回一个“word”类型的标记,没有同义词:

curl 'localhost:9200/test/_analyze?analyzer=my_synonyms&text=foo+bar'

(返回)

{
  "tokens": [
    {
      "position": 1,
      "type": "word",
      "end_offset": 7,
      "start_offset": 0,
      "token": "foo bar"
    }
  ]
}

为什么分析API不同时返回SYNONYM类型的“foo bar”和“fooo bar”标记?

1 回复 | 直到 11 年前

1

13

Jeff 11 年前

“tokenizer”:“keyword”键值也需要添加到my_syn_filt过滤器声明中,如下所示:

curl -XPOST "http://localhost:9200/test" -d'
{
  "settings": {
    "index": {
      "analysis": {
        "filter": {
          "my_syn_filt": {
            "tokenizer": "keyword",
            "type": "synonym",
            "synonyms": [
              "foo bar, fooo bar", 
              "bazzz, baz"
            ]
          }
        },
        "analyzer": {
          "my_synonyms": {
            "filter": [
              "lowercase",
              "my_syn_filt"
            ],
            "tokenizer": "keyword"
          }
        }
      }
    }
  }
}'

通过以上映射,_analyze API返回所需的SYNONYM标记:

{
  "tokens": [
    {
      "position": 1,
      "type": "SYNONYM",
      "end_offset": 7,
      "start_offset": 0,
      "token": "foo bar"
    },
    {
      "position": 1,
      "type": "SYNONYM",
      "end_offset": 7,
      "start_offset": 0,
      "token": "fooo bar"
    }
  ]
}