代码之家  ›  专栏  ›  技术社区  ›  sramalingam24

在ElasticSearch 5.5中使用ingest插件时,如何获取术语向量?

  •  0
  • sramalingam24  · 技术社区  · 8 年前

    全部的

    public class Document
    {
      public string Id { get; set; }
      public string Content { get; set; }
      public Attachment Attachment { get; set; }
    }
    
    var indexResponse = client.CreateIndex("documents", c => c
      .Settings(s => s
        .Analysis(a => a
         .TokenFilters(f=>f.Stemmer("english_stem",st=>st.Language("english")).Stop("english_stop",sp=>sp.StopWords("_english_")))
         .CharFilters(cf => cf.PatternReplace("num_filter", nf => nf.Pattern("(\\d+)").Replacement(" ")))                
         .Analyzers(an => an.Custom("tm_analyzer", ta => ta.CharFilters("num_filter").Tokenizer("standard").Filters("english_stem","english_stop","lowercase")))))
      .Mappings(m => m
                 .Map<Document>(mm => mm
                  .AllField(al=>al.Enabled(false))
                  .Properties(p => p                 
                   .Object<Attachment>(o=>o
                    .Name(n=>n.Attachment)
                    .Properties(ps=>ps
                     .Text(s => s
                      .Name(nm => nm.Content)
                      .TermVector(TermVectorOption.Yes)
                      .Store(true)
                      .Analyzer("tm_analyzer")))))));
    
    client.PutPipeline("attachments", p => p
      .Description("Document attachment pipeline")
      .Processors(pr => pr
        .Attachment<Document>(a => a
          .Field(f => f.Content)
          .TargetField(f => f.Attachment)
        )
        .Remove<Document>(r => r
          .Field(f => f.Content)
        )
      )
    );
    
    var base64File = Convert.ToBase64String(File.ReadAllBytes("file1.xml"));
    client.Index(new Document
    {
      Id = "file1.xml", 
      Content = base64File
    }, i => i.Pipeline("attachments"));
    

    如你所见,我在内容字段中将termvector otpion设置为yes。 但是当我使用postman或C#Nest进行如下查询时,我什么也没有得到

    POST /documents/document/_mtermvectors
    {
        "ids" : ["1.xml"],
        "parameters": {
            "fields": [
                    "content"
            ],
            "term_statistics": true
        }
    }
    

    1 回复  |  直到 8 年前
        1
  •  2
  •   Russ Cam    8 年前

    您正在删除 content 此处的摄取处理器中的字段

    .Remove<Document>(r => r
      .Field(f => f.Content)
    )
    

    这可能是您想要的,因为它将包含base64编码的附件。我认为您的API调用应该关注 attachment.content 字段,其中将包含从附件中提取的内容

    POST /documents/document/_mtermvectors
    {
        "ids" : ["1.xml"],
        "parameters": {
            "fields": [
                "attachment.content"
            ],
            "term_statistics": true
        }
    }