把句子改成词性

蟒蛇的 Regular Expression package可用于将句子拆分为标记。

import re
sentence='I am done; Look at that, cat!' 

dictionary={'!': 'sentinel', ',': 'sentinel', 
            'I': 'pronoun', 'am': 'verb', 
            'Look': 'verb', 'that': 'pronoun', 
             'at': 'preposition', ';': 'preposition', 
             'done': 'verb', ',': 'sentinel', 
             'cat': 'noun', '!': 'sentinel'}

tags = list()
for word in re.findall(r"[A-Za-z]+|\S", sentence):
    tags.append(dictionary[word])

print (' '.join(tags))

输出

代词动词动词介词动词介词代词前哨名词前哨

正则表达式 [A-Za-z]+|\S 基本上选择所有出现一个或多个字母(大写和小写) [A-Za-z]+ ,连同(由 | ,这意味着改变)所有非空白 \s .

下面是一个不那么令人印象深刻但更具解释性的解决方案:

让我们从定义问题中的示例词典和句子开始:

sentence = 'I am done; Look at that, cat!' 

dictionary = {
    '!':    'sentinel', 
    ',':    'sentinel', 
    ',':    'sentinel', 
    'I':    'pronoun', 
    'that': 'pronoun', 
    'cat':  'noun', 
    'am':   'verb', 
    'Look': 'verb', 
    'done': 'verb', 
    'at':   'preposition', 
    ';':    'preposition', 
}

对于我的解决方案,我定义了一个递归解析函数,恰当地命名为 parse . 解析 首先将一个句子按空格分隔成单词,然后尝试通过在提供的词典中查找每个单词来对其进行分类。 如果在字典里找不到这个词(因为它附有标点符号等), 解析 然后将该单词拆分为其组件标记,并从中递归地解析它。

def parse(sentence, dictionary):
  # split the words apart by whitespace
  # some tokens may still be stuck together. (i.e. "that,")
  words = sentence.split() 

  # this is a list of strings containing the 'category' of each word
  output = [] 

  for word in words:
    if word in dictionary:
      # base case, the word is in the dictionary
      output.append(dictionary[word])
    else:
      # recursive case, the word still has tokens attached

      # get all the tokens in the word
      tokens = [key for key in dictionary.keys() if key in word]

      # sort all the tokens by length - this makes sure big words are more likely to be preserved. (scat -> s, cat or sc, at) check 
      tokens.sort(key=len)

      # this is where we'll store the output 
      sub_output = None

      # iterate through the tokens to find if there's a valid way to split the word
      for token in tokens:
        try: 

          # pad the tokens inside each word
          sub_output = parse(
            word.replace(token, f" {token} "), 
            dictionary
          )

          # if the word is parsable, no need to try other combinations
          break
        except: 
          pass # the word couldn't be split

      # make sure that the word was split - if it wasn't it's not a valid word and the sentence can't be parsed
      assert sub_output is not None

      output.append(sub_output)

  # put it all together into a neat little string
  return ' '.join(output)

下面是您使用它的方法:

# usage of parse
output = parse(sentence, dictionary)

# display the example output
print(output)

我希望我的回答能让你对解决这个问题的另一种方法有更多的了解。

塔达!

如果要查找基于非正则表达式的方法,可以尝试以下操作:

def tag_pos(sentence):
    output = []
    for word in sentence.split():
        if word not in dictionary:
            literal = ''.join([char for char in word if not char.isalpha()])
            word = ''.join([char for char in word if char.isalpha()])
            output.append(dictionary[word])
            if not len(literal)>1:

                output.append(dictionary[literal])
            else:

                for literal in other:
                    output.append(dictionary[literal])
        else:
            output.append(dictionary[word])

    return " ".join(output)


output = tag_pos(sentence)
print(output)