如何在Pandas中过滤分类数据

您可以使用isin()函数根据任何列值过滤DataFrame,该函数返回一个布尔序列,可以将其传递给DataFrame以获得过滤结果。
您可以将此布尔序列传递给DataFrame,然后DataFrame根据传递的布尔序列过滤行后返回DataFrame。

import pandas as pd

people = {
    'sex': ['Male', 'Male', 'Male', 'Female', 'Male'],
    'age': [0.204082, 0.163265, 0.204082, 0.214082, 0.204082],
    'race': ['Hispanic', 'African-American', 'Asian', 'Asian', 'Asian']
}

df = pd.DataFrame(people)

filter_ = ~df['race'].isin(['African-American', 'Asian'])

print(filter_)

# 0     True
# 1    False
# 2    False
# 3    False
# 4    False
# Name: race, dtype: bool

df_filtered = df[filter_]
print(df_filtered)

#     sex       age      race
# 0  Male  0.204082  Hispanic

诀窍是检查每个元素是否都在给定的列表中 ~df['race'].isin(['a', 'b', c']) 。以下是一个示例:

from io import StringIO as sio

data = sio("""
 sex     age         race        
    Male    0.204082    Hispanic    
    Male    0.122449    African-American    
    Female  0.163265    African-American    
    Male    0.081633    African-American    
    Male    0.530612    African-American
""")

import pandas as pd
df = pd.read_csv(data, sep='\s+').astype({'race': 'category'})

df_train_val_scaled = df[~df["race"].isin(["Native American", "Asian"])]
df_train_val_scaled