代码之家 › 专栏 › 技术社区 › anfas2

如何使用MediaPipe在Python中检测到的地标上叠加自定义形状?

mediapipe computer-vision opencv python

anfas2 · 技术社区 · 10 月前

我正在使用 MediaPipe 以检测手部标志。虽然检测工作正常,但我想自定义可视化。代替由提供的默认连接和点 mp_drawing.draw_landmarks() ,我想在特定的地标上叠加自定义形状(如圆形、方形甚至图像)。

以下是我迄今为止所尝试的:

我用过 mp_drawing.draw_landmarks() 以可视化地标。
我修改了 mp_drawing.DrawingSpec 更改颜色和厚度,但仍使用默认渲染。

以下是我目前拥有的最小代码片段:

import cv2
import mediapipe as mp

mp_hands = mp.solutions.hands
mp_drawing = mp.solutions.drawing_utils

cap = cv2.VideoCapture(0)

with mp_hands.Hands() as hands:
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break

        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        results = hands.process(frame)

        frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS)
        
        cv2.imshow("MediaPipe Hands", frame)
        if cv2.waitKey(10) & 0xFF == ord('q'):
            break

cap.release()
cv2.destroyAllWindows()

我的目标:

我不想只画点,而是想在手腕或指尖等特定地标上添加自定义形状(例如,更大的圆圈、正方形或星形图标等图像)。

问题:

如何将自定义形状或图像叠加在MediaPipe检测到的特定地标上?
我可以完全跳过吗 mp_drawing.draw_landmarks() 手动绘制所有地标和连接?如果是,最好的方法是什么?

任何帮助或指导都将不胜感激!

1 回复 | 直到 10 月前

-1

Lewis 10 月前

下面是一个例子,说明如何做到这一点

import cv2
import mediapipe as mp
import numpy as np

mp_hands = mp.solutions.hands
mp_drawing = mp.solutions.drawing_utils

def overlay_shape(image, landmark, shape_type='circle', color=(0, 0, 255), radius=5):
    height, width, _ = image.shape
    x = int(landmark.x * width)
    y = int(landmark.y * height)

    if shape_type == 'circle':
        cv2.circle(image, (x, y), radius, color, -1)
    elif shape_type == 'square':
        cv2.rectangle(image, (x - radius, y - radius), (x + radius, y + radius), color, -1)
    elif shape_type == 'star':
        pts = []
        for i in range(5):
            angle = i * 2 * np.pi / 5
            x_star = x + radius * np.cos(angle)
            y_star = y + radius * np.sin(angle)
            pts.append((int(x_star), int(y_star)))
        pts.append(pts[0])  # Close the polygon
        cv2.fillPoly(image, [np.array(pts)], color)

    return image

cap = cv2.VideoCapture(0)
with mp_hands.Hands() as hands:
    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            print("Ignoring empty camera frame.")
            continue

    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    results = hands.process(frame)

    frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
    if results.multi_hand_landmarks:
        for hand_landmarks in results.multi_hand_landmarks:
            wrist = hand_landmarks.landmark[0]
            index_tip = hand_landmarks.landmark[8]
            middle_tip = hand_landmarks.landmark[12]
            # Apply shapes to the landmarks
            frame = overlay_shape(frame, wrist, shape_type='square', color=(0, 255, 0), radius=10)
            frame = overlay_shape(frame, index_tip, shape_type='star', color=(255, 0, 0), radius=8)
            frame = overlay_shape(frame, middle_tip, shape_type='circle', color=(0, 0, 255), radius=15)

            mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS)

    cv2.imshow('MediaPipe Hands', frame)
    if cv2.waitKey(10) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

我在中找到的手地标索引 Hand Tracking with 21 landmarks