代码之家 › 专栏 › 技术社区 › The Illusive Man

混合参数化测试和标记

pytest python-3.x python

The Illusive Man · 技术社区 · 8 年前

好吧,我正在努力解决一件让我大吃一惊的事情。

虽然我的实际代码不同,但这基本上解决了这个问题。假设以下示例代码:

import pytest

@pytest.mark.parametrize('type',(
    pytest.param('stability', marks=pytest.mark.stability),
    pytest.param('integration', marks=pytest.mark.integration),
))
@pytest.mark.integration
@pytest.mark.stability
def test_meh(type):
    assert type == 'integration'

下面是运行该测试的输出。

$pytest -m integration test_meeh.py
========================================================= test session starts =========================================================
platform darwin -- Python 3.6.4, pytest-3.1.2, py-1.5.2, pluggy-0.4.0
rootdir: /Users/yzT/Desktop, inifile:
collected 2 items

test_meeh.py F

============================================================== FAILURES ===============================================================
_________________________________________________________ test_meh[stability] _________________________________________________________

type = 'stability'

    @pytest.mark.parametrize('type',(
        pytest.param('stability', marks=pytest.mark.stability),
        pytest.param('integration', marks=pytest.mark.integration),
    ))
    @pytest.mark.integration
    @pytest.mark.stability
    def test_meh(type):
>       assert type == 'integration'
E       AssertionError: assert 'stability' == 'integration'
E         - stability
E         + integration

test_meeh.py:10: AssertionError
========================================================= 1 tests deselected ==========================================================
=============================================== 1 failed, 1 deselected in 0.07 seconds ================================================


$ pytest -m stability test_meeh.py
========================================================= test session starts =========================================================
platform darwin -- Python 3.6.4, pytest-3.1.2, py-1.5.2, pluggy-0.4.0
rootdir: /Users/yzT/Desktop, inifile:
collected 2 items

test_meeh.py .

========================================================= 1 tests deselected ==========================================================
=============================================== 1 passed, 1 deselected in 0.02 seconds ================================================

这是怎么回事?为什么当我使用 -m integration 它使用 stability 当我使用 -m stability 它使用 integration ?

1 回复 | 直到 8 年前

Samuel Dion-Girardeau 8 年前

您提供的代码示例似乎有两个问题。

首先,你不应该标记两个单独的参数化测试和测试功能。一旦 -m 您提供的选项与函数上的标记装饰符匹配,测试将被选中。

这里有一个最小的比较。功能和单个参数化测试上都有标记:

# parametrized_tests.py
import pytest
@pytest.mark.parametrize('smiley', [
    pytest.param(':)', marks=[pytest.mark.happy]),
    pytest.param(':(', marks=[pytest.mark.unhappy]),
])
@pytest.mark.happy
@pytest.mark.unhappy
def test_meh(smiley):
    assert smiley == ':)'

您将收集并选择两个测试:

$ pytest -m happy --collect-only parametrized_tests.py
========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: [redacted], inifile:
collected 2 items
<Module 'params-individual-tests.py'>
  <Function 'test_smiley[:)]'>
  <Function 'test_smiley[:(]'>

====================================================================================== no tests ran in 0.01 seconds ======================================================================================

但如果只标记参数化测试:

# parametrized_tests.py
import pytest

@pytest.mark.parametrize('smiley', [
    pytest.param(':)', marks=[pytest.mark.happy]),
    pytest.param(':(', marks=[pytest.mark.unhappy]),
])
def test_smiley(smiley):
    assert smiley == ':)'

您将收集两个测试,但仅选择一个,正如您所期望的:

$ pytest -m happy --collect-only parametrized_tests.py
========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: [redacted], inifile:
collected 2 items / 1 deselected
<Module 'params-individual-tests.py'>
  <Function 'test_smiley[:)]'>

====================================================================================== 1 deselected in 0.01 seconds ======================================================================================

第二,这本书似乎有一个令人费解的错误(或未记录的“功能”) pytest.param ,其中使用标记的确切名称作为参数值会使测试未被选中。

从您的(稍加修改的)代码:

# mark_name.py
import pytest

@pytest.mark.parametrize('type_', [
    pytest.param('integration', marks=[pytest.mark.integration]),
    pytest.param('stability', marks=[pytest.mark.stability]),
])
def test_meh(type_):
    assert type_ == 'integration'

如果我尝试运行 integration 仅测试,它不会选择任何:

$ pytest -m integration mark_name.py
========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: [redacted], inifile:
collected 2 items / 2 deselected

但只需修改值(在本例中,我只将其改为大写)就可以让一切按预期运行:

# mark_name.py
import pytest

@pytest.mark.parametrize('type_', [
    pytest.param('INTEGRATION', marks=[pytest.mark.integration]),
    pytest.param('STABILITY', marks=[pytest.mark.stability]),
])
def test_meh(type_):
    assert type_ == 'INTEGRATION'

现在我可以正确地选择测试:

$ pytest -m integration mark_name.py
========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: [redacted], inifile:
collected 2 items / 1 deselected

mark_name.py .                                                                                                                                                                                     [100%]

================================================================================= 1 passed, 1 deselected in 0.01 seconds =================================================================================
$ pytest -m stability mark_name.py
========================================================================================== test session starts ===========================================================================================
platform darwin -- Python 3.6.5, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: [redacted], inifile:
collected 2 items / 1 deselected

mark_name.py F                                                                                                                                                                                     [100%]

================================================================================================ FAILURES ================================================================================================
__________________________________________________________________________________________ test_meh[STABILITY] ___________________________________________________________________________________________

type_ = 'STABILITY'

    @pytest.mark.parametrize('type_', [
        pytest.param('INTEGRATION', marks=[pytest.mark.integration]),
        pytest.param('STABILITY', marks=[pytest.mark.stability]),
    ])
    def test_meh(type_):
>       assert type_ == 'INTEGRATION'
E       AssertionError: assert 'STABILITY' == 'INTEGRATION'
E         - STABILITY
E         + INTEGRATION

mark_name.py:9: AssertionError
================================================================================= 1 failed, 1 deselected in 0.08 seconds =================================================================================

我怀疑这与用作测试ID的字符串值有关,但建议打开 GitHub issue 如果这是你不想解决的问题。

而且,完全不相关,但最好不要将论点定义为 type ,因为您正在隐藏内置函数 type . PEP-8 建议使用尾随下划线以防止名称冲突。