自动化框架的两种断言设计(pytest 版)

自动化测试断言失败时，根据不同业务场景，可能需要立即终止或继续执行。这里以 Appium + pytest 为例。

一. 断言失败立即终止

用途一：用例的预期结果是其他用例的前提条件时，assert 失败需要立即终止，这是我们设计自动化测试用例时最常见的场景。

用途二：用例中的任何一个步骤执行失败时，立即终止，因为步骤都执行失败了，没有继续执行下一步的必要。方案：思路与用途一类似，这里把每个测试步骤当做都有一个预期结果(True)，封装每一个具体步骤，步骤结果返回布尔值。对该结果进行 assert，False时立即终止。

def click(self, step):
    method_name = sys._getframe().f_code.co_name
    try:
        element = self.find_element(**step['element_loc'])
        if step.has_key('times'):
            for i in range(step['times']):
                element.click()
        else:
            element.click()
        return True
    except:
        print u'%s失败' % method_name
        return False

for step in self.case_steps:
    assert self.operate(step), 'error in step: %s' % step

二. 断言失败继续执行

主要使用了两个函数 expect, assert_expectations 。

Demo: test_delayed_assert.py

from delayed_assert import expect, assert_expectations
 
def test_should_pass():
    expect(1 == 1, 'one is one')
    assert_expectations()
 
def test_should_fail():
    expect(1 == 2)
    x = 1
    y = 2
    expect(x == y, 'x:%s y:%s' % (x,y))
    expect(1 == 1)
    assert_expectations()

Module: delayedAssert.py

'''
Implements one form of delayed assertions.

Interface is 2 functions:

  expect(expr, msg=None)  
  : Evaluate 'expr' as a boolean, and keeps track of failures

  assert_expectations()        
  : raises an assert if an expect() calls failed  

Usage Example:

    from expectations import expect, assert_expectations

    def test_should_pass():
        expect(1 == 1, 'one is one')
        assert_expectations()

    def test_should_fail():
        expect(1 == 2, 'one is two')
        expect(1 == 3, 'one is three')
        assert_expectations() 
'''

# ---------------------------------------------------

def expect(expr, msg=None):
    'keeps track of failed expectations'
    if not expr:
        _log_failure(msg)

def assert_expectations():
    'raise an assert if there are any failed expectations'
    if _failed_expectations:
        assert False, _report_failures()

# ---------------------------------------------------

import inspect
import os.path

_failed_expectations = []

def _log_failure(msg=None):
    (filename, line, funcname, contextlist) =  inspect.stack()[2][1:5]
    filename = os.path.basename(filename)
    context = contextlist[0]
    _failed_expectations.append('file "%s", line %s, in %s()%s\n%s' % 
        (filename, line, funcname, (('\n%s' % msg) if msg else ''), context))

def _report_failures():
    global _failed_expectations
    if _failed_expectations:
        (filename, line, funcname) =  inspect.stack()[2][1:4]
        report = [
            '\n\nassert_expectations() called from',
            '"%s" line %s, in %s()\n' % (os.path.basename(filename), line, funcname),
            'Failed Expectations:%s\n' % len(_failed_expectations)]
        for i,failure in enumerate(_failed_expectations, start=1):
            report.append('%d: %s' % (i, failure))
        _failed_expectations = []
    return ('\n'.join(report))

# ---------------------------------------------------
# _log_failure() notes
#
# stack() returns a list of frame records
#   0 is the _log_failure() function
#   1 is the expect() function 
#   2 is the function that called expect(), that's what we want
#
# a frame record is a tuple like this:
#   (frame, filename, line, funcname, contextlist, index)
# we're mainly interested in the middle 4, 
# ---------------------------------------------------

参考：Delayed assert / multiple failures per test