Checking your work

Basic principles of software testing

What could go wrong?

math error
The problem here was not the error; it was the failure of NASA’s systems engineering, and the checks and balances in our processes, to detect the error. That’s why we lost the spacecraft.
— Edward Weiler, NASA associate administrator for space science

What could go wrong?

knight capital AP

What could go wrong?

warehouse hit by scud

Summary

  • Sometimes little errors can cause big problems

  • A lot of problems can be prevented through careful testing

Kinds of tests

  • Unit tests

    • test components (functions) in isolation

  • Integration tests

    • test interfaces between components

  • System tests

    • test system functionality

Test cases

  • Happy path

    • inputs and expected outputs are "normal"

  • Edge cases ("unhappy path")

    • inputs are unusual or special values

    • may trigger exceptions

assert statements

An assert statement tells Python to verify an expression. If the expression evaluates to True or a truthy value, the program continues. Otherwise, an AssertionError is raised.

Two forms:

  • assert expression

    assert cube(3) == 27

  • assert expression, error_msg

    assert cube(3) == 27, \
    ("cube(3) returned"
     " unexpected result")

assert in action

annual permit fees

Pretend we have a function faculty_parking() that takes an annual income and returns a permit fee.

# some happy path cases
assert faculty_parking(25_000) == 494
assert faculty_parking(40_000) == 559
assert faculty_parking(50_000) == 627
assert faculty_parking(70_000) == 932
assert faculty_parking(90_000) == 986
# some edge cases
assert faculty_parking(30_000) == 494
assert faculty_parking(30_001) == 559
# etc.

Testing floating-point values

True or false: 1/3 == (10/3)/10

  • Floats can be imprecise (=inexact)

  • Calculations can result in additional loss of precision

  • We almost never care about the exact value of a float

When testing floats, we almost always want to know if they are "close enough"

Testing floating-point values

What not to do:

assert myfloat == 0.3333333333333333

What to do:

assert abs(myfloat - 0.3333333333333333) <= 0.000000001

Even better:

from math import isclose

assert isclose(myfloat, 0.3333333333333333)

Summary

  • assert statements tell Python to verify an expression

  • If the expression evaluates to False or a falsy value, an AssertionError is raised

    • You can specify an error message to go along with an AssertionError

  • When testing floating-point values, we want to check for a "close" value rather than an exact value

    • math.isclose() is our friend

Modules, part 1: reusing other people’s work

  • Modules allow code to be reused in other programs

  • Some modules you may be familiar with:

    • math

    • random

    • sys

    • pandas

import statements, part 1

  • import statements allow us to access code in a module

  • import statements come in different flavors

Flavor #1:

import modulename

import math
import statistics

We can then access objects from the math module like this:

math.pi
math.sqrt(5)
statistics.stdev([1, 3, 2, 5, 6])

Namespaces

  • When you import the math module, Python creates an object called math that contains the functions and constants defined in the module. This object is a namespace.

  • A namespace is a container for holding Python objects (functions, variables, etc.).

  • Namespaces help prevent name collisions.

  • Every program gets its own namespace: the global namespace.

import statements, part 2

Flavor #2:

import modulename as othername

import math as m
import statistics as stats

m.pi
m.sqrt(5)
stats.stdev([1, 3, 2, 5, 6])

Imports modulename but renames the namespace to othername.

import statements, part 3

Flavor #3:

from modulename import object [, object …​]

from math import pi, sqrt
from statistics import stdev

pi
sqrt(5)
stdev([1, 3, 2, 5, 6])

Imports specific objects from modulename into the current (global) namespace.

It’s okay to import multiple objects with a single import statement.

import statements, part 4

Flavor #4:

from modulename import object as othername [, object as othername …​]

from math import pi as PI, sqrt as square_root
from statistics import stdev as st_dev

PI
square_root(5)
st_dev([1, 3, 2, 5, 6])

Imports specific objects from modulename into the current (global) namespace but renames them to othername.

import statements, part 5

Flavor #5:

from modulename import *

from math import *
from statistics import *

pi
sqrt(5)
stdev([1, 3, 2, 5, 6])

Imports all objects from modulename into the current (global) namespace.

  • Can clobber objects in your namespace

  • Obscures the origin of imported objects (where is pi defined?)

Summary

  • Modules make it possible to package up code so it can be reused

  • There are several flavors of import statements for importing modules:

    • import modulename

    • import modulename as othername

    • from modulename import object [, object …​]

    • from modulename import object as othername [, object as othername …​]

    • from modulename import *

  • The first two flavors of import statements create namespaces

    • Namespaces are containers that help prevent name collisions

Modules, part 2: now it’s personal

  • Turns out any script can be used as a module

  • Some scripts make better modules than others

  • We want most scripts we write to work as modules

    • Easier to test

    • Can reuse functions

What a module shouldn’t do

surprise

A little bit of magic

A good program is

  • 99% definitions

  • 1% instructions to do something

Put the 1% inside the following statement:

if __name__ == "__main__":
    # instructions go here

Decoding the magic

  • Whenever Python loads a script, it creates a special variable, __name__

  • For modules, __name__ is the name of the module

  • For the main script, __name__ gets the special value "__main__"

Bottom line: the value of __name__ will be different depending on whether your script was run or imported.

Summary

  • We want to write scripts that work as both programs and modules

  • Modules generally shouldn’t execute code when they are loaded

  • if __name__ == "__main__": helps you isolate instructions that should only run when your program runs.

Pytest

  • Pytest is a popular testing framework for Python

  • If you can import a module and write functions and assert statements, you can use Pytest

Example script to test

facparking.py
import sys

def faculty_parking(income):
    """ Determine the cost of faculty
    parking based on a faculty member's
    annual salary. """
    return (494 if income < 30_001 else
            559 if income < 45_001 else
            627 if income < 60_001 else
            932 if income < 80_001 else
            986)
if __name__ == "__main__":
    try:
        income = int(sys.argv[1])
    except IndexError:
        print("Please provide your income"
              " as a command-line argument")
    except ValueError:
        print("Please provide your income"
              " as an integer as the first"
              " command-line argument")
    print("You would pay",
          faculty_parking(income),
          "for an annual parking pass")

Example Pytest test script

test_facparking.py
import facparking as fp

def test_faculty_parking_happy_path():
    """ some happy path cases to test
    faculty_parking() """
    assert fp.faculty_parking(25_000) == 494
    assert fp.faculty_parking(40_000) == 559
    assert fp.faculty_parking(50_000) == 627
    assert fp.faculty_parking(70_000) == 932
    assert fp.faculty_parking(90_000) == 986
def test_faculty_parking_edge_cases():
    """ some edge cases to test
    faculty_parking() """
    assert fp.faculty_parking(30_000) == 494
    assert fp.faculty_parking(30_001) == 559
    # etc.

Running your tests

The following instructions assume you have Pytest installed.

  1. Open a terminal in the directory where your script and test script live.

  2. Type pytest followed by a space and the name of your test script, e.g.:
    pytest test_facparking.py

Interpreting the output: passed all tests

========================== test session starts ===========================
platform linux -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/aric/Documents/INST326/2020_fall/module03
collected 2 items

test_facparking.py ..                                              [100%]

=========================== 2 passed in 0.00s ============================

Interpreting the output: failed test

========================== test session starts ===========================
platform linux -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /home/aric/Documents/INST326/2020_fall/module03
collected 2 items

test_facparking.py F.                                              [100%]

================================ FAILURES ================================
____________________ test_faculty_parking_happy_path _____________________

    def test_faculty_parking_happy_path():
        """ some happy path cases to test
        faculty_parking() """
        assert fp.faculty_parking(25_000) == 494
>       assert fp.faculty_parking(40_000) == 559
E       assert 550 == 559
E        +  where 550 = (40000)
E        +    where  = fp.faculty_parking

test_facparking.py:7: AssertionError

======================== short test summary info =========================
FAILED test_facparking.py::test_faculty_parking_happy_path - assert 550...
====================== 1 failed, 1 passed in 0.02s =======================

Two ways a test can fail

  • There can be an error in the code being tested

  • There can be an error in the test code

I passed all tests! My code is perfect, right?

  • Passing tests is not a guarantee of correctness

  • The better your test set, the more confidence you can have in your code

Summary

  • Pytest is a popular, easy-to-use test framework for Python

  • Tests go in fuctions whose names start with test

  • Tests are built on assert statements