What follows is a commented example of some tests in a single file demonstrating many of the Test Format features. See Loader, below, for the Python needed to integrate with a testing harness.
# Fixtures can be used to set any necessary configuration, such as a
# persistence layer, and establish sample data. They operate per
# file. They are context managers, each one wrapping the next in the
# sequence.
fixtures:
- ConfigFixture
- SampleDataFixture
# There is an included fixture named "SkipAllFixture" which can be
# used to declare that all the tests in the given file are to be
# skipped.
# Each test file can specify a set of defaults that will be used for
# every request. This is useful for always specifying a particular
# header or always requiring SSL. These values will be used on every
# test in the file unless overriden. Lists and dicts are merged one
# level deep, except for "data" which is copied verbatim whether it
# is a string, list or dict (it can be all three).
defaults:
ssl: True
request_headers:
x-my-token: zoom
# The tests themselves are a list under a "tests" key. It's useful
# to use plenty of whitespace to help readability.
tests:
# Each request *must* have a name which is unique to the file. When it
# becomes a TestCase the name will be lowercased and spaces will
# become "_". Use that generated name when limiting test runs.
- name: a test for root
desc: Some explanatory text that could be used by other tooling
# The URL can either be relative to a host specified elsewhere or
# be a fully qualified "http" or "https" URL. *You* are responsible
# for url-encoding the URL.
url: /
# If no status or method are provided they default to "200" and
# "GET".
# A single test can override settings in defaults (set above).
- name: root without ssl redirects
ssl: False
url: /
status: 302
# When evaluating response headers it is possible to use a regular
# expression to not have to test the whole value.
response_headers:
location: /https/
# By default redirects will not be followed. This can be changed.
- name: follow root without ssl redirect
ssl: False
redirects: True
url: /
status: 200 # This is the response code after the redirect.
# URLs can express query parameters in two ways: either in the url
# value directly, or as query_parameters. If both are used then
# query_parameters are appended. In this example the resulting URL
# will be equivalient to
# /foo?section=news&article=1&article=2&date=yesterday
# but not necessarily in that order.
- name: create a url with parameters
url: /foo?section=news
query_parameters:
article:
- 1
- 2
date: yesterday
# Request headers can be used to declare media-type choices and
# experiment with authorization handling (amongst other things).
# Response headers allow evaluating headers in the response. These
# two together form the core value of gabbi.
- name: test accept
url: /resource
request_headers:
accept: application/json
response_headers:
content-type: /application/json/
# If a header must not be present in a response at all that can be
# expressed in a test as follows.
- name: test forbidden headers
url: /resource
response_forbidden_headers:
- x-special-header
# All of the above requests have defaulted to a "GET" method. When
# using "POST", "PUT" or "PATCH", the "data" key provides the
# request body.
- name: post some text
url: /text_repo
method: POST
request_headers:
content-type: text/plain
data: "I'm storing this"
status: 201
# If the data is not a string, it will be transformed into JSON.
# You must supply an appropriate content-type request header.
- name: post some json
url: /json_repo
method: POST
request_headers:
content-type: application/json
data:
name: smith
abode: castle
status: 201
# If the data is a string prepended with "<@" the value will be
# treated as the name of a file in the same directory as the YAML
# file. Again, you must supply an appropriate content-type. If the
# content-type is one of several "text-like" types, the content will
# be assumed to be UTF-8 encoded.
- name: post an image
url: /image_repo
method: POST
request_headers:
content-type: image/png
data: <@kittens.png
# A single request can be marked to be skipped.
- name: patch an image
skip: patching images not yet implemented
url: /image_repo/12d96fb8-e78c-11e4-8c03-685b35afa334
method: PATCH
# Or a single request can be marked that it is expected to fail.
- name: check allow headers
desc: the framework doesn't do allow yet
xfail: True
url: /post_only_url
method: PUT
status: 405
response_headers:
allow: POST
# The body of a response can be evaluated with response handlers.
# The most simple checks for a set of strings anywhere in the
# response. Note that the strings are members of a list.
- name: check for css file
url: /blog/posts/12
response_strings:
- normalize.css
# For JSON responses, JSONPath rules can be used.
- name: post some json get back json
url: /json_repo
method: POST
request_headers:
content-type: application/json
data:
name: smith
abode: castle
status: 201
response_json_paths:
$.name: smith
$.abode: castle
# Requests run in sequence. One test can make reference to the test
# immediately prior using some special variables.
# "$LOCATION" contains the "location" header in the previous
# response.
# "$HEADERS" is a pseudo dictionary containing all the headers of
# the previous response.
# "$ENVIRON" is a pseudo dictionary providing access to the current
# environment.
# "$RESPONSE" provides access to the JSON in the prior response, via
# JSONPath. See http://jsonpath-rw.readthedocs.io/ for
# jsonpath-rw formatting.
# $SCHEME and $NETLOC provide access to the current protocol and
# location (host and port).
- name: get the thing we just posted
url: $LOCATION
request_headers:
x-magic-exchange: $HEADERS['x-magic-exchange']
x-token: $ENVIRON['OS_TOKEN']
response_json_paths:
$.name: $RESPONSE['$.name']
$.abode: $RESPONSE['$.abode']
response_headers:
content-location: /$SCHEME://$NETLOC/
# For APIs where resource creation is asynchronous it can be
# necessary to poll for the resulting resource. First we create the
# resource in one test. The next test uses the "poll" key to loop
# with a delay for a set number of times.
- name: create asynch
url: /async_creator
method: POST
request_headers:
content-type: application/json
data:
name: jones
abode: bungalow
status: 202
- name: poll for created resource
url: $LOCATION
poll:
count: 10 # try up to ten times
delay: .5 # wait .5 seconds between each try
response_json_paths:
$.name: $RESPONSE['$.name']
$.abode: $RESPONSE['$.abode']
To run the tests they must be generated in some fashion and then run. This is accomplished by a test loader. Initially gabbi only supported those test harnesses that supported the load_tests protocol in UnitTest. It now possible to also build and run tests with pytest with some limitations described below.
To run the tests with a load_tests style loader a test file containing a load_tests method is required. That will look a bit like:
"""A sample test module."""
# For pathname munging
import os
# The module that build_tests comes from.
from gabbi import driver
# We need access to the WSGI application that hosts our service
from myapp import wsgiapp
# We're using fixtures in the YAML files, we need to know where to
# load them from.
from myapp.test import fixtures
# By convention the YAML files are put in a directory named
# "gabbits" that is in the same directory as the Python test file.
TESTS_DIR = 'gabbits'
def load_tests(loader, tests, pattern):
"""Provide a TestSuite to the discovery process."""
test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
return driver.build_tests(test_dir, loader,
intercept=wsgiapp.app,
fixture_module=fixtures)
For details on the arguments available when building tests see build_tests().
Once the test loader has been created, it needs to be run. There are many options. Which is appropriate depends very much on your environment. Here are some examples using unittest or testtools that require minimal knowledge to get started.
By file:
python -m testtools.run -v test/test_loader.py
By module:
python -m testttols.run -v test.test_loader
python -m unittest -v test.test_loader
Using test discovery to locate all tests in a directory tree:
python -m testtools.run discover
python -m unittest discover test
See the source distribution and the tutorial repo for more advanced options, including using testrepository and subunit.
Since pytest does not support the load_tests system, a different way of generating tests is required. A test file must be created that calls py_test_generator() and yields the generated tests. That will look a bit like this:
"""A sample pytest module."""
# For pathname munging
import os
# The module that build_tests comes from.
from gabbi import driver
# We need access to the WSGI application that hosts our service
from myapp import wsgiapp
# We're using fixtures in the YAML files, we need to know where to
# load them from.
from myapp.test import fixtures
# By convention the YAML files are put in a directory named
# "gabbits" that is in the same directory as the Python test file.
TESTS_DIR = 'gabbits'
def test_gabbits():
test_dir = os.path.join(os.path.dirname(__file__), TESTS_DIR)
test_generator = driver.py_test_generator(
test_dir, intercept=wsgiapp.app,
fixture_module=fixtures)
for test in test_generator:
yield test
This can then be run with the usual pytest commands. For example:
py.test -svx pytest-example.py