==========
Vdsm tests
==========

This tests suite is built on tox, pytest and nose. For more info on
these projects see:

- tox: https://tox.readthedocs.io/
- pytest: https://pytest.readthedocs.io/
- nose: http://nose.readthedocs.io/

Tox and pytest based tests
==========================

Currently only storage tests are using tox and pytest.

Running the tests
-----------------

Run storage tests with all installed python versions:

    tox -e storage-py27,storage-py35,storage-py36

Decreasing test verbosity:

    tox -e storage-py27 -- -qq

You can also use enviroment variables

   export PYTEST_ADDOPTS=-qq
   tox -e storage-py35

Profiling tests, showing 10 slowest tests:

   tox -e storage-py27 -- --durations=10

Using pytest directly
---------------------

Running all the tests takes couple of minutes. For quicker feedback you can use
pytest directly to run a module or some tests in a module.

To use pytest directly, you must set the python path properly:

    export PYTHONPATH=../lib

Running entire module:

    pytest storage/misc_test.py

Running only one class:

    pytest storage/misc_test.py::TestSamplingMethod

Running only single test:

    pytest storage/misc_test.py::TestSamplingMethod::test_single_thread

Running all tests matching a pattern:

    pytest -k test_thread storage/misc_test.py

Running without slow and stress tests, using markers:

    pytest -m "not (slow or stress)" storage

Running only stress tests, using markers:

    pytest -m "stress" storage

Running the tests as root
-------------------------

When using sudo, use env to pass PYTHONPATH to the underlying program:

    sudo env PYTHONPATH=$PYTHONPATH  tox ...

When running the tests as root, use another --basetemp directory for pytest:

    sudo ... pytest --basetemp=/var/tmp/vdsm-root storage/blockdev_test.py

This will avoid failures when you run the tests later as regular user, when
pytest will try to remove old temporary directories created by root.

Nose based tests
================

Running the tests
-----------------

From within the source directory you can execute some test modules:
    ./run_tests_local.sh *.py

From within an installed directory:
    ./run_tests.sh *.py

Individual files, test cases class or test function can be specified:
    ./run_tests.sh foo_test.py
    ./run_tests.sh foo_test:TestBar
    ./run_tests.sh foo_test:TestBar.test_baz

Enabling slow tests:
-------------------

Some tests are too slow to run on each build. these are marked in the source
with the @slowtest decorator and are disabled by default.

To enable slow tests:
     ./run_tests_local.sh --enable-slow-tests filenme [...]

Slow tests are also enabled if NOSE_SLOW_TESTS environment variable is set.

Enabling stress tests:
---------------------

Some tests stress the resources of the system running the tests. These tests
are too slow to run on each build, and may fail on overloaded system or when
running tests in parrallel. These tests are marked in the source with the
@stresstest decorator and are disabled by default.

To enable stress tests:
     ./run_tests_local.sh --enable-stress-tests filename [...]

Stress tests are also enabled if NOSE_STRESS_TESTS environment variable is set.

Enabling threads leak check
---------------------------

To find tests leaking threads, you can enable the thread leak checker plugin:

    ./run_tests_local.sh --with-thread-leak-check filename [...]

To run the entire test suit with thread leak detection:

    make check NOSE_WITH_THREAD_LEAK_CHECK=1

Enabling processs leak check
----------------------------

To find tests leaking child processes, you can enable the process leak checker
plugin:

    ./run_tests_local.sh --with-process-leak-check filename [...]

To run the entire test suit with process leak detection:

    make check NOSE_WITH_PROCESS_LEAK_CHECK=1

Enabling file leak check
------------------------

To find tests leaking file descriptors, you can enable the process leak checker
plugin:

    ./run_tests_local.sh --with-file-leak-check filename [...]

To run the entire test suit with file leak detection:

    make check NOSE_WITH_FILE_LEAK_CHECK=1

Control verbose level:
---------------------

By default, tests run with verbose level 3.
To run without verbose output, set verbose level to 1.

To set verbose level:
    make check NOSE_VERBOSE=<VERBOSE LEVEL>

Functional test suite:
----------------------

The functional test suite is designed to test a running vdsm instance.  To run
the full suite of functional tests from within the installed directory:
    ./run_tests.sh functional/*.py

Test timeout
------------

If TIMEOUT environment variable is set, and a test run is too slow, it is
aborted, and a backtrace of all threads is printed.

Example usage:

    TIMEOUT=600 ./run_tests_local.sh foo_test.py
