ci_watson is a test helper for Continuous Integration (CI) testing
at STScI using Jenkins and Artifactory.
This package has two components:
pytestplugin containing markers and fixtures.
- Generic CI helpers for STScI tests using Jenkins and Artifactory.
The plugin portion of
@pytest.mark.slowdecorator to run or skip tests that are resource intensive. What counts as resource intensive is up to the author of the test.
@pytest.mark.bigdatadecorator to run or skip tests that require intranet (Artifactory, Central Storage, etc) access. Additional setup is required for these tests to run successfully (see Setting Up For Big Data). It is up to the author of the test to perform such setup properly.
envoptfixture to set the test environment to
"stable". This plugin only sets the value. It is up to the author of the test to use this environment setting properly.
_jailfixture to enable a test to run in a pristine temporary working directory. This is particularly useful for pipeline tests.
results_root - The ‘bigdata’ remote repository name/local
data root directory for testing input/output files. Setting the value of
either option will make it availble to tests via the
Test code can then obtain the name of the artifactory repository/local data
root directory to use when accessing locations needed for running tests.
Note: If used, these values should appear in either
pytest.ini OR the appropriate
setup.cfg, not both.
Example configuration within
[tool:pytest] inputs_root = my_data_repo results_root = my_results_repo
Example configuration within
[pytest] inputs_root = my_data_repo results_root = my_results_repo
The value(s) defined in the pytest configuration file may be accessed as a list
by test code via the
pytestconfig fixture which must be passed in as an
argument to the test method or function that will use the value.
Example of accessing configuration values within test code itself:
def test_important_thing(pytestconfig): setup_cfg_inputs_root = pytestconfig.getini('inputs_root') assert setup_cfg_inputs_root == 'my_data_repo'
From within a fixture or a test class the configuration values must be accessed using a slightly different approach:
import pytest inputs_root = pytest.config.getini('inputs_root')
Setting Up For Big Data¶
--bigdata option is used together with the environment variable,
TEST_BIGDATA, as used by
get_bigdata_root(). For local testing,
set this variable to where you downloaded your Artifactory data.
For remote testing (e.g., with Jenkins CI), set it to your Artifactory path
JenkinsfileRT, as appropriate. For more details,
please refer to STScI Innerspace document for
“Users Guide: Running Regression Tests”.
Helpers for Artifactory or local big data handling.
||Determine if URL can be resolved without error.|
||Find and returns the path to the nearest big datasets.|
||Acquire requested data from a managed resource to the current directory.|
||Compare output with “truth” using appropriate diff routine; namely:|
||Generate pattern, target, and test name for
||Write out JSON file to upload Jenkins results from test to Artifactory storage area.|
Helper module for HST tests.
||Return a list of reference filenames, as defined in the primary header of the given input image, necessary for calibration.|
||Return a list of RAW input files in a given ASN.|
||Download a CRDS file from HTTP to current directory.|