- Edited: 6/18/2016: Updated RollbackImporter to support relative imports and “from package import module” syntax.
In my last post, I went over writing scripts that can be used to write unit tests for Maya tools and then run those tests from the commandline outside of Maya. In this post, I'll go over running those tests interactively in Maya. Being able to run tests interactively in Maya will greatly speed up debug times when tracking down issues because you'll be able to see what is going on in your tests. To make the process more user-friendly, I'll also add a UI to help execute the tests. Again, all of this code is available on my Github.
The interface is written in PySide and uses a QTreeView to display a derived QAbstractItemModel holding the discovered test data. Since the UI is pretty standard PySide code, I will only go over the code related to actually running the tests. The UI provides three different methods of running tests as a convenience to the user: run all tests, run selected tests, and run failed tests. To run all the tests, we can use the get_tests function mentioned in the previous post.
Run All Tests
def run_all_tests(self):
"""Callback method to run all the tests found in MAYA_MODULE_PATH."""
self.reset_rollback_importer()
test_suite = unittest.TestSuite()
mayaunittest.get_tests(test_suite=test_suite)
self.output_console.clear()
self.model.run_tests(self.stream, test_suite)
The first line resets what I'll call the rollback importer. It is used to reload code updates without having to rely on sprinkling reload calls throughout your code which is typically bad form. I'll go over the rollback importer shortly. Next, we populate a TestSuite with all tests found in the paths of MAYA_MODULE_PATH by calling our get_tests function. Then we clear our output console and run the tests.
The rollback importer helps to create a better testing workflow by getting the latest version of the code without having to reload anything. It allows for a iterative update/test workflow without having to restart Maya or insert several reload statements in your code. I learned about this technique from PyUnit.
The Rollback Importer
import __builtin__
class RollbackImporter(object):
"""Used to remove imported modules from the module list.
This allows tests to be rerun after code updates without doing any reloads.
From: http://pyunit.sourceforge.net/notes/reloading.html
Usage:
def run_tests(self):
if self.rollback_importer:
self.rollback_importer.uninstall()
self.rollback_importer = RollbackImporter()
self.load_and_execute_tests()
"""
def __init__(self):
"""Creates an instance and installs as the global importer."""
self.previous_modules = set(sys.modules.keys())
def uninstall(self):
for modname in sys.modules.keys():
if modname not in self.previous_modules:
# Force reload when modname next imported
del(sys.modules[modname])
The rollback importer temporarily replaces the Python import function to store all the modules imported from that point forward which it can then use to remove the loaded modules from the sys.modules dictionary. As a result, the module will be import fresh each time it is imported. Users just have to run the rollback importer (which runs by opening the UI) before importing any other modules.
The PySide data model class contains its own run_tests method to run all the specified tests.
Running Tests from the UI
def run_tests(self, stream, test_suite):
"""Runs the given TestSuite.
@param stream: A stream object with write functionality to capture the test output.
@param test_suite: The TestSuite to run.
"""
runner = unittest.TextTestRunner(stream=stream, verbosity=2, resultclass=mayaunittest.TestResult)
runner.failfast = False
runner.buffer = mayaunittest.Settings.buffer_output
result = runner.run(test_suite)
self._set_test_result_data(result.failures, TestStatus.fail)
self._set_test_result_data(result.errors, TestStatus.error)
self._set_test_result_data(result.skipped, TestStatus.skipped)
for test in result.successes:
node = self.node_lookup[str(test)]
index = self.get_index_of_node(node)
self.setData(index, 'Test Passed', QtCore.Qt.ToolTipRole)
self.setData(index, TestStatus.success, QtCore.Qt.DecorationRole)
def _set_test_result_data(self, test_list, status):
"""Store the test result data in model.
@param test_list: A list of tuples of test results.
@param status: A TestStatus value."""
for test, reason in test_list:
node = self.node_lookup[str(test)]
index = self.get_index_of_node(node)
self.setData(index, reason, QtCore.Qt.ToolTipRole)
self.setData(index, status, QtCore.Qt.DecorationRole)
The code is similar to the run_tests function in previous post except this time we are capturing the result to display in the UI. You'll see some code related to looking up the QModelIndex of an item in the data model given a test object. This is used to aid in updating the UI with the proper result per test. The stream that is passed in is used to capture the output of the tests and display them in the output console (a QTestEdit) with color output.
Capturing Test Output
class TestCaptureStream(object):
"""Allows the output of the tests to be displayed in a QTextEdit."""
success_color = QtGui.QColor(92, 184, 92)
fail_color = QtGui.QColor(240, 173, 78)
error_color = QtGui.QColor(217, 83, 79)
skip_color = QtGui.QColor(88, 165, 204)
normal_color = QtGui.QColor(200, 200, 200)
def __init__(self, text_edit):
self.text_edit = text_edit
def write(self, text):
"""Write text into the QTextEdit."""
# Color the output
if text.startswith('ok'):
self.text_edit.setTextColor(TestCaptureStream.success_color)
elif text.startswith('FAIL'):
self.text_edit.setTextColor(TestCaptureStream.fail_color)
elif text.startswith('ERROR'):
self.text_edit.setTextColor(TestCaptureStream.error_color)
elif text.startswith('skipped'):
self.text_edit.setTextColor(TestCaptureStream.skip_color)
self.text_edit.insertPlainText(text)
self.text_edit.setTextColor(TestCaptureStream.normal_color)
def flush(self):
pass
To run a given test interactively, we will want to be able to disable the functionality of creating a new scene after each test. That way we can run a test and see actually see what it did in Maya.
Disabling New Scenes After a Test
class MayaTestRunnerDialog(MayaQWidgetBaseMixin, QtGui.QMainWindow):
def __init__(self, *args, **kwargs):
super(MayaTestRunnerDialog, self).__init__(*args, **kwargs)
# snip...
action = menu.addAction('New Scene Between Test')
action.setToolTip('Creates a new scene file after each test.')
action.setCheckable(True)
action.setChecked(mayaunittest.Settings.file_new)
action.toggled.connect(mayaunittest.set_file_new)
# In mayaunittest.py
def set_file_new(value):
"""Set whether a new file should be created after each test.
@param value: True or False
"""
Settings.file_new = value
Running the selected tests is similar to running all the tests except we can now use the test parameter of the get_tests function to only load specific tests.
Running Selected Tests
def run_selected_tests(self):
"""Callback method to run the selected tests in the UI."""
self.reset_rollback_importer()
test_suite = unittest.TestSuite()
indices = self.test_view.selectedIndexes()
if not indices:
return
# Remove any child nodes if parent nodes are in the list. This will prevent duplicate tests from being run.
paths = [index.internalPointer().path() for index in indices]
test_paths = []
for path in paths:
tokens = path.split('.')
for i in range(len(tokens) - 1):
p = '.'.join(tokens[0:i+1])
if p in paths:
break
else:
test_paths.append(path)
# Now get the tests with the pruned paths
for path in test_paths:
mayaunittest.get_tests(test=path, test_suite=test_suite)
self.output_console.clear()
self.model.run_tests(self.stream, test_suite)
Also it is nice to be able to run all failed tests. It is almost identical to run_all_test except we only use the tests with fail or error statuses.
Running Failed Tests
def run_failed_tests(self):
"""Callback method to run all the tests with fail or error statuses."""
self.reset_rollback_importer()
test_suite = unittest.TestSuite()
for node in self.model.node_lookup.values():
if isinstance(node.test, unittest.TestCase) and node.get_status() in {TestStatus.fail, TestStatus.error}:
mayaunittest.get_tests(test=node.path(), test_suite=test_suite)
self.output_console.clear()
self.model.run_tests(self.stream, test_suite)
With all the above code, we now have a way to interactively run our unit tests inside of Maya. Also all of this code is available for free on Github.