Initial commit

This commit is contained in:
AbstractConcept 2022-09-13 00:36:34 -05:00
commit 3c7cc0c973
8391 changed files with 704313 additions and 0 deletions

View file

@ -0,0 +1,187 @@
# Changelog
## [1.1.22] - 2021-01-21
- Fixed issue where test result of an explicit test was set to skipped in case it was passing and running from command line with testfilter set to the explicit test (DS-1236).
- Fixed an issue where tests located in assemblies that did not directly reference any test assemblies were not included (DSTR-30).
- Fixed an issue where UnitySetup methods were incorrectly being rerun when entering playmode, rather than being skipped (DSTR-68).
- Internal: Remove ##utp message AssemblyCompilationErrors (DS-1277)
- Fixed issue where if the timeout was exceeded in SetUp the timeout exception was not thrown(DSTR-21).
- Removed ability to `Enable playmode tests for all assemblies` from the TestRunner UI, since it is a deprecated behavior. It enforces to use of assembly definition files (DSTR-45).
- Fixed typo in `LogAssert.cs` documentation.
## [1.1.21] - 2021-01-14
- Fixed issue where test result of an explicit test was set to skipped in case it was passing and running from command line with testfilter set to the explicit test (DS-1236).
- Fixed an issue where tests located in assemblies that did not directly reference any test assemblies were not included (DSTR-30).
- Fixed an issue where UnitySetup methods were incorrectly being rerun when entering playmode, rather than being skipped (DSTR-68).
- Internal: Remove ##utp message AssemblyCompilationErrors (DS-1277)
- Fixed issue where if the timeout was exceeded in SetUp the timeout exception was not thrown(DSTR-21).
- Removed ability to `Enable playmode tests for all assemblies` from the TestRunner UI, since it is a deprecated behavior. It enforces to use of assembly definition files (DSTR-45).
## [1.1.20] - 2020-12-04
- The logscope is now available in OneTimeTearDown.
- Fixed an issue where failing tests would not result in the correct exit code if a domain reload happens after the test has run (DS-1304).
- If a player build fails, the test specific build settings should be cleaned up and the original values restored as intended (DS-1001).
- Added better error message when using TestRunCallbackAttribute and the implementation is stripped away (DS-454).
- Fixed an issue where the test results xml would have a zero end-time for tests executed before a domain reload (DSTR-63).
- Fixed OpenSource in case of a Test in a nested class (DSTR-6)
- UnityTests with a domain reload now works correctly in combination with Retry and Repeat attributes (DS-428).
- Fixed OpenSource in case of Tests located inside a package (DS-432)
## [1.1.19] - 2020-11-17
- Command line runs with an inconclusive test result now exit with exit code 2 (case DS-951).
- Fixed timeout during UnitySetUp which caoused test to pass instead of failing due to wrong time format.
- Timeout exeption thrown when timeout time is exeded in the UnitySetup when using `WaitForSeconds(n)`.
- Updating `com.unity.ext.nunit` version
- Method marked with UnityTest that are not returning IEnumerator is now giving a proper error (DS-1059).
## [1.1.18] - 2020-10-07
- Fixed issue of timeout during UnitySetUp which wasn't detected and allowed the test to pass instead of failing (case DSTR-21)
## [1.1.17] - 2020-10-05
- Fixed an issue where the WaitForDomainReload yield instruction would sometimes let the test continue for one frame before the domain reload.
- Added support for negation in filters using !. E.g. !CategoryToExclude.
- Fixed an issue where if the first test enters PlayMode from UnitySetup then the test body will not run on consecutive runs (case 1260901).
- Clear Results button clears the test results in the GUI (DSTR-16)
- Improved UI in Test Runner window, added new options:
- Run Selected Tests in player
- Build/Export project with all tests in player
- Build/Export project with selected tests in player
- Fixed issue on loading EditMode or Playmode test tree in the wrong tab when switching between tabs when TestRunner is loading (DS-865)
## [1.1.16] - 2020-07-09
- Follow up on fix when UTF picks up on outdated compilation errors
## [1.1.15] - 2020-07-02
- Fixed an issue where an exception is thrown on getting the enumerator of a UnityTest would result in stopping the test run instead of failing it (case 1212000).
- Including a trailing semi-colon in a testName filter no longer results in all tests being run (case 1171200).
- Fixed and issue when Unity Test Framework exits editor on an outdated script compilation error (during api updates)
## [1.1.14] - 2020-04-03
- Added the 'assemblyNames' command line argument for filtering on the assembly level.
- The dll and project level of the tree view should now correctly show the results when running tests in a player (case 1197026).
- Optimize usage of player connection when transfering test results (case 1229200).
- Ignore internal test framework tests assertions (case 1206961).
## [1.1.13] - 2020-03-16
- Fixed an issue where a combination of Entering / Exiting playmode and recompiling scripts would result in the test run repeating (case 1213958).
- Fixed a regression from 1.1.12 where prefabs left in the scene would be cleaned up to aggressively.
- Fixed Test execution timed out. No activity received from the player in 600 seconds error when player is not supposed to start (case 1225147)
## [1.1.12] - 2020-03-02
- Now 'Open error line' for a failed UTF test does not throw exceptions for corrupted testable pdb in Editor release mode (case 1118259)
- Fixed an issue where running a test fixture would also run other fixtures with the same full name (namespace plus classname) in other assemblies (case 1197385).
- Running tests with the same full name, with a domain reload inbetween, will no longer fail to initialize the fixture of the second class (case 1205240).
- Running a playmode tests with "Maximize on Play" will now correctly show the result of the tests in the test runner window (case 1014908).
- Fixed an issue where leaving a game object in a scene with a DontSaveInEditor hideFlags would result in an error on cleanup (case 1136883).
- Now ITestPlayerBuildModifier.ModifyOptions is called as expected when running tests on a device (case 1213845)
## [1.1.11] - 2020-01-16
- Fixed test runner dlls got included into player build (case 1211624)
- Passing a non-full-path of XML file for -testResults in Unity Batchmode issue resolved, now passing "result.xml" creates the result file in the project file directory (case 959078)
- Respect Script Debugging build setting when running tests
## [1.1.10] - 2019-12-19
- Introduced PostSuccessfulLaunchAction callback
- Fixed an issue where canceling a UnityTest while it was running would incorrectly mark it as passed instead of canceled.
- Added command line argument for running tests synchronously.
- The test search bar now handles null values correctly.
- The test output pane now retains its size on domain reloads.
## [1.1.9] - 2019-12-12
- Rolled back refactoring to the test run system, as it caused issues in some corner cases.
## [1.1.8] - 2019-11-15
- Ensured that a resumed test run is continued instantly.
## [1.1.7] - 2019-11-14
- Fixed an issue with test runs after domain reload.
## [1.1.6] - 2019-11-12
- Building a player for test will no longer look in unrelated assemblies for relevant attributes.
## [1.1.5] - 2019-10-23
- Fixed a regression to synchronous runs introduced in 1.1.4.
## [1.1.4] - 2019-10-15
- Running tests in batch mode now correctly returns error code 3 (RunError) when a timeout or a build error occurs.
- Fixed an issue where a test run in a player would time out, if the player takes longer than 10 minutes to run.
- Added command line argument and api setting for specifying custom heartbeat timeout for running on players.
## [1.1.3] - 2019-09-23
- Fixed a regression where tests in a player would report a timeout after a test run is finished.
- Made it possible for the ui to change its test items when the test tree changes without script compilation.
- Added synchronous runs as an option to the TestRunnerApi.
## [1.1.2] - 2019-09-11
- Fixed an issue where Run Selected would run all tests in the category, if a category filter was selected, regardless of what tests were selected.
- Unsupported attributes used in UnityTests now give an explicit error.
- Added support for the Repeat and Retry attributes in UnityTests (case 1131940).
- Tests with a explicit timeout higher than 10 minutes, no longer times out after running longer than 10 minutes when running from command line (case 1125991).
- Fixed a performance regression in the test runner api result reporting, introduced in 2018.3 (case 1109865).
- Fixed an issue where parameterized test fixtures would not run if selected in the test tree (case 1092244).
- Pressing Clear Results now also correctly clears the counters on the test list (case 1181763).
- Prebuild setup now handles errors logged with Debug.LogError and stops the run if any is logged (case 1115240). It now also supports LogAssert.Expect.
## [1.1.1] - 2019-08-07
- Tests retrieved as a test list with the test runner api incorrectly showed both mode as their TestMode.
- Fixed a compatibility issue with running tests from rider.
## [1.1.0] - 2019-07-30
- Introduced the TestRunnerApi for running tests programmatically from elsewhere inside the Editor.
- Introduced yield instructions for recompiling scripts and awaiting a domain reload in Edit Mode tests.
- Added a button to the Test Runner UI for clearing the results.
## [1.0.18] - 2019-07-15
- Included new full documentation of the test framework.
## [1.0.17] - 2019-07-11
- Fixed an issue where the Test Runner window wouldnt frame selected items after search filter is cleared.
- Fixed a regression where playmode test application on the IOS platform would not quit after the tests are done.
## [1.0.16] - 2019-06-20
- Fixed an issue where the Test Runner window popped out if it was docked, or if something else was docked next to it, when re-opened (case 1158961)
- Fixed a regression where the running standalone playmode tests from the ui would result in an error.
## [1.0.15] - 2019-06-18
- Added new `[TestMustExpectAllLogs]` attribute, which automatically does `LogAssert.NoUnexpectedReceived()` at the end of affected tests. See docs for this attribute for more info on usage.
- Fixed a regression where no tests would be run if multiple filters are specified. E.g. selecting both a whole assembly and an individual test in the ui.
- Fixed an issue where performing `Run Selected` on a selected assembly would run all assemblies.
- Introduced the capability to do a split build and run, when running playmode tests on standalone devices.
- Fixed an error in ConditionalIgnore, if the condition were not set.
## [1.0.14] - 2019-05-27
- Fixed issue preventing scene creation in IPrebuildSetup.Setup callback when running standalone playmode tests.
- Fixed an issue where test assemblies would sometimes not be ordered alphabetically.
- Added module references to the package for the required modules: imgui and jsonserialize.
- Added a ConditionalIgnore attribute to help ignoring tests only under specific conditions.
- Fixed a typo in the player test window (case 1148671).
## [1.0.13] - 2019-05-07
- Fixed a regression where results from the player would no longer update correctly in the UI (case 1151147).
## [1.0.12] - 2019-04-16
- Added specific unity release to the package information.
## [1.0.11] - 2019-04-10
- Fixed a regression from 1.0.10 where test-started events were triggered multiple times after a domain reload.
## [1.0.10] - 2019-04-08
- Fixed an issue where test-started events would not be fired correctly after a test performing a domain reload (case 1141530).
- The UI should correctly run tests inside a nested class, when that class is selected.
- All actions should now correctly display a prefix when reporting test result. E.g. "TearDown :".
- Errors logged with Debug.LogError in TearDowns now append the error, rather than overwriting the existing result (case 1114306).
- Incorrect implementations of IWrapTestMethod and IWrapSetUpTearDown now gives a meaningful error.
- Fixed a regression where the Test Framework would run TearDown in a base class before the inheriting class (case 1142553).
- Fixed a regression introduced in 1.0.9 where tests with the Explicit attribute could no longer be executed.
## [1.0.9] - 2019-03-27
- Fixed an issue where a corrupt instance of the test runner window would block for a new being opened.
- Added the required modules to the list of package requirements.
- Fixed an issue where errors would happen if the test filter ui was clicked before the ui is done loading.
- Fix selecting items with duplicate names in test hierarchy of Test Runner window (case 987587).
- Fixed RecompileScripts instruction which we use in tests (case 1128994).
- Fixed an issue where using multiple filters on tests would sometimes give an incorrect result.
## [1.0.7] - 2019-03-12
### This is the first release of *Unity Package com.unity.test-framework*.
- Migrated the test-framework from the current extension in unity.

View file

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: d691174143fd3774ba63d7c493633b99
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,9 @@
# Contributing
## If you are interested in contributing, here are some ground rules:
* ... Define guidelines & rules for what contributors need to know to successfully make Pull requests against your repo ...
## All contributions are subject to the [Unity Contribution Agreement(UCA)](https://unity3d.com/legal/licenses/Unity_Contribution_Agreement)
By making a pull request, you are confirming agreement to the terms and conditions of the UCA, including that your Contributions are your original creation and that you have complete right and authority to make your Contributions.
## Once you have a change ready following these ground rules. Simply make a pull request

View file

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 57d2ac5c7d5786e499d4794973fe0d4e
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,66 @@
* [Unity Test Framework overview](./index.md)
* [Edit Mode vs. Play Mode tests](./edit-mode-vs-play-mode-tests.md)
* [Getting started with UTF](./getting-started.md)
* [How to create a new test assembly](./workflow-create-test-assembly.md)
* [How to create a test](./workflow-create-test.md)
* [How to run a test](./workflow-run-test.md)
* [How to create a Play Mode test](./workflow-create-playmode-test.md)
* [How to run a Play Mode test as standalone](./workflow-run-playmode-test-standalone.md)
* [Resources](./resources.md)
* [Extending UTF](./extending.md)
* [How to split the build and run process for standalone Play Mode tests](./reference-attribute-testplayerbuildmodifier.md#split-build-and-run-for-player-mode-tests)
* [How to run tests programmatically](./extension-run-tests.md)
* [How to get test results](./extension-get-test-results.md)
* [How to retrieve the list of tests](./extension-retrieve-test-list.md)
* [Reference](./manual.md#reference)
* [Running tests from the command-line](./reference-command-line.md)
* [UnityTest attribute](./reference-attribute-unitytest.md)
* [Setup and cleanup at build time](./reference-setup-and-cleanup.md)
* [IPrebuildSetup](./reference-setup-and-cleanup.md#iprebuildsetup)
* [IPostBuildCleanup](./reference-setup-and-cleanup.md#ipostbuildcleanup)
* [Actions outside of tests](./reference-actions-outside-tests.md)
* [Action execution order](./reference-actions-outside-tests.md#action-execution-order)
* [UnitySetUp and UnityTearDown](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [OuterUnityTestAction](./reference-actions-outside-tests.md#outerunitytestaction)
* [Domain Reloads](./reference-actions-outside-tests.md#domain-reloads)
* [Custom attributes](./reference-custom-attributes.md)
* [ConditionalIgnore attribute](./reference-attribute-conditionalignore.md)
* [PostBuildCleanup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [PrebuildSetup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [TestMustExpectAllLogs attribute](./reference-attribute-testmustexpectalllogs.md)
* [TestPlayerBuildModifier attribute](./reference-attribute-testplayerbuildmodifier.md)
* [TestRunCallback attribute](./reference-attribute-testruncallback.md)
* [UnityPlatform attribute](./reference-attribute-unityplatform.md)
* [UnitySetUp attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTearDown attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTest attribute](./reference-attribute-unitytest.md)
* [Custom equality comparers](./reference-custom-equality-comparers.md)
* [ColorEqualityComparer](./reference-comparer-color.md)
* [FloatEqualityComparer](./reference-comparer-float.md)
* [QuaternionEqualityComparer](./reference-comparer-quaternion.md)
* [Vector2EqualityComparer](./reference-comparer-vector2.md)
* [Vector3EqualityComparer](./reference-comparer-vector3.md)
* [Vector4EqualityComparer](./reference-comparer-vector4.md)
* [Custom equality comparers with equals operator](./reference-comparer-equals.md)
* [Test Utils](./reference-test-utils.md)
* [Custom yield instructions](./reference-custom-yield-instructions.md)
* [IEditModeTestYieldInstruction](./reference-custom-yield-instructions.md#IEditModeTestYieldInstruction)
* [EnterPlayMode](./reference-custom-yield-instructions.md#enterplaymode)
* [ExitPlayMode](./reference-custom-yield-instructions.md#exitplaymode)
* [RecompileScripts](./reference-recompile-scripts.md)
* [WaitForDomainReload](./reference-wait-for-domain-reload.md)
* [Custom assertion](./reference-custom-assertion.md)
* [LogAssert](./reference-custom-assertion.md#logassert)
* [Custom constraints](./reference-custom-constraints.md)
* [Is](./reference-custom-constraints.md#is)
* [Parameterized tests](./reference-tests-parameterized.md)
* [MonoBehaviour tests](./reference-tests-monobehaviour.md)
* [MonoBehaviourTest<T>](./reference-tests-monobehaviour.md#monobehaviourtestt)
* [IMonoBehaviourTest](./reference-tests-monobehaviour.md#imonobehaviourtest)
* [TestRunnerApi](./reference-test-runner-api.md)
* [ExecutionSettings](./reference-execution-settings.md)
* [Filter](./reference-filter.md)
* [ITestRunSettings](./reference-itest-run-settings.md)
* [ICallbacks](./reference-icallbacks.md)
* [IErrorCallbacks](./reference-ierror-callbacks.md)

View file

@ -0,0 +1,53 @@
# Edit Mode vs. Play Mode tests
Lets clarify a bit what Play Mode and Edit Mode test means from the Unity Test Framework perspective:
## Edit Mode tests
**Edit Mode** tests (also known as Editor tests) are only run in the Unity Editor and have access to the Editor code in addition to the game code.
With Edit Mode tests it is possible to test any of your [Editor extensions](https://docs.unity3d.com/Manual/ExtendingTheEditor.html) using the [UnityTest](./reference-attribute-unitytest.md) attribute. For Edit Mode tests, your test code runs in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
> **Note**: You can also control entering and exiting Play Mode from your Edit Mode test. This allow your test to make changes before entering Play Mode.
Edit Mode tests should meet one of the following conditions:
* They should have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll* and has only the Editor as a target platform:
```assembly
"includePlatforms": [
"Editor"
],
```
* Legacy condition: put tests in the projects [Editor](https://docs.unity3d.com/Manual/SpecialFolders.html) folder.
## Play Mode tests
You can run **Play Mode** tests as a [standalone in a Player](./workflow-run-playmode-test-standalone.md) or inside the Editor. Play Mode tests allow you to exercise your game code, as the tests run as [coroutines](https://docs.unity3d.com/ScriptReference/Coroutine.html) if marked with the `UnityTest` attribute.
Play Mode tests should correspond to the following conditions:
* Have an [assembly definition](./workflow-create-test-assembly.md) with reference to *nunit.framework.dll*.
* Have the test scripts located in a folder with the .asmdef file.
* The test assembly should reference an assembly within the code that you need to test.
```assembly
"references": [
"NewAssembly"
],
"optionalUnityReferences": [
"TestAssemblies"
],
"includePlatforms": [],
```
## Recommendations
### Attributes
Use the [NUnit](http://www.nunit.org/) `Test` attribute instead of the `UnityTest` attribute, unless you need to [yield special instructions](./reference-custom-yield-instructions.md), in Edit Mode, or if you need to skip a frame or wait for a certain amount of time in Play Mode.
### References
It is possible for your Test Assemblies to reference the test tools in `UnityEngine.TestRunner` and `UnityEditor.TestRunner`. The latter is only available in Edit Mode. You can specify these references in the `Assembly Definition References` on the Assembly Definition.

View file

@ -0,0 +1,10 @@
# Extending Unity Test Framework
It is possible to extend the Unity Test Framework (UTF) in many ways, for custom workflows for your projects and for other packages to build on top of UTF.
These extensions are a supplement to the ones already offered by [NUnit](https://github.com/nunit/docs/wiki/Framework-Extensibility).
Some workflows for extending UTF include:
* [How to split the build and run process for standalone Play Mode tests](./reference-attribute-testplayerbuildmodifier.md#split-build-and-run-for-player-mode-tests)
* [How to run tests programmatically](./extension-run-tests.md)
* [How to get test results](./extension-get-test-results.md)
* [How to retrieve the list of tests](./extension-retrieve-test-list.md) 

View file

@ -0,0 +1,45 @@
# How to get test results
You can receive callbacks when the active test run, or individual tests, starts and finishes. You can register callbacks by invoking `RegisterCallbacks` on the [TestRunnerApi](./reference-test-runner-api.md) with an instance of a class that implements [ICallbacks](./reference-icallbacks.md). There are four `ICallbacks` methods for the start and finish of both the whole run and each level of the test tree.
## Example
An example of how listeners can be set up:
> **Note**: Listeners receive callbacks from all test runs, regardless of the registered `TestRunnerApi` for that instance.
``` C#
public void SetupListeners()
{
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(new MyCallbacks());
}
private class MyCallbacks : ICallbacks
{
public void RunStarted(ITestAdaptor testsToRun)
{
}
public void RunFinished(ITestResultAdaptor result)
{
}
public void TestStarted(ITestAdaptor test)
{
}
public void TestFinished(ITestResultAdaptor result)
{
if (!result.HasChildren && result.ResultState != "Success")
{
Debug.Log(string.Format("Test {0} {1}", result.Test.Name, result.ResultState));
}
}
}
```
> **Note**: The registered callbacks are not persisted on domain reloads. So it is necessary to re-register the callback after a domain reloads, usually with [InitializeOnLoad](https://docs.unity3d.com/Manual/RunningEditorCodeOnLaunch.html).
It is possible to provide a `priority` as an integer as the second argument when registering a callback. This influences the invocation order of different callbacks. The default value is zero. It is also possible to provide `RegisterCallbacks` with a class instance that implements [IErrorCallbacks](./reference-ierror-callbacks.md) that is an extended version of `ICallbacks`. `IErrorCallbacks` also has a callback method for `OnError` that invokes if the run fails to start, for example, due to compilation errors or if an [IPrebuildSetup](./reference-setup-and-cleanup.md) throws an exception.

View file

@ -0,0 +1,13 @@
# How to retrieve the list of tests
It is possible to use the [TestRunnerApi](./reference-test-runner-api.md) to retrieve the test tree for a given test mode (**Edit Mode** or **Play Mode**). You can retrieve the test tree by invoking `RetrieveTestList` with the desired `TestMode` and a callback action, with an [ITestAdaptor](./reference-itest-adaptor.md) representing the test tree.
## Example
The following example retrieves the test tree for Edit Mode tests and prints the number of total test cases:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RetrieveTestList(TestMode.EditMode, (testRoot) =>
{
Debug.Log(string.Format("Tree contains {0} tests.", testRoot.TestCaseCount));
});
```

View file

@ -0,0 +1,72 @@
# How to run tests programmatically
## Filters
Run tests by calling `Execute` on the [TestRunnerApi](./reference-test-runner-api.md), and provide some execution settings that consists of a [Filter](./reference-filter.md). The `Filter` specifies what tests to run.
### Example
The following is an example of how to run all **Play Mode** tests in a project:
``` C#
var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
var filter = new Filter()
{
testMode = TestMode.PlayMode
};
testRunnerApi.Execute(new ExecutionSettings(filter));
```
## Multiple filter values
It is possible to specify a more specific filter by filling out the fields on the `Filter` class in more detail.
Many of the fields allow for multiple values. The runner tries to match tests against at least one of the values provided and then runs any tests that match.
### Example
In this example, the API runs tests with full names that fit either of the two names provided:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.Execute(new ExecutionSettings(new Filter()
{
testNames = new[] {"MyTestClass.NameOfMyTest", "SpecificTestFixture.NameOfAnotherTest"}
}));
```
## Multiple filter fields
If using multiple different fields on the filter, then it matches against tests that fulfill all the different fields.
### Example
In this example, it runs any test that fits either of the two test names, and that also belongs to a test assembly that fits the given name.
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.Execute(new ExecutionSettings(new Filter()
{
assemblyNames = new [] {"MyTestAssembly"},
testNames = new [] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}));
```
## Multiple constructor filters
The execution settings take one or more filters in its constructor. If there is no filter provided, then it runs all **Edit Mode** tests by default. If there are multiple filters provided, then a test runs if it matches any of the filters.
### Example
In this example, it runs any tests that are either in the assembly named `MyTestAssembly` or if the full name of the test matches either of the two provided test names:
``` C#
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.Execute(new ExecutionSettings(
new Filter()
{
assemblyNames = new[] {"MyTestAssembly"},
},
new Filter()
{
testNames = new[] {"MyTestClass.NameOfMyTest", "MyTestClass.AnotherNameOfATest"}
}
));
```
> **Note**: Specifying different test modes or platforms in each `Filter` is not currently supported. The test mode and platform is from the first `Filter` only and defaults to Edit Mode, if not supplied.

View file

@ -0,0 +1,18 @@
# Getting started with Unity Test Framework
To access the Unity Test Framework (UTF) in the Unity Editor, open the **Test Runner** window; go to **Window** > **General** > **Test Runner**.
![Unity Test Runner window](./images/test-runner-window.png)
To get started with UTF, follow the workflows below:
* [How to create a new test assembly](./workflow-create-test-assembly.md)
* [How to create a test](./workflow-create-test.md)
* [How to run a test](./workflow-run-test.md)
* [How to create a Play Mode test](./workflow-create-playmode-test.md)
* [How to run a Play Mode test as standalone](./workflow-run-playmode-test-standalone.md)
For further information, see the [resources](./resources.md) and [reference](./manual.md#reference) sections.

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 41 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 82 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

View file

@ -0,0 +1,54 @@
# About Unity Test Framework
The Unity Test Framework (UTF) enables Unity users to test their code in both **Edit Mode** and **Play Mode**, and also on target platforms such as [Standalone](https://docs.unity3d.com/Manual/Standalone.html), Android, iOS, etc.
This package provides a standard test framework for users of Unity and developers at Unity so that both benefit from the same features and can write tests the same way.
UTF uses a Unity integration of NUnit library, which is an open-source unit testing library for .Net languages. For more information about NUnit, see the [official NUnit website](http://www.nunit.org/) and the [NUnit documentation on GitHub](https://github.com/nunit/docs/wiki/NUnit-Documentation).
> **Note**: UTF is not a new concept or toolset; it is an adjusted and more descriptive naming for the toolset otherwise known as Unity Test Runner, which is now available as this package.
# Installing Unity Test Framework
To install this package, follow the instructions in the [Package Manager documentation](https://docs.unity3d.com/Packages/com.unity.package-manager-ui@latest/index.html).
> **Note**: Search for the Test Framework package. In Unity 2019.2 and higher, you may need to enable the package before use.
# Using Unity Test Framework
To learn how to use the Unity Test Framework package in your project, read the [manual](./manual.md).
# Technical details
## Requirements
This version of the Unity Test Framework is compatible with the following versions of the Unity Editor:
* 2019.2 and later.
## Known limitations
Unity Test Framework version 1.0.18 includes the following known limitations:
* The `UnityTest` attribute does not support WebGL and WSA platforms.
* The `UnityTest` attribute does not support [Parameterized tests](https://github.com/nunit/docs/wiki/Parameterized-Tests) (except for `ValueSource`).
* The `UnityTest` attribute does not support the `NUnit` [Repeat](https://github.com/nunit/docs/wiki/Repeat-Attribute) attribute.
* Nested test fixture cannot run from the Editor UI.
* When using the `NUnit` [Retry](https://github.com/nunit/docs/wiki/Retry-Attribute) attribute in PlayMode tests, it throws `InvalidCastException`.
## Package contents
The following table indicates the root folders in the package where you can find useful resources:
| Location | Description |
| :----------------------------------------- | :------------------------------------------ |
| _/com.unity.test-framework/Documentation~_ | Contains the documentation for the package. |
## Document revision history
| Date | Reason |
| :----------- | :---------------------------------------------------- |
| August 23, 2019 | Applied feedback to the documentation |
| July 25, 2019 | Documentation updated to include features in version 1.1.0 |
| July 11, 2019 | Documentation updated. Matches package version 1.0.18 |
| May 27, 2019 | Documentation created. Matches package version 1.0.14 |

View file

@ -0,0 +1,80 @@
# Unity Test Framework manual
This is the manual for the Unity Test Framework (UTF):
## **Introduction**
* [Unity Test Framework overview](./index.md)
* [Edit Mode vs. Play Mode tests](edit-mode-vs-play-mode-tests.md)
## **Getting started**
* [Getting started with UTF](./getting-started.md)
* Workflows:
* [How to create a new test assembly](./workflow-create-test-assembly.md)
* [How to create a test](./workflow-create-test.md)
* [How to run a test](workflow-run-test.md)
* [How to create a Play Mode test](./workflow-create-playmode-test.md)
* [How to run a Play Mode test in player](./workflow-run-playmode-test-standalone.md)
* [Resources](./resources.md)
## Extending UTF
* [Extending UTF](./extending.md)
* Workflows:
* [How to split the build and run process for standalone Play Mode tests](./reference-attribute-testplayerbuildmodifier.md#split-build-and-run)
* [How to run tests programmatically](./extension-run-tests.md)
* [How to get test results](./extension-get-test-results.md)
* [How to retrieve the list of tests](./extension-retrieve-test-list.md)
## Reference
* [Running tests from the command-line](./reference-command-line.md)
* [UnityTest attribute](./reference-attribute-unitytest.md)
* [Setup and cleanup at build time](./reference-setup-and-cleanup.md)
* [IPrebuildSetup](./reference-setup-and-cleanup.md#iprebuildsetup)
* [IPostBuildCleanup](./reference-setup-and-cleanup.md#ipostbuildcleanup)
* [Actions outside of tests](./reference-actions-outside-tests.md)
* [Action execution order](./reference-actions-outside-tests.md#action-execution-order)
* [UnitySetUp and UnityTearDown](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [OuterUnityTestAction](./reference-actions-outside-tests.md#outerunitytestaction)
* [Domain Reloads](./reference-actions-outside-tests.md#domain-reloads)
* [Custom attributes](./reference-custom-attributes.md)
* [ConditionalIgnore attribute](./reference-attribute-conditionalignore.md)
* [PostBuildCleanup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [PrebuildSetup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [TestMustExpectAllLogs attribute](./reference-attribute-testmustexpectalllogs.md)
* [TestPlayerBuildModifier attribute](./reference-attribute-testplayerbuildmodifier.md)
* [TestRunCallback attribute](./reference-attribute-testruncallback.md)
* [UnityPlatform attribute](./reference-attribute-unityplatform.md)
* [UnitySetUp attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTearDown attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTest attribute](./reference-attribute-unitytest.md)
* [Custom equality comparers](./reference-custom-equality-comparers.md)
* [ColorEqualityComparer](./reference-comparer-color.md)
* [FloatEqualityComparer](./reference-comparer-float.md)
* [QuaternionEqualityComparer](./reference-comparer-quaternion.md)
* [Vector2EqualityComparer](./reference-comparer-vector2.md)
* [Vector3EqualityComparer](./reference-comparer-vector3.md)
* [Vector4EqualityComparer](./reference-comparer-vector4.md)
* [Custom equality comparers with equals operator](./reference-comparer-equals.md)
* [Test Utils](./reference-test-utils.md)
* [Custom yield instructions](./reference-custom-yield-instructions.md)
* [IEditModeTestYieldInstruction](./reference-custom-yield-instructions.md#IEditModeTestYieldInstruction)
* [EnterPlayMode](./reference-custom-yield-instructions.md#enterplaymode)
* [ExitPlayMode](./reference-custom-yield-instructions.md#exitplaymode)
* [Custom assertion](./reference-custom-assertion.md)
* [LogAssert](./reference-custom-assertion.md#logassert)
* [Custom constraints](./reference-custom-constraints.md)
* [Is](./reference-custom-constraints.md#is)
* [Parameterized tests](./reference-tests-parameterized.md)
* [MonoBehaviour tests](./reference-tests-monobehaviour.md)
* [MonoBehaviourTest<T>](./reference-tests-monobehaviour.md#monobehaviourtestt)
* [IMonoBehaviourTest](./reference-tests-monobehaviour.md#imonobehaviourtest)
* [TestRunnerApi](./reference-test-runner-api.md)
* [ExecutionSettings](./reference-execution-settings.md)
* [Filter](./reference-filter.md)
* [ITestRunSettings](./reference-itest-run-settings.md)
* [ICallbacks](./reference-icallbacks.md)
* [IErrorCallbacks](./reference-ierror-callbacks.md)

View file

@ -0,0 +1,98 @@
# Actions outside of tests
When writing tests, it is possible to avoid duplication of code by using the [SetUp and TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) methods built into [NUnit](http://www.nunit.org/). The Unity Test Framework has extended these methods with extra functionality, which can yield commands and skip frames, in the same way as [UnityTest](./reference-attribute-unitytest.md).
## Action execution order
The actions related to a test run in the following order:
* Attributes implementing [IApplyToContext](https://github.com/nunit/docs/wiki/IApplyToContext-Interface)
* Any attribute implementing [OuterUnityTestAction](#outerunitytestaction) has its `BeforeTest` invoked
* Tests with [UnitySetUp](#unitysetup-and-unityteardown) methods in their test class.
* Attributes implementing [IWrapSetUpTearDown](https://github.com/nunit/docs/wiki/ICommandWrapper-Interface)
* Any [SetUp](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attributes
* [Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `BeforeTest` method invoked
* Attributes implementing of [IWrapTestMethod](https://github.com/nunit/docs/wiki/ICommandWrapper-Interface)
* **The test itself runs**
* [Action attributes](https://nunit.org/docs/2.6/actionAttributes.html) have their `AfterTest` method invoked
* Any method with the [TearDown](https://github.com/nunit/docs/wiki/SetUp-and-TearDown) attribute
* Tests with [UnityTearDown](#unitysetup-and-unityteardown) methods in their test class
* Any [OuterUnityTestAction](#outerunitytestaction) has its `AfterTest` invoked
The list of actions is the same for both `Test` and `UnityTest`.
## UnitySetUp and UnityTearDown
The `UnitySetUp` and `UnityTearDown` attributes are identical to the standard `SetUp` and `TearDown` attributes, with the exception that they allow for [yielding instructions](reference-custom-yield-instructions.md). The `UnitySetUp` and `UnityTearDown` attributes expect a return type of [IEnumerator](https://docs.microsoft.com/en-us/dotnet/api/system.collections.ienumerator?view=netframework-4.8).
### Example
```c#
public class SetUpTearDownExample
{
[UnitySetUp]
public IEnumerator SetUp()
{
yield return new EnterPlayMode();
}
[Test]
public void MyTest()
{
Debug.Log("This runs inside playmode");
}
[UnityTearDown]
public IEnumerator TearDown()
{
yield return new ExitPlayMode();
}
}
```
## OuterUnityTestAction
`OuterUnityTestAction` is a wrapper outside of the tests, which allows for any tests with this attribute to run code before and after the tests. This method allows for yielding commands in the same way as `UnityTest`. The attribute must inherit the `NUnit` attribute and implement `IOuterUnityTestAction`.
### Example
```c#
using System.Collections;
using NUnit.Framework;
using NUnit.Framework.Interfaces;
using UnityEngine;
using UnityEngine.TestTools;
public class MyTestClass
{
[UnityTest, MyOuterActionAttribute]
public IEnumerator MyTestInsidePlaymode()
{
Assert.IsTrue(Application.isPlaying);
yield return null;
}
}
public class MyOuterActionAttribute : NUnitAttribute, IOuterUnityTestAction
{
public IEnumerator BeforeTest(ITest test)
{
yield return new EnterPlayMode();
}
public IEnumerator AfterTest(ITest test)
{
yield return new ExitPlayMode();
}
}
```
## Domain Reloads
In **Edit Mode** tests it is possible to yield instructions that can result in a domain reload, such as entering or exiting **Play Mode** (see [Custom yield instructions](./reference-custom-yield-instructions.md)). When a domain reload happens, all non-Unity actions (such as `OneTimeSetup` and `Setup`) are rerun before the code, which initiated the domain reload, continues. Unity actions (such as `UnitySetup`) are not rerun. If the Unity action is the code that initiated the domain reload, then the rest of the code in the `UnitySetup` method runs after the domain reload.

View file

@ -0,0 +1,39 @@
# ConditionalIgnore attribute
This attribute is an alternative to the standard `Ignore` attribute in [NUnit](http://www.nunit.org/). It allows for ignoring tests only under a specified condition. The condition evaluates during `OnLoad`, referenced by ID.
## Example
The following example shows a method to use the `ConditionalIgnore` attribute to ignore a test if the Unity Editor is running macOS:
```C#
using UnityEditor;
using NUnit.Framework;
using UnityEngine.TestTools;
[InitializeOnLoad]
public class OnLoad
{
static OnLoad()
{
var editorIsOSX = false;
#if UNITY_EDITOR_OSX
editorIsOSX = true;
#endif
ConditionalIgnoreAttribute.AddConditionalIgnoreMapping("IgnoreInMacEditor", editorIsOSX);
}
}
public class MyTestClass
{
[Test, ConditionalIgnore("IgnoreInMacEditor", "Ignored on Mac editor.")]
public void TestNeverRunningInMacEditor()
{
Assert.Pass();
}
}
```
> **Note**: You can only use `InitializeOnLoad` in **Edit Mode** tests.

View file

@ -0,0 +1,11 @@
# TestMustExpectAllLogs attribute
The presence of this attribute causes the **Test Runner** to expect every single log. By default, the Test Runner only fails on error logs, but `TestMustExpectAllLogs` fails on warnings and info level messages as well. It is the same as calling the method [LogAssert.NoUnexpectedReceived](./reference-custom-assertion.md#static-methods) at the bottom of every affected test.
## Assembly-wide usage
You can apply this attribute to test assemblies (that affects every test in the assembly), fixtures (affects every test in the fixture), or on individual test methods. It is also inherited from base fixtures.
The `MustExpect` property (`true` by default) lets you enable or disable the higher level value.
For example when migrating an assembly to this more strict checking method, you might attach `[assembly:TestMustExpectAllLogs]` to the assembly itself, but then whitelist failing fixtures and test methods with `[TestMustExpectAllLogs(MustExpect=false)]` until you have migrated them. This also means new tests in that assembly would have the more strict checking.

View file

@ -0,0 +1,105 @@
# TestPlayerBuildModifier attribute
You can use the `TestPlayerBuildModifier` attribute to accomplish a couple of different scenarios:
## Modify the Player build options for Play Mode tests
It is possible to change the [BuildPlayerOptions](https://docs.unity3d.com/ScriptReference/BuildPlayerOptions.html) for the test **Player**, to achieve custom behavior when running **Play Mode** tests. Modifying the build options allows for changing the target location of the build as well as changing [BuildOptions](https://docs.unity3d.com/ScriptReference/BuildOptions.html).
To modify the `BuildPlayerOptions`, do the following:
* Implement the `ITestPlayerBuildModifier`
* Reference the implementation type in a `TestPlayerBuildModifier` attribute on an assembly level.
### Example
```c#
using UnityEditor;
using UnityEditor.TestTools;
[assembly:TestPlayerBuildModifier(typeof(BuildModifier))]
public class BuildModifier : ITestPlayerBuildModifier
{
public BuildPlayerOptions ModifyOptions(BuildPlayerOptions playerOptions)
{
if (playerOptions.target == BuildTarget.iOS)
{
playerOptions.options |= BuildOptions.SymlinkLibraries; // Enable symlink libraries when running on iOS
}
playerOptions.options |= BuildOptions.AllowDebugging; // Enable allow Debugging flag on the test Player.
return playerOptions;
}
}
```
> **Note:** When building the Player, it includes all `TestPlayerBuildModifier` attributes across all loaded assemblies, independent of the currently used test filter. As the implementation references the `UnityEditor` namespace, the code is typically implemented in an Editor only assembly, as the `UnityEditor` namespace is not available otherwise.
## Split build and run
It is possible to use the Unity Editor for building the Player with tests, without [running the tests](./workflow-run-playmode-test-standalone.md). This allows for running the Player on e.g. another machine. In this case, it is necessary to modify the Player to build and implement a custom handling of the test result.
By using `TestPlayerBuildModifier`, you can alter the `BuildOptions` to not start the Player after the build as well as build the Player at a specific location. Combined with [PostBuildCleanup](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup), you can automatically exit the Editor on completion of the build.
### Example
```c#
using System;
using System.IO;
using System.Linq;
using Tests;
using UnityEditor;
using UnityEditor.TestTools;
using UnityEngine;
using UnityEngine.TestTools;
[assembly:TestPlayerBuildModifier(typeof(HeadlessPlayModeSetup))]
[assembly:PostBuildCleanup(typeof(HeadlessPlayModeSetup))]
namespace Tests
{
public class HeadlessPlayModeSetup : ITestPlayerBuildModifier, IPostBuildCleanup
{
private static bool s_RunningPlayerTests;
public BuildPlayerOptions ModifyOptions(BuildPlayerOptions playerOptions)
{
// Do not launch the player after the build completes.
playerOptions.options &= ~BuildOptions.AutoRunPlayer;
// Set the headlessBuildLocation to the output directory you desire. It does not need to be inside the project.
var headlessBuildLocation = Path.GetFullPath(Path.Combine(Application.dataPath, ".//..//PlayModeTestPlayer"));
var fileName = Path.GetFileName(playerOptions.locationPathName);
if (!string.IsNullOrEmpty(fileName))
{
headlessBuildLocation = Path.Combine(headlessBuildLocation, fileName);
}
playerOptions.locationPathName = headlessBuildLocation;
// Instruct the cleanup to exit the Editor if the run came from the command line.
// The variable is static because the cleanup is being invoked in a new instance of the class.
s_RunningPlayerTests = true;
return playerOptions;
}
public void Cleanup()
{
if (s_RunningPlayerTests && IsRunningTestsFromCommandLine())
{
// Exit the Editor on the next update, allowing for other PostBuildCleanup steps to run.
EditorApplication.update += () => { EditorApplication.Exit(0); };
}
}
private static bool IsRunningTestsFromCommandLine()
{
var commandLineArgs = Environment.GetCommandLineArgs();
return commandLineArgs.Any(value => value == "-runTests");
}
}
}
```
If the Editor is still running after the Play Mode tests have run, the Player tries to report the results back, using [PlayerConnection](https://docs.unity3d.com/ScriptReference/Networking.PlayerConnection.PlayerConnection.html), which has a reference to the IP address of the Editor machine, when built.
To implement a custom way of reporting the results of the test run, let one of the assemblies in the Player include a [TestRunCallback](./reference-attribute-testruncallback.md). At `RunFinished`, it is possible to get the full test report as XML from the [NUnit](http://www.nunit.org/) test result by calling `result.ToXml(true)`. You can save the result and then save it on the device or send it to another machine as needed.

View file

@ -0,0 +1,46 @@
# TestRunCallback attribute
It is possible for the test framework to invoke callbacks as the current test run progresses. To do this, there is a `TestRunCallback` attribute which takes the type of `ITestRunCallback` implementation. You can invoke the callbacks with [NUnit](http://www.nunit.org/) `ITest` and `ITestResult` classes.
At the `RunStarted` and `RunFinished` methods, the test and test results are for the whole test tree. These methods invoke at each node in the test tree; first with the whole test assembly, then with the test class, and last with the test method.
From these callbacks, it is possible to read the partial or the full results, and it is furthermore possible to save the XML version of the result for further processing or continuous integration.
## Example
```C#
using NUnit.Framework.Interfaces;
using UnityEngine;
using UnityEngine.TestRunner;
[assembly:TestRunCallback(typeof(MyTestRunCallback))]
public class MyTestRunCallback : ITestRunCallback
{
public void RunStarted(ITest testsToRun)
{
}
public void RunFinished(ITestResult testResults)
{
}
public void TestStarted(ITest test)
{
}
public void TestFinished(ITestResult result)
{
if (!result.Test.IsSuite)
{
Debug.Log($"Result of {result.Name}: {result.ResultState.Status}");
}
}
}
```
> **Note:** The `TestRunCallback` does not need any references to the `UnityEditor` namespace and is thus able to run in standalone Players, on the **Player** side.

View file

@ -0,0 +1,35 @@
# UnityPlatform attribute
Use this attribute to define a specific set of platforms you want or do not want your test(s) to run on.
You can use this attribute on the test method, test class, or test assembly level. Use the supported [RuntimePlatform](https://docs.unity3d.com/ScriptReference/RuntimePlatform.html) enumeration values to specify the platforms. You can also specify which platforms to test by passing one or more `RuntimePlatform` values along with or without the include or exclude properties as parameters to the [Platform](https://github.com/nunit/docs/wiki/Platform-Attribute) attribute constructor.
The test(s) skips if the current target platform is:
- Not explicitly specified in the included platforms list
- In the excluded platforms list
```c#
using UnityEngine;
using UnityEngine.TestTools;
using NUnit.Framework;
[TestFixture]
public class TestClass
{
[Test]
[UnityPlatform(RuntimePlatform.WindowsPlayer)]
public void TestMethod()
{
Assert.AreEqual(Application.platform, RuntimePlatform.WindowsPlayer);
}
}
```
## Properties
| Syntax | Description |
| --------------------------- | ------------------------------------------------------------ |
| `RuntimePlatform[] exclude` | List the platforms you do not want to have your tests run on. |
| `RuntimePlatform[] include` | A subset of platforms you need to have your tests run on. |

View file

@ -0,0 +1,51 @@
# UnityTest attribute
`UnityTest` attribute is the main addition to the standard [NUnit](http://www.nunit.org/) library for the Unity Test Framework. This type of unit test allows you to skip a frame from within a test (so background tasks can finish) or give certain commands to the Unity **Editor**, such as performing a domain reload or entering **Play Mode** from an **Edit Mode** test.
In Play Mode, the `UnityTest` attribute runs as a [coroutine](https://docs.unity3d.com/Manual/Coroutines.html). Whereas Edit Mode tests run in the [EditorApplication.update](https://docs.unity3d.com/ScriptReference/EditorApplication-update.html) callback loop.
The `UnityTest` attribute is, in fact, an alternative to the `NUnit` [Test attribute](https://github.com/nunit/docs/wiki/Test-Attribute), which allows yielding instructions back to the framework. Once the instruction is complete, the test run continues. If you `yield return null`, you skip a frame. That might be necessary to ensure that some changes do happen on the next iteration of either the `EditorApplication.update` loop or the [game loop](https://docs.unity3d.com/Manual/ExecutionOrder.html).
## Edit Mode example
The most simple example of an Edit Mode test could be the one that yields `null` to skip the current frame and then continues to run:
```C#
[UnityTest]
public IEnumerator EditorUtility_WhenExecuted_ReturnsSuccess()
{
var utility = RunEditorUtilityInTheBackground();
while (utility.isRunning)
{
yield return null;
}
Assert.IsTrue(utility.isSuccess);
}
```
## Play Mode example
In Play Mode, a test runs as a coroutine attached to a [MonoBehaviour](https://docs.unity3d.com/ScriptReference/MonoBehaviour.html). So all the yield instructions available in coroutines, are also available in your test.
From a Play Mode test you can use one of Unitys [Yield Instructions](https://docs.unity3d.com/ScriptReference/YieldInstruction.html):
- [WaitForFixedUpdate](https://docs.unity3d.com/ScriptReference/WaitForFixedUpdate.html): to ensure changes expected within the next cycle of physics calculations.
- [WaitForSeconds](https://docs.unity3d.com/ScriptReference/WaitForSeconds.html): if you want to pause your test coroutine for a fixed amount of time. Be careful about creating long-running tests.
The simplest example is to yield to `WaitForFixedUpdate`:
```c#
[UnityTest]
public IEnumerator GameObject_WithRigidBody_WillBeAffectedByPhysics()
{
var go = new GameObject();
go.AddComponent<Rigidbody>();
var originalPosition = go.transform.position.y;
yield return new WaitForFixedUpdate();
Assert.AreNotEqual(originalPosition, go.transform.position.y);
}
```

View file

@ -0,0 +1,120 @@
# Running tests from the command line
Its pretty simple to run a test project from the command line. Here is an example in Windows:
```bash
Unity.exe -runTests -batchmode -projectPath PATH_TO_YOUR_PROJECT -testResults C:\temp\results.xml -testPlatform PS4
```
For more information, see [Command line arguments](https://docs.unity3d.com/Manual/CommandLineArguments.html).
## Commands
### batchmode
Runs Unity in batch mode and ensures no pop-up windows appear to eliminate the need for any human intervention.
### forgetProjectPath
Don't save your current **Project** into the Unity launcher/hub history.
### runTests
Runs tests in the Project.
### testCategory
A semicolon-separated list of test categories to include in the run. A semi-colon separated list should be formatted as a string enclosed in quotation marks, e.g. `testCategory="firstCategory;secondCategory"`. If using both `testFilter` and `testCategory`, then only tests that match both are run. This argument supports negation using '!'. If using '!MyCategory' then no tests with the 'MyCategory' category will be included in the run.
### testFilter
A semicolon-separated list of test names to run, or a regular expression pattern to match tests by their full name. A semi-colon separated list should be formatted as a string enclosed in quotation marks, e.g. `testFilter="Low;Medium"`. This argument supports negation using '!'. If using the test filter '!MyNamespace.Something.MyTest', then all tests except that test will be run.
### testPlatform
The platform you want to run tests on. Available platforms are **EditMode** and **PlayMode**.
> **Note**: If unspecified, tests run in Edit Mode by default.
Platform/Type convention is from the [BuildTarget](https://docs.unity3d.com/ScriptReference/BuildTarget.html) enum. Supported platforms are:
* StandaloneWindows
* StandaloneWindows64
* StandaloneLinux64
* StandaloneOSX
* iOS
* Android
* PS4
* XboxOne
### assemblyNames
A semicolon-separated list of test assemblies to include in the run. A semi-colon separated list should be formatted as a string enclosed in quotation marks, e.g. `assemblyNames="firstAssembly;secondAssembly"`.
### testResults
The path where Unity should save the result file. By default, Unity saves it in the Projects root folder.
### playerHeartbeatTimeout
The time, in seconds, the editor should wait for heartbeats after starting a test run on a player. This defaults to 10 minutes.
### runSynchronously
If included, the test run will run tests synchronously, guaranteeing that all tests runs in one editor update call. Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. `[UnityTest]` tests, or tests with `[UnitySetUp]` or `[UnityTearDown]` scaffolding) will be filtered out.
### testSettingsFile
Path to a *TestSettings.json* file that allows you to set up extra options for your test run. An example of the *TestSettings.json* file could look like this:
```json
{
"scriptingBackend":2,
"Architecture":null,
"apiProfile":0
}
```
#### apiProfile
The .Net compatibility level. Set to one of the following values:
- 1 - .Net 2.0
- 2 - .Net 2.0 Subset
- 3 - .Net 4.6
- 5 - .Net micro profile (used by Mono scripting backend if **Stripping Level** is set to **Use micro mscorlib**)
- 6 - .Net Standard 2.0
#### appleEnableAutomaticSigning
Sets option for automatic signing of Apple devices.
#### appleDeveloperTeamID
Sets the team ID for the apple developer account.
#### architecture
Target architecture for Android. Set to one of the following values:
* None = 0
* ARMv7 = 1
* ARM64 = 2
* X86 = 4
* All = 4294967295
#### iOSManualProvisioningProfileType
Set to one of the following values:
* 0 - Automatic
* 1 - Development
* 2 - Distribution iOSManualProvisioningProfileID
#### scriptingBackend
Set to one of the following values:
- Mono2x = 0
- IL2CPP = 1
- WinRT DotNET = 2

View file

@ -0,0 +1,47 @@
# ColorEqualityComparer
Use this class to compare two `Color` objects. `ColorEqualityComparer.Instance` has default calculation error value set to 0.01f. To set a test specific error value instantiate a comparer instance using the [one argument constructor](#constructors).
## Static properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `Instance` | A singleton instance of the comparer with a default error value set to 0.01f. |
## Constructors
| Syntax | Description |
| ------------------------------------ | ------------------------------------------------------------ |
| `ColorEqualityComparer(float error)` | Creates an instance of the comparer with a custom error value. |
## Public methods
| Syntax | Description |
| -------------------------------------------- | ------------------------------------------------------------ |
| `bool Equals(Color expected, Color actual);` | Compares the actual and expected `Color` objects for equality using `Utils.AreFloatsEqualAbsoluteError` to compare the `RGB` and `Alpha` attributes of `Color`. Returns `true` if expected and actual objects are equal otherwise, it returns `false`. |
## Example
```c#
[TestFixture]
public class ColorEqualityTest
{
[Test]
public void GivenColorsAreEqual_WithAllowedCalculationError()
{
// Using default error
var firstColor = new Color(0f, 0f, 0f, 0f);
var secondColor = new Color(0f, 0f, 0f, 0f);
Assert.That(firstColor, Is.EqualTo(secondColor).Using(ColorEqualityComparer.Instance));
// Allowed error 10e-5f
var comparer = new ColorEqualityComparer(10e-5f);
firstColor = new Color(0f, 0f, 0f, 1f);
secondColor = new Color(10e-6f, 0f, 0f, 1f);
Assert.That(firstColor, Is.EqualTo(secondColor).Using(comparer));
}
}
```

View file

@ -0,0 +1,27 @@
# Custom equality comparers with equals operator
If you need to compare Vectors using the overloaded operator == (see [Vector2.operator ==](https://docs.unity3d.com/ScriptReference/Vector2-operator_eq.html), [Vector3.operator ==](https://docs.unity3d.com/ScriptReference/Vector3-operator_eq.html), and [Vector4.operator ==](https://docs.unity3d.com/ScriptReference/Vector4-operator_eq.html)) you should use the respective comparer implementations:
- Vector2ComparerWithEqualsOperator
- Vector3ComparerWithEqualsOperator
- Vector4ComparerWithEqualsOperator
The interface is the same as for other [equality comparers](./reference-custom-equality-comparers.md) except the public [constructor](./reference-custom-equality-comparers.md#constructors) `error` parameter is inapplicable in this case.
## Example
```c#
[TestFixture]
public class Vector3Test
{
[Test]
public void VerifyThat_TwoVector3ObjectsAreEqual()
{
var actual = new Vector3(10e-7f, 10e-7f, 10e-7f);
var expected = new Vector3(0f, 0f, 0f);
Assert.That(actual, Is.EqualTo(expected).Using(Vector3ComparerWithEqualsOperator.Instance));
}
}
```

View file

@ -0,0 +1,46 @@
# FloatEqualityComparer
Use this class to compare two float values for equality with [NUnit](http://www.nunit.org/) constraints. Use `FloatEqualityComparer.Instance` comparer to have the default error value set to 0.0001f. For any other error, use the [one argument constructor](#constructors) to create a comparer.
## Static Properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `Instance` | A singleton instance of the comparer with a default error value set to 0.0001f. |
## Constructors
| Syntax | Description |
| ------------------------------------------- | ------------------------------------------------------------ |
| `FloatEqualityComparer(float allowedError)` | Creates an instance of the comparer with a custom error value. |
## Public methods
| Syntax | Description |
| -------------------------------------------- | ------------------------------------------------------------ |
| `bool Equals(float expected, float actual);` | Compares the `actual` and `expected` float values for equality using `Utils.AreFloatsEqual`. |
## Example
```c#
[TestFixture]
public class FloatsTest
{
[Test]
public void VerifyThat_TwoFloatsAreEqual()
{
var comparer = new FloatEqualityComparer(10e-6f);
var actual = -0.00009f;
var expected = 0.00009f;
Assert.That(actual, Is.EqualTo(expected).Using(comparer));
// Default relative error 0.0001f
actual = 10e-8f;
expected = 0f;
Assert.That(actual, Is.EqualTo(expected).Using(FloatEqualityComparer.Instance));
}
}
```

View file

@ -0,0 +1,46 @@
# QuaternionEqualityComparer
Use this utility to compare two [Quaternion](https://docs.unity3d.com/ScriptReference/Quaternion.html) objects for equality with [NUnit](http://www.nunit.org/) assertion constraints. Use the static instance `QuaternionEqualityComparer.Instance` to have the default calculation error value set to 0.00001f. For any other custom error value, use the [one argument constructor](#constructors).
## Static properties
| Syntax | Description |
| ---------- | ---------------------------------------------------------- |
| `Instance` | A comparer instance with the default error value 0.00001f. |
## Constructors
| Syntax | Description |
| ------------------------------------------------ | ------------------------------------------------------------ |
| `QuaternionEqualityComparer(float allowedError)` | Creates an instance of the comparer with a custom allowed error value. |
## Public methods
| Syntax | Description |
| ----------------------------------------------------- | ------------------------------------------------------------ |
| `bool Equals(Quaternion expected, Quaternion actual)` | Compares the `actual` and `expected` `Quaternion` objects for equality using the [Quaternion.Dot](https://docs.unity3d.com/ScriptReference/Quaternion.Dot.html) method. |
## Example
```c#
[TestFixture]
public class QuaternionTest
{
[Test]
public void VerifyThat_TwoQuaternionsAreEqual()
{
var actual = new Quaternion(10f, 0f, 0f, 0f);
var expected = new Quaternion(1f, 10f, 0f, 0f);
var comparer = new QuaternionEqualityComparer(10e-6f);
Assert.That(actual, Is.EqualTo(expected).Using(comparer));
//Using default error 0.00001f
actual = new Quaternion(10f, 0f, 0.1f, 0f);
expected = new Quaternion(1f, 10f, 0.1f, 0f);
Assert.That(actual, Is.EqualTo(expected).Using(QuaternionEqualityComparer.Instance));
}
}
```

View file

@ -0,0 +1,47 @@
# Vector2EqualityComparer
Use this class to compare two [Vector2](https://docs.unity3d.com/ScriptReference/Vector2.html) objects for equality with [NUnit](http://www.nunit.org/) constraints. Use the static `Vector2EqualityComparer.Instance` to have the calculation error value set to default 0.0001f. For any other error value, instantiate a new comparer object with the [one argument constructor](#constructors).
## Static properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `Instance` | A comparer instance with the default error value set to 0.0001f. |
## Constructors
| Syntax | Description |
| -------------------------------------- | ---------------------------------------------- |
| `Vector2EqualityComparer(float error)` | Creates an instance with a custom error value. |
## Public methods
| Syntax | Description |
| ------------------------------------------ | ------------------------------------------------------------ |
| `Equals(Vector2 expected, Vector2 actual)` | Compares the `actual` and `expected` `Vector2` objects for equality using the [Utils.AreFloatsEqual](./reference-test-utils.md) method. |
## Example
```c#
[TestFixture]
public class Vector2Test
{
[Test]
public void VerifyThat_TwoVector2ObjectsAreEqual()
{
// Custom calculation error
var actual = new Vector2(10e-7f, 10e-7f);
var expected = new Vector2(0f, 0f);
var comparer = new Vector2EqualityComparer(10e-6f);
Assert.That(actual, Is.EqualTo(expected).Using(comparer));
//Default error 0.0001f
actual = new Vector2(0.01f, 0.01f);
expected = new Vector2(0.01f, 0.01f);
Assert.That(actual, Is.EqualTo(expected).Using(Vector2EqualityComparer.Instance));
}
}
```

View file

@ -0,0 +1,47 @@
# Vector3EqualityComparer
Use this class to compare two [Vector3](https://docs.unity3d.com/ScriptReference/Vector3.html) objects for equality with `NUnit` constraints. Call `Vector3EqualityComparer.Instance` comparer to perform a comparison with the default calculation error value 0.0001f. To specify a different error value, use the [one argument constructor](#constructors) to instantiate a new comparer.
## Static properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `Instance` | A comparer instance with the default calculation error value equal to 0.0001f. |
## Constructors
| Syntax | Description |
| --------------------------------------------- | ---------------------------------------------- |
| `Vector3EqualityComparer(float allowedError)` | Creates an instance with a custom error value. |
## Public methods
| Syntax | Description |
| ----------------------------------------------- | ------------------------------------------------------------ |
| `bool Equals(Vector3 expected, Vector3 actual)` | Compares the `actual` and `expected` `Vector3` objects for equality using [Utils.AreFloatsEqual](http://todo) to compare the `x`, `y`, and `z` attributes of `Vector3`. |
## Example
```c#
[TestFixture]
public class Vector3Test
{
[Test]
public void VerifyThat_TwoVector3ObjectsAreEqual()
{
// Custom error 10e-6f
var actual = new Vector3(10e-8f, 10e-8f, 10e-8f);
var expected = new Vector3(0f, 0f, 0f);
var comparer = new Vector3EqualityComparer(10e-6f);
Assert.That(actual, Is.EqualTo(expected).Using(comparer));
//Default error 0.0001f
actual = new Vector3(0.01f, 0.01f, 0f);
expected = new Vector3(0.01f, 0.01f, 0f);
Assert.That(actual, Is.EqualTo(expected).Using(Vector3EqualityComparer.Instance));
}
}
```

View file

@ -0,0 +1,47 @@
# Vector4EqualityComparer
Use this class to compare two [Vector4](https://docs.unity3d.com/ScriptReference/Vector4.html) objects for equality with [NUnit](http://www.nunit.org/) constraints. Call `Vector4EqualityComparer.Instance` to perform comparisons using default calculation error value 0.0001f. To set a custom test value, instantiate a new comparer using the [one argument constructor](#constructor).
## Static Properties
| Syntax | Description |
| ---------------------------------- | ------------------------------------------------------------ |
| `Vector4EqualityComparer Instance` | A comparer instance with the default calculation error value set to 0.0001f. |
## Constructors
| Syntax | Description |
| --------------------------------------------- | ---------------------------------------------- |
| `Vector4EqualityComparer(float allowedError)` | Creates an instance with a custom error value. |
## Public methods
| Syntax | Description |
| ------------------------------------------------ | ------------------------------------------------------------ |
| `bool Equals(Vector4 expected, Vector4 actual);` | Compares the `actual` and `expected` `Vector4` objects for equality using [Utils.AreFloatsEqual](http://todo) to compare the `x`, `y`, `z`, and `w` attributes of `Vector4`. |
## Example
```c#
[TestFixture]
public class Vector4Test
{
[Test]
public void VerifyThat_TwoVector4ObjectsAreEqual()
{
// Custom error 10e-6f
var actual = new Vector4(0, 0, 1e-6f, 1e-6f);
var expected = new Vector4(1e-6f, 0f, 0f, 0f);
var comparer = new Vector4EqualityComparer(10e-6f);
Assert.That(actual, Is.EqualTo(expected).Using(comparer));
// Default error 0.0001f
actual = new Vector4(0.01f, 0.01f, 0f, 0f);
expected = new Vector4(0.01f, 0.01f, 0f, 0f);
Assert.That(actual, Is.EqualTo(expected).Using(Vector4EqualityComparer.Instance));
}
}
```

View file

@ -0,0 +1,66 @@
# Custom assertion
A test fails if Unity logs a message other than a regular log or warning message. Use [LogAssert](#logassert) to check for an expected message in the log so that the test does not fail when Unity logs the message.
Use `LogAssert.Expect` before running the code under test, as the check for expected logs runs at the end of each frame.
A test also reports a failure, if an expected message does not appear, or if Unity does not log any regular log or warning messages.
## Example
```c#
[Test]
public void LogAssertExample()
{
// Expect a regular log message
LogAssert.Expect(LogType.Log, "Log message");
// The test fails without the following expected log message
Debug.Log("Log message");
// An error log
Debug.LogError("Error message");
// Without expecting an error log, the test would fail
LogAssert.Expect(LogType.Error, "Error message");
}
```
## LogAssert
`LogAssert` lets you expect Unity log messages that would otherwise cause the test to fail.
### Static properties
| Syntax | Description |
| ---------------------------- | ------------------------------------------------------------ |
| `bool ignoreFailingMessages` | Set this property to `true` to prevent unexpected error log messages from triggering an assertion. By default, it is `false`. |
### Static Methods
| Syntax | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| `void Expect(LogType type, string message);` `void Expect(LogType type, Regex message);` | Verifies that a log message of a specified type appears in the log. A test wont fail from an expected error, assertion, or exception log message. It does fail if an expected message does not appear in the log. |
| `void NoUnexpectedReceived();` | Triggers an assertion when receiving any log messages and fails the test if some are unexpected messages. If multiple tests need to check for no received unexpected logs, consider using the [TestMustExpectAllLogs](./reference-attribute-testmustexpectalllogs.md) attribute instead. |
### Expect string message
`void Expect(LogType type, string message);`
#### Parameters
| Syntax | Description |
| ---------------- | ------------------------------------------------------------ |
| `LogType type` | A type of log to expect. It can take one of the [LogType enum](https://docs.unity3d.com/ScriptReference/LogType.html) values. |
| `string message` | A string value that should equate to the expected message. |
### Expect Regex message
`void Expect(LogType type, Regex message);`
#### Parameters
| Syntax | Description |
| --------------- | ------------------------------------------------------------ |
| `LogType type` | A type of log to expect. It can take one of the [LogType enum](https://docs.unity3d.com/ScriptReference/LogType.html) values. |
| `Regex message` | A regular expression pattern to match the expected message. |

View file

@ -0,0 +1,15 @@
# Custom attributes
As a part of UTFs public API we provide the following attributes:
* [ConditionalIgnore attribute](./reference-attribute-conditionalignore.md)
* [PostBuildCleanup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [PrebuildSetup attribute](./reference-setup-and-cleanup.md#prebuildsetup-and-postbuildcleanup)
* [TestMustExpectAllLogs attribute](./reference-attribute-testmustexpectalllogs.md)
* [TestPlayerBuildModifier attribute](./reference-attribute-testplayerbuildmodifier.md)
* [TestRunCallback attribute](./reference-attribute-testruncallback.md)
* [UnityPlatform attribute](./reference-attribute-unityplatform.md)
* [UnitySetUp attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTearDown attribute](./reference-actions-outside-tests.md#unitysetup-and-unityteardown)
* [UnityTest attribute](./reference-attribute-unitytest.md)

View file

@ -0,0 +1,31 @@
# Custom constraints
`NUnit` allows you to write test assertions in a more descriptive and human readable way using the [Assert.That](https://github.com/nunit/docs/wiki/Assertions) mechanism, where the first parameter is an object under test and the second parameter describes conditions that the object has to meet.
## Is
Weve extended `NUnit` API with a custom constraint type and declared an overlay `Is` class. To resolve ambiguity between the original implementation and the custom one you must explicitly declare it with a using statement or via addressing through the full type name `UnityEngine.TestTools.Constraints.Is`.
### Static Methods
| Syntax | Description |
| -------------------- | ------------------------------------------------------------ |
| `AllocatingGCMemory` | A constraint type that invokes the delegate you provide as the parameter of `Assert.That` and checks whether it causes any GC memory allocations. It passes if any GC memory is allocated and fails if not. |
## Example
```c#
using Is = UnityEngine.TestTools.Constraints.Is;
class MyTestClass
{
[Test]
public void MyTest()
{
Assert.That(() => {
var i = new int[500];
}, Is.AllocatingGCMemory());
}
}
```

View file

@ -0,0 +1,32 @@
# Custom equality comparers
To enable easier verification of custom Unity type values in your tests we provide you with some custom equality comparers:
* [ColorEqualityComparer](./reference-comparer-color.md)
* [FloatEqualityComparer](./reference-comparer-float.md)
* [QuaternionEqualityComparer](./reference-comparer-quaternion.md)
* [Vector2EqualityComparer](./reference-comparer-vector2.md)
* [Vector3EqualityComparer](./reference-comparer-vector3.md)
* [Vector4EqualityComparer](./reference-comparer-vector4.md)
Use these classes to compare two objects of the same type for equality within the range of a given tolerance using [NUnit ](https://github.com/nunit/docs/wiki/Constraints)or [custom constraints](./reference-custom-constraints.md) . Call Instance to apply the default calculation error value to the comparison. To set a specific error value, instantiate a new comparer object using a one argument constructor `ctor(float error)`.
## Static properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `Instance` | A singleton instance of the comparer with a predefined default error value. |
## Constructors
| Syntax | Description |
| ------------------- | ------------------------------------------------------------ |
| `ctor(float error)` | Creates an instance of comparer with a custom error `value.allowedError`. The relative error to be considered while comparing two values. |
## Public methods
| Syntax | Description |
| ------------------------------------ | ------------------------------------------------------------ |
| `bool Equals(T expected, T actual);` | Compares the actual and expected objects for equality using a custom comparison mechanism. Returns `true` if expected and actual objects are equal, otherwise it returns `false`. |

View file

@ -0,0 +1,60 @@
# Custom yield instructions
By implementing this interface below, you can define custom yield instructions in **Edit Mode** tests.
## IEditModeTestYieldInstruction
In an Edit Mode test, you can use `IEditModeTestYieldInstruction` interface to implement your own instruction. There are also a couple of commonly used implementations available:
- [EnterPlayMode](#enterplaymode)
- [ExitPlayMode](#exitplaymode)
- [RecompileScripts](./reference-recompile-scripts.md)
- [WaitForDomainReload](./reference-wait-for-domain-reload.md)
## Example
```c#
[UnityTest]
public IEnumerator PlayOnAwakeDisabled_DoesntPlayWhenEnteringPlayMode()
{
var videoPlayer = PrefabUtility.InstantiatePrefab(m_VideoPlayerPrefab.GetComponent<VideoPlayer>()) as VideoPlayer;
videoPlayer.playOnAwake = false;
yield return new EnterPlayMode();
var videoPlayerGO = GameObject.Find(m_VideoPlayerPrefab.name);
Assert.IsFalse(videoPlayerGO.GetComponent<VideoPlayer>().isPlaying);
yield return new ExitPlayMode();
Object.DestroyImmediate(GameObject.Find(m_VideoPlayerPrefab.name));
}
```
## Properties
| Syntax | Description |
| ---------------------------- | ------------------------------------------------------------ |
| `bool ExpectDomainReload` | Returns `true` if the instruction expects a domain reload to occur. |
| `bool ExpectedPlaymodeState` | Returns `true` if the instruction expects the Unity Editor to be in **Play Mode**. |
## Methods
| Syntax | Description |
| ----------------------- | ------------------------------------------------------------ |
| `IEnumerator Perform()` | Used to define multi-frame operations performed when instantiating a yield instruction. |
## EnterPlayMode
* Implements `IEditModeTestYieldInstruction`. Creates a yield instruction to enter Play Mode.
* When creating an Editor test that uses the `UnityTest` attribute, use this to trigger the Editor to enter Play Mode.
* Throws an exception if the Editor is already in Play Mode or if there is a [script compilation error](https://support.unity3d.com/hc/en-us/articles/205930539-How-do-I-interpret-a-compiler-error-).
## ExitPlayMode
* Implements `IEditModeTestYieldInstruction`. A new instance of the class is a yield instruction to exit Play Mode.
* Throws an exception if the Editor is not in Play Mode.

View file

@ -0,0 +1,17 @@
# ExecutionSettings
The `ExecutionSettings` is a set of filters and other settings provided when running a set of tests from the [TestRunnerApi](./reference-test-runner-api.md).
## Constructors
| Syntax | Description |
| ----------------------------------------------------- | -------------------------------------------------------- |
| `ExecutionSettings(params Filter[] filtersToExecute)` | Creates an instance with a given set of filters, if any. |
## Fields
| Syntax | Description |
| ---------------------------- | ------------------------------------------------------------ |
| `Filter[] filters` | A collection of [Filters](./reference-filter.md) to execute tests on. |
| `ITestRunSettings overloadTestRunSettings` | An instance of [ITestRunSettings](./reference-itest-run-settings.md) to set up before running tests on a Player. |
| `bool runSynchronously` | If true, the call to `Execute()` will run tests synchronously, guaranteeing that all tests have finished running by the time the call returns. Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. `[UnityTest]` tests, or tests with `[UnitySetUp]` or `[UnityTearDown]` scaffolding) will be filtered out. |
| 'int playerHeartbeatTimeout' | The time, in seconds, the editor should wait for heartbeats after starting a test run on a player. This defaults to 10 minutes. |

View file

@ -0,0 +1,15 @@
# Filter
The filter class provides the [TestRunnerApi](./reference-test-runner-api.md) with a specification of what tests to run when [running tests programmatically](./extension-run-tests.md).
## Fields
| Syntax | Description |
| ----------------------------- | ------------------------------------------------------------ |
| `TestMode testMode` | An enum flag that specifies if **Edit Mode** or **Play Mode** tests should run. Applying both Edit Mode and Play Mode is currently not supported when running tests from the API. |
| `string[] testNames` | The full name of the tests to match the filter. This is usually in the format `FixtureName.TestName`. If the test has test arguments, then include them in parenthesis. E.g. `MyTestClass2.MyTestWithMultipleValues(1)`. |
| `string[] groupNames` | The same as `testNames`, except that it allows for Regex. This is useful for running specific fixtures or namespaces. E.g. `"^MyNamespace\\."` Runs any tests where the top namespace is `MyNamespace`. |
| `string[] categoryNames` | The name of a [Category](https://nunit.org/docs/2.2.7/category.html) to include in the run. Any test or fixtures runs that have a `Category` matching the string. |
| `string[] assemblyNames` | The name of assemblies included in the run. That is the assembly name, without the .dll file extension. E.g., `MyTestAssembly`. |
| `BuildTarget? targetPlatform` | The [BuildTarget](https://docs.unity3d.com/ScriptReference/BuildTarget.html) platform to run the test on. If set to `null`, then the Editor is the target for the tests. |

View file

@ -0,0 +1,48 @@
# ICallbacks
An interface for receiving callbacks when running tests. All test runs invoke the callbacks until the next domain reload.
The `RunStarted` method runs when the whole test run starts. Then the `TestStarted` method runs with information about the tests it is about to run on an assembly level. Afterward, it runs on a test fixture level and then on the individual test. If the test is a [parameterized test](./https://github.com/nunit/docs/wiki/Parameterized-Tests), then it is also invoked for each parameter combination. After each part of the test tree have completed running, the corresponding `TestFinished` method runs with the test result. At the end of the run, the `RunFinished` event runs with the test result.
An extended version of the callback, [IErrorCallbacks](./reference-ierror-callbacks.md), extends this `ICallbacks` to receive calls when a run fails due to a build error.
## Public methods
| Syntax | Description |
| ---------------------------------------------- | ------------------------------------------------------------ |
| `void RunStarted(ITestAdaptor testsToRun)` | Invoked when the test run starts. The [ITestAdaptor](./reference-itest-adaptor.md) represents the tree of tests to run. |
| `void RunFinished(ITestResultAdaptor result)` | Invoked when the test run finishes. The [ITestResultAdaptor](./reference-itest-result-adaptor.md) represents the results of the set of tests that have run. |
| `void TestStarted(ITestAdaptor test)` | Invoked on each node of the test tree, as that part of the tree starts to run. |
| `void TestFinished(ITestResultAdaptor result)` | Invoked on each node of the test tree once that part of the test tree has finished running. The [ITestResultAdaptor](./reference-itest-result-adaptor.md) represents the results of the current node of the test tree. |
## Example
An example that sets up a listener on the API. The listener prints the number of failed tests after the run has finished:
``` C#
public void SetupListeners()
{
var api = ScriptableObject.CreateInstance<TestRunnerApi>();
api.RegisterCallbacks(new MyCallbacks());
}
private class MyCallbacks : ICallbacks
{
public void RunStarted(ITestAdaptor testsToRun)
{
}
public void RunFinished(ITestResultAdaptor result)
{
Debug.Log(string.Format("Run finished {0} test(s) failed.", result.FailCount));
}
public void TestStarted(ITestAdaptor test)
{
}
public void TestFinished(ITestResultAdaptor result)
{
}
}
```

View file

@ -0,0 +1,9 @@
# IErrorCallbacks
An extended version of the [ICallbacks](./reference-icallbacks.md), which get invoked if the test run fails due to a build error or if any [IPrebuildSetup](./reference-setup-and-cleanup.md) has a failure.
## Public methods
| Syntax | Description |
| ---------------------------- | ------------------------------------------------------------------- |
| void OnError(string message) | The error message detailing the reason for the run to fail. |

View file

@ -0,0 +1,31 @@
# ITestAdaptor
`ITestAdaptor` is a representation of a node in the test tree implemented as a wrapper around the [NUnit](http://www.nunit.org/) [ITest](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/ITest.cs) interface.
## Properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `string Id` | The ID of the test tree node. The ID can change if you add new tests to the suite. Use `UniqueName`, if you want to have a more permanent point of reference. |
| `string Name` | The name of the test. E.g., `MyTest`. |
| `string FullName` | The full name of the test. E.g., `MyNamespace.MyTestClass.MyTest`. |
| `int TestCaseCount` | The total number of test cases in the node and all sub-nodes. |
| `bool HasChildren` | Whether the node has any children. |
| `bool IsSuite` | Whether the node is a test suite/fixture. |
| `IEnumerable<ITestAdaptor> Children` | The child nodes. |
| `ITestAdaptor Parent` | The parent node, if any. |
| `int TestCaseTimeout` | The test case timeout in milliseconds. Note that this value is only available on TestFinished. |
| `ITypeInfo TypeInfo` | The type of test class as an `NUnit` [ITypeInfo](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/ITypeInfo.cs). If the node is not a test class, then the value is `null`. |
| `IMethodInfo Method` | The [Nunit IMethodInfo](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/IMethodInfo.cs) of the test method. If the node is not a test method, then the value is `null`. |
| `string[] Categories` | An array of the categories applied to the test or fixture. |
| `bool IsTestAssembly` | Whether the node represents a test assembly. |
| `RunState RunState` | The run state of the test node. Either `NotRunnable`, `Runnable`, `Explicit`, `Skipped`, or `Ignored`. |
| `string Description` | The description of the test. |
| `string SkipReason` | The skip reason. E.g., if ignoring the test. |
| `string ParentId` | The ID of the parent node. |
| `string ParentFullName` | The full name of the parent node. |
| `string UniqueName` | A unique generated name for the test node. E.g., `Tests.dll/MyNamespace/MyTestClass/[Tests][MyNamespace.MyTestClass.MyTest]`. |
| `string ParentUniqueName` | A unique name of the parent node. E.g., `Tests.dll/MyNamespace/[Tests][MyNamespace.MyTestClass][suite]`. |
| `int ChildIndex` | The child index of the node in its parent. |
| `TestMode TestMode` | The mode of the test. Either **Edit Mode** or **Play Mode**. |
> **Note**: Some properties are not available when receiving the test tree as a part of a test result coming from a standalone Player, such as `TypeInfo` and `Method`.

View file

@ -0,0 +1,25 @@
# ITestResultAdaptor
The `ITestResultAdaptor` is the representation of the test results for a node in the test tree implemented as a wrapper around the [NUnit](http://www.nunit.org/) [ITest](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/ITestResults.cs) interface.
## Properties
| Syntax | Description |
| ---------- | ------------------------------------------------------------ |
| `ITestAdaptor Test` | The test details of the test result tree node as a [TestAdaptor](./reference-itest-adaptor.md). |
| `string Name` | The name of the test node. |
| `string FullName` | Gets the full name of the test result |
| `string ResultState` | The state of the result as a string. E.g., `Success`, `Skipped`, `Failure`, `Explicit`, `Cancelled`. |
| `TestStatus TestStatus` | The status of the test as an enum. Either `Inconclusive`, `Skipped`, `Passed`, or `Failed`. |
| `double Duration` | Gets the elapsed time for running the test in seconds. |
| `DateTime StartTime` | Gets or sets the time the test started running. |
| `DateTime EndTime` | Gets or sets the time the test finished running. |
| `string Message` | Gets the message associated with a test failure or with not running the test |
| `string StackTrace` | Gets any stack trace associated with an error or failure. Not available in the [Compact Framework](https://en.wikipedia.org/wiki/.NET_Compact_Framework) 1.0. |
| `int AssertCount` | Gets the number of asserts that ran during the test and all its children. |
| `int FailCount` | Gets the number of test cases that failed when running the test and all its children. |
| `int PassCount` | Gets the number of test cases that passed when running the test and all its children. |
| `int SkipCount` | Gets the number of test cases skipped when running the test and all its children. |
| `int InconclusiveCount` | Gets the number of test cases that were inconclusive when running the test and all its children. |
| `bool HasChildren` | Indicates whether this result has any child results. Accessing HasChildren should not force the creation of the Children collection in classes implementing this interface. |
| `IEnumerable<ITestResultAdaptor> Children` | Gets the collection of child results. |
| `string Output` | Gets any text output written to this result. |
| `TNode ToXml` | Gets the test results as an `NUnit` XML node. Use this to save the results to an XML file. |

View file

@ -0,0 +1,29 @@
# ITestRunSettings
`ITestRunSettings` lets you set any of the global settings right before building a Player for a test run and then reverts the settings afterward.
`ITestRunSettings` implements [IDisposable](https://docs.microsoft.com/en-us/dotnet/api/system.idisposable), and runs after building the Player with tests.
## Public methods
| Syntax | Description |
| ---------------- | ------------------------------------------------------------ |
| `void Apply()` | A method called before building the Player. |
| `void Dispose()` | A method called after building the Player or if the build failed. |
## Example
The following example sets the iOS SDK version to be the simulator SDK and resets it to the original value after the run.
``` C#
public class MyTestSettings : ITestRunSettings
{
private iOSSdkVersion originalSdkVersion;
public void Apply()
{
originalSdkVersion = PlayerSettings.iOS.sdkVersion;
PlayerSettings.iOS.sdkVersion = iOSSdkVersion.SimulatorSDK;
}
public void Dispose()
{
PlayerSettings.iOS.sdkVersion = originalSdkVersion;
}
}
```

View file

@ -0,0 +1,22 @@
# RecompileScripts
`RecompileScripts` is an [IEditModeTestYieldInstruction](./reference-custom-yield-instructions.md) that you can yield in Edit Mode tests. It lets you trigger a recompilation of scripts in the Unity Editor.
## Constructors
| Syntax | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| `RecompileScripts(bool expectScriptCompilation = true, bool expectScriptCompilationSuccess = true)` | Creates a new instance of the `RecompileScripts` yield instruction. The parameter `expectScriptCompilation` indicates if you expect a script compilation to start (defaults to true). If a script compilation does not start and `expectScriptCompilation` is `true`, then it throws an exception. |
## Example
``` C@
[UnitySetUp]
public IEnumerator SetUp()
{
using (var file = File.CreateText("Assets/temp/myScript.cs"))
{
file.Write("public class ATempClass { }");
}
AssetDatabase.Refresh();
yield return new RecompileScripts();
}
```

View file

@ -0,0 +1,91 @@
# Setup and cleanup at build time
In some cases, it is relevant to perform changes to Unity or the file system before building the tests. In the same way, it may be necessary to clean up such changes after the test run. In response to such needs, you can incorporate the pre-build setup and post-build cleanup concepts into your tests in one of the following ways:
1. Via implementation of `IPrebuildSetup` and `IPostBuildCleanup` interfaces by a test class.
2. Via applying the `PrebuildSetup` attribute and `PostBuildCleanup` attribute on your test class, one of the tests or the test assembly, providing a class name that implements the corresponding interface as an argument (fx `[PrebuildSetup("MyTestSceneSetup")]`).
## Execution order
All setups run in a deterministic order one after another. The first to run are the setups defined with attributes. Then any test class implementing the interface runs, in alphabetical order inside their namespace, which is the same order as the tests run.
> **Note**: Cleanup runs right away for a standalone test run, but only after related tests run in the Unity Editor.
## PrebuildSetup and PostBuildCleanup
Both `PrebuildSetup` and `PostBuildCleanup` attributes run if the respective test or test class is in the current test run. The test is included either by running all tests or setting a [filter](./workflow-create-test.md#filters) that includes the test. If multiple tests reference the same pre-built setup or post-build cleanup, then it only runs once.
## IPrebuildSetup
Implement this interface if you want to define a set of actions to run as a pre-build step.
### Public methods
| Syntax | Description |
| -------------- | ------------------------------------------------------------ |
| `void Setup()` | Implement this method to call actions automatically before the build process. |
## IPostBuildCleanup
Implement this interface if you want to define a set of actions to execute as a post-build step. Cleanup runs right away for a standalone test run, but only after all the tests run within the Editor.
### Public methods
| Syntax | Description |
| ---------------- | ------------------------------------------------------------ |
| `void Cleanup()` | Implement this method to specify actions that should run as a post-build cleanup step. |
## Example
```c#
[TestFixture]
public class CreateSpriteTest : IPrebuildSetup
{
Texture2D m_Texture;
Sprite m_Sprite;
public void Setup()
{
#if UNITY_EDITOR
var spritePath = "Assets/Resources/Circle.png";
var ti = UnityEditor.AssetImporter.GetAtPath(spritePath) as UnityEditor.TextureImporter;
ti.textureCompression = UnityEditor.TextureImporterCompression.Uncompressed;
ti.SaveAndReimport();
#endif
}
[SetUp]
public void SetUpTest()
{
m_Texture = Resources.Load<Texture2D>("Circle");
}
[Test]
public void WhenNullTextureIsPassed_CreateShouldReturnNullSprite()
{
// Check with Valid Texture.
LogAssert.Expect(LogType.Log, "Circle Sprite Created");
Sprite.Create(m_Texture, new Rect(0, 0, m_Texture.width, m_Texture.height), new Vector2(0.5f, 0.5f));
Debug.Log("Circle Sprite Created");
// Check with NULL Texture. Should return NULL Sprite.
m_Sprite = Sprite.Create(null, new Rect(0, 0, m_Texture.width, m_Texture.heig`t), new Vector2(0.5f, 0.5f));
Assert.That(m_Sprite, Is.Null, "Sprite created with null texture should be null");
}
}
```
> **Tip**: Use `#if UNITY_EDITOR` if you want to access Editor only APIs, but the setup/cleanup is inside a **Play Mode** assembly.

View file

@ -0,0 +1,23 @@
# TestRunnerApi
The `TestRunnerApi` retrieves and runs tests programmatically from code inside the project, or inside other packages. `TestRunnerApi` is a [ScriptableObject](https://docs.unity3d.com/ScriptReference/ScriptableObject.html).
You can initialize the API like this:
```c#
var testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
```
> **Note**: You can subscribe and receive test results in one instance of the API, even if the run starts from another instance.
The `TestRunnerApi` supports the following workflows:
* [How to run tests programmatically](./extension-run-tests.md)
* [How to get test results](./extension-get-test-results.md)
* [How to retrieve the list of tests](./extension-retrieve-test-list.md)
## Public methods
| Syntax | Description |
| ------------------------------------------ | ------------------------------------------------------------ |
| `void Execute(ExecutionSettings executionSettings)` | Starts a test run with a given set of [ExecutionSettings](./reference-execution-settings.md). |
| `void RegisterCallbacks(ICallbacks testCallbacks, int priority = 0)` | Sets up a given instance of [ICallbacks](./reference-icallbacks.md) to be invoked on test runs. |
| `void UnregisterCallbacks(ICallbacks testCallbacks)` | Unregisters an instance of ICallbacks to no longer receive callbacks from test runs. |
| `void RetrieveTestList(TestMode testMode, Action<ITestAdaptor> callback)` | Retrieve the full test tree as [ITestAdaptor](./reference-itest-adaptor.md) for a given test mode. |

View file

@ -0,0 +1,40 @@
# Test Utils
This contains test utility functions for float value comparison and creating primitives.
## Static Methods
| Syntax | Description |
| ------------------------------------------------------------ | ------------------------------------------------------------ |
| `bool AreFloatsEqual(float expected, float actual, float allowedRelativeError)` | Relative epsilon comparison of two float values for equality. `allowedRelativeError` is the relative error to be used in relative epsilon comparison. The relative error is the absolute error divided by the magnitude of the exact value. Returns `true` if the actual value is equivalent to the expected value. |
| `bool AreFloatsEqualAbsoluteError(float expected, float actual, float allowedAbsoluteError)` | Compares two floating point numbers for equality under the given absolute tolerance. `allowedAbsoluteError` is the permitted error tolerance. Returns `true` if the actual value is equivalent to the expected value under the given tolerance. |
| `GameObject CreatePrimitive( type)` | Creates a [GameObject](https://docs.unity3d.com/ScriptReference/GameObject.html) with a primitive [MeshRenderer](https://docs.unity3d.com/ScriptReference/MeshRenderer.html). This is an analogue to the [GameObject.CreatePrimitive](https://docs.unity3d.com/ScriptReference/GameObject.CreatePrimitive.html), but creates a primitive `MeshRenderer` with a fast [Shader](https://docs.unity3d.com/ScriptReference/Shader.html) instead of the default built-in `Shader`, optimized for testing performance. `type` is the [primitive type](https://docs.unity3d.com/ScriptReference/PrimitiveType.html) of the required `GameObject`. Returns a `GameObject` with primitive `MeshRenderer` and [Collider](https://docs.unity3d.com/ScriptReference/Collider.html). |
## Example
```c#
[TestFixture]
class UtilsTests
{
[Test]
public void CheckThat_FloatsAreEqual()
{
float expected = 10e-8f;
float actual = 0f;
float allowedRelativeError = 10e-6f;
Assert.That(Utils.AreFloatsEqual(expected, actual, allowedRelativeError), Is.True);
}
[Test]
public void CheckThat_FloatsAreAbsoluteEqual()
{
float expected = 0f;
float actual = 10e-6f;
float error = 10e-5f;
Assert.That(Utils.AreFloatsEqualAbsoluteError(expected, actual, error), Is.True);
}
}
```

View file

@ -0,0 +1,51 @@
# MonoBehaviour tests
`MonoBehaviourTest` is a [coroutine](https://docs.unity3d.com/ScriptReference/Coroutine.html) and a helper for writing [MonoBehaviour](https://docs.unity3d.com/ScriptReference/MonoBehaviour.html) tests.
Yield a `MonoBehaviourTest` when using the `UnityTest` attribute to instantiate the `MonoBehaviour` you wish to test and wait for it to finish running. Implement the `IMonoBehaviourTest` interface on the `MonoBehaviour` to state when the test completes.
## Example
```c#
[UnityTest]
public IEnumerator MonoBehaviourTest_Works()
{
yield return new MonoBehaviourTest<MyMonoBehaviourTest>();
}
public class MyMonoBehaviourTest : MonoBehaviour, IMonoBehaviourTest
{
private int frameCount;
public bool IsTestFinished
{
get { return frameCount > 10; }
}
void Update()
{
frameCount++;
}
}
```
## MonoBehaviourTest&lt;T&gt;
This is a wrapper that allows running tests on `MonoBehaviour` scripts. Inherits from [CustomYieldInstruction](https://docs.unity3d.com/ScriptReference/CustomYieldInstruction.html).
### Properties
| Syntax | Description |
| ----------------------- | ------------------------------------------------------------ |
| `T component` | A `MonoBehaviour` component created for the test and attached to the tests [GameObject](https://docs.unity3d.com/ScriptReference/GameObject.html). |
| `GameObject gameObject` | A `GameObject` created as a container for the test component. |
| `bool keepWaiting` | (Inherited) Returns `true` if the test is not finished yet, which keeps the coroutine suspended. |
## IMonoBehaviourTest
An interface implemented by a `MonoBehaviour` test.
### Properties
| Syntax | Description |
| --------------------- | ----------------------------------------------- |
| `bool IsTestFinished` | Indicates when the test is considered finished. |

View file

@ -0,0 +1,18 @@
# Parameterized tests
For data-driven testing, you may want to have your tests parameterized. You may use both the [NUnit](http://www.nunit.org/) attributes [TestCase](https://github.com/nunit/docs/wiki/TestCase-Attribute) and [ValueSource](https://github.com/nunit/docs/wiki/ValueSource-Attribute) with a unit test.
> **Note**: With `UnityTest` it is recommended to use `ValueSource` since `TestCase` is not supported.
## Example
```c#
static int[] values = new int[] { 1, 5, 6 };
[UnityTest]
public IEnumerator MyTestWithMultipleValues([ValueSource("values")] int value)
{
yield return null;
}
```

View file

@ -0,0 +1,19 @@
# WaitForDomainReload
`WaitForDomainReload` is an [IEditModeTestYieldInstruction](./reference-custom-yield-instructions.md) that you can yield in Edit Mode tests. It delays the execution of scripts until after an incoming domain reload. If the domain reload results in a script compilation failure, then it throws an exception.
## Constructors
| Syntax | Description |
| ---------------------------- | ------------------------------------------------------------ |
| `WaitForDomainReload()` | Create a new instance of the `WaitForDomainReload` yield instruction. |
## Example
``` C@
[UnitySetUp]
public IEnumerator SetUp()
{
File.Copy("Resources/MyDll.dll", @"Assets/MyDll.dll", true); // Trigger a domain reload.
AssetDatabase.Refresh();
yield return new WaitForDomainReload();
}
```

View file

@ -0,0 +1,6 @@
# Resources
Here you can find other related resources to the Unity Test Framework:
* [Performance Benchmarking in Unity: How to Get Started](https://blogs.unity3d.com/2018/09/25/performance-benchmarking-in-unity-how-to-get-started/) [Blog]
* [Testing Test-Driven Development with the Unity Test Runner](https://blogs.unity3d.com/2018/11/02/testing-test-driven-development-with-the-unity-test-runner/) [Blog]

View file

@ -0,0 +1,28 @@
# Workflow: How to create a Play Mode test
To create a **Play Mode** test, you can follow a similar process as when you want to create an **Edit Mode** test.
1. Start with switching to the **PlayMode** tab in the **Test Runner** window.
2. Create a test assembly folder (see [How to create a new test assembly)](./workflow-create-test-assembly.md). The folder name is *Tests* by default (or *Tests 1*, *Tests 2*, etc. if the preceding name is already in use).
![PlayMode tab](./images/playmode-tab.png)
> **Note**: If you dont see the **Create Play Mode Test Assembly Folder** button enabled, make sure that in the Project window you navigate out of a folder with another .asmdef (such as one for Edit Mode tests).
3. When you have your Play Mode test assembly folder ready, then [create your Play Mode test](./workflow-create-test.md).
> **Note**: [Pre-defined Unity assemblies](https://docs.unity3d.com/Manual/ScriptCompileOrderFolders.html) (such as _Assembly-CSharp.dll_) do not reference your new assembly.
## References and builds
Unity Test Framework adds a reference to `TestAssemblies` in the [Assembly Definition](https://docs.unity3d.com/Manual/ScriptCompilationAssemblyDefinitionFiles.html) file, but it won't include any other references (e.g., to other scripting assemblies within the Unity project). So you need to add other assemblies yourself if you want to test them too.
Unity does not include `TestAssemblies` in Player builds, but in the Test Runner window, we have such an option. If you need to test code in pre-defined assemblies, you can reference `TestAssemblies` from other assemblies. You must remove these tests after the test run so that Unity does not add them to the final Player build.
To do this, in the Test Runner window choose **Enable playmode tests for all assemblies** option from the drop-down menu (to expand, click on the small list item in the top right corner). In the dialog box, click **OK** to manually restart the Editor.
![Enable Play Mode tests for all assemblies](./images/playmode-enable-all.png)
> **Note**: **Enabling Play Mode tests for all assemblies** includes additional assemblies in your project build, which can increase the projects size as well as the build time.
For more information, see [Edit Mode vs. Play Mode tests](./edit-mode-vs-play-mode-tests.md).

View file

@ -0,0 +1,17 @@
# Workflow: **How to create a new test assembly**
Unity Test Framework looks for a test inside any assembly that references [NUnit](http://www.nunit.org/). We refer to such assemblies as `TestAssemblies`. The [Test Runner](./getting-started.md) UI can help you set up `TestAssemblies`. **Play Mode** and **Edit Mode** tests need to be in separate assemblies.
In the **Test Runner** window, you will see an **EditMode** tab enabled by default, as well as a **Create EditMode Test Assembly Folder** button.
![Test Runner window EditMode tab](./images/editmode-tab.png)
Click the button to create a *Tests* folder with a respective .asmdef file by default. Change the name of the new [Assembly Definition](https://docs.unity3d.com/Manual/ScriptCompilationAssemblyDefinitionFiles.html), if necessary, and press Enter to accept it.
![New Test folder and assembly file](./images/tests-folder-assembly.png)
In the Inspector window, it should have references to **nunit.framework.dll***,* **UnityEngine.TestRunner,** and **UnityEditor.TestRunner** assemblies, as well as **Editor** preselected as a target platform.
> **Note**: The **UnityEditor.TestRunner** reference is only available for [Edit Mode tests](./edit-mode-vs-play-mode-tests.md#edit-mode-tests).
![Assembly definition import settings](./images/import-settings.png)

View file

@ -0,0 +1,36 @@
# Workflow: How to create a test
To create a test, do the following:
1. Create your *Test* [assembly folder](./workflow-create-test-assembly.md) and select it in the **Project** window.
2. Click the button **Create Test Script in current folder** option in the **Test Runner** window.
![EditMode create test script](./images/editmode-create-test-script.png)
3. It creates a *NewTestScript.cs* file in the *Tests* folder. Change the name of the script, if necessary, and press Enter to accept it.
![NewTestScript.cs](./images/new-test-script.png)
Now youll see two sample tests in the Test Runner window:
![Test templates](./images/test-templates.png)
Now you can open the tests in your favorite script editor.
You can also create test scripts by navigating to **Assets** > **Create > Testing** > **C# Test Script**, unless adding a test script would result in a compilation error.
> **Note**: Unity does not include `TestAssemblies` ([NUnit](http://www.nunit.org/), Unity Test Framework, and user script assemblies) when using the normal build pipeline, but does include them when using **Run on &lt;Platform&gt;** in the Test Runner window.
## Filters
If you have a lot of tests, and you only want to view/run a sub-set of them, you can filter them in three ways (see image above):
* Type in the search box in the top left
* Click a test class or fixture (such as **NewTestScript** in the image above)
* Click one of the test result icon buttons in the top right
For more information, see [Edit Mode vs. Play Mode tests](./edit-mode-vs-play-mode-tests.md).

View file

@ -0,0 +1,21 @@
# Workflow: How to run a Play Mode test in player
If you run a **Play Mode** test in the same way as an [Editor test](./workflow-run-test.md), it runs inside the Unity Editor. You can also run Play Mode tests on specific platforms. Click **Run all in the player** to build and run your tests on the currently active target platform.
![Run PlayMode test in player](./images/playmode-run-standalone.png)
> **Note**: Your current platform displays in brackets on the button. For example, in the image above, the button reads **Run all in player (StandaloneWindows)**, because the current platform is Windows. The target platform is always the current Platform selected in [Build Settings](https://docs.unity3d.com/Manual/BuildSettings.html) (menu: **File** > **Build Settings**).
The test result displays in the build once the test completes:
![Results of PlayMode in player test run](./images/playmode-results-standalone.png)
The application running on the platform reports back the test results to the Editor UI then displays the executed tests and shuts down. To make sure you receive the test results from the Player on your target platform back into the Editor thats running the test, both should be on the same network.
> **Note:** Some platforms do not support shutting down the application with `Application.Quit`, so it will continue running after reporting the test results.
If Unity cannot instantiate the connection, you can see the tests succeed in the running application. Running tests on platforms with arguments, in this state, does not provide XML test results.
For more information, see [Edit Mode vs Play Mode tests](./edit-mode-vs-play-mode-tests.md).

View file

@ -0,0 +1,19 @@
# Workflow: How to run a test
To run a test, you need to double-click on the test or test fixture name in the **Test Runner** window.
You can also use one of the buttons on the top bar, **Run All** or **Run Selected**. As a result, youll see the test status icon changed and a counter in the top right corner updated:
![EditMode Run Test](./images/editmode-run-test.png)
You may also use a context menu option **Run**, right-click on any item in the test tree to have it (with all its children if any) run.
![EditMode Run Tests](./images/editmode-run-tests.png)
## Run tests within Rider
It is possible to run unit tests in the Unity Test Framework directly from [JetBrains Rider](https://www.jetbrains.com/rider/).
For more information, see the [JetBrains official documentation](https://www.jetbrains.com/help/rider/Running_and_Debugging_Unity_Tests.html) and their blog post [Run Unity tests in Rider 2018.1](https://blog.jetbrains.com/dotnet/2018/04/18/run-unity-tests-rider-2018-1/).

View file

@ -0,0 +1,5 @@
Test Framework copyright © 2020 Unity Technologies ApS
Licensed under the Unity Companion License for Unity-dependent projects--see [Unity Companion License](http://www.unity3d.com/legal/licenses/Unity_Companion_License).
Unless expressly provided otherwise, the Software under this license is made available strictly on an “AS IS” BASIS WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. Please review the license for details on these and other terms and conditions.

View file

@ -0,0 +1,7 @@
fileFormatVersion: 2
guid: 3ec7596410385054a9e0bc90377fbe63
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: 95cdf27b47eb82747ba9e51f41e72a35
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,8 @@
fileFormatVersion: 2
guid: fa423365b1ce06a4dbdc6fb4a8597bfa
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,136 @@
using System;
using System.Linq;
using System.Text;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal;
using UnityEngine;
using UnityEngine.TestRunner.TestLaunchers;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal class CallbacksDelegator : ICallbacksDelegator
{
private static CallbacksDelegator s_instance;
public static CallbacksDelegator instance
{
get
{
if (s_instance == null)
{
s_instance = new CallbacksDelegator(CallbacksHolder.instance.GetAll, new TestAdaptorFactory());
}
return s_instance;
}
}
private readonly Func<ICallbacks[]> m_CallbacksProvider;
private readonly ITestAdaptorFactory m_AdaptorFactory;
public CallbacksDelegator(Func<ICallbacks[]> callbacksProvider, ITestAdaptorFactory adaptorFactory)
{
m_CallbacksProvider = callbacksProvider;
m_AdaptorFactory = adaptorFactory;
}
public void RunStarted(ITest testsToRun)
{
m_AdaptorFactory.ClearResultsCache();
var testRunnerTestsToRun = m_AdaptorFactory.Create(testsToRun);
TryInvokeAllCallbacks(callbacks => callbacks.RunStarted(testRunnerTestsToRun));
}
public void RunStartedRemotely(byte[] testsToRunData)
{
var testData = Deserialize<RemoteTestResultDataWithTestData>(testsToRunData);
var testsToRun = m_AdaptorFactory.BuildTree(testData);
TryInvokeAllCallbacks(callbacks => callbacks.RunStarted(testsToRun));
}
public void RunFinished(ITestResult testResults)
{
var testResult = m_AdaptorFactory.Create(testResults);
TryInvokeAllCallbacks(callbacks => callbacks.RunFinished(testResult));
}
public void RunFinishedRemotely(byte[] testResultsData)
{
var remoteTestResult = Deserialize<RemoteTestResultDataWithTestData>(testResultsData);
var testResult = m_AdaptorFactory.Create(remoteTestResult.results.First(), remoteTestResult);
TryInvokeAllCallbacks(callbacks => callbacks.RunFinished(testResult));
}
public void RunFailed(string failureMessage)
{
Debug.LogError(failureMessage);
TryInvokeAllCallbacks(callbacks =>
{
var errorCallback = callbacks as IErrorCallbacks;
if (errorCallback != null)
{
errorCallback.OnError(failureMessage);
}
});
}
public void TestStarted(ITest test)
{
var testRunnerTest = m_AdaptorFactory.Create(test);
TryInvokeAllCallbacks(callbacks => callbacks.TestStarted(testRunnerTest));
}
public void TestStartedRemotely(byte[] testStartedData)
{
var testData = Deserialize<RemoteTestResultDataWithTestData>(testStartedData);
var testsToRun = m_AdaptorFactory.BuildTree(testData);
TryInvokeAllCallbacks(callbacks => callbacks.TestStarted(testsToRun));
}
public void TestFinished(ITestResult result)
{
var testResult = m_AdaptorFactory.Create(result);
TryInvokeAllCallbacks(callbacks => callbacks.TestFinished(testResult));
}
public void TestFinishedRemotely(byte[] testResultsData)
{
var remoteTestResult = Deserialize<RemoteTestResultDataWithTestData>(testResultsData);
var testResult = m_AdaptorFactory.Create(remoteTestResult.results.First(), remoteTestResult);
TryInvokeAllCallbacks(callbacks => callbacks.TestFinished(testResult));
}
public void TestTreeRebuild(ITest test)
{
m_AdaptorFactory.ClearTestsCache();
var testAdaptor = m_AdaptorFactory.Create(test);
TryInvokeAllCallbacks(callbacks =>
{
var rebuildCallbacks = callbacks as ITestTreeRebuildCallbacks;
if (rebuildCallbacks != null)
{
rebuildCallbacks.TestTreeRebuild(testAdaptor);
}
});
}
private void TryInvokeAllCallbacks(Action<ICallbacks> callbackAction)
{
foreach (var testRunnerApiCallback in m_CallbacksProvider())
{
try
{
callbackAction(testRunnerApiCallback);
}
catch (Exception ex)
{
Debug.LogException(ex);
}
}
}
private static T Deserialize<T>(byte[] data)
{
return JsonUtility.FromJson<T>(Encoding.UTF8.GetString(data));
}
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 0de03ebd74e2b474fa23d05ab42d0cd8
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,28 @@
using UnityEngine;
using UnityEngine.TestTools.TestRunner;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal class CallbacksDelegatorListener : ScriptableObject, ITestRunnerListener
{
public void RunStarted(NUnit.Framework.Interfaces.ITest testsToRun)
{
CallbacksDelegator.instance.RunStarted(testsToRun);
}
public void RunFinished(NUnit.Framework.Interfaces.ITestResult testResults)
{
CallbacksDelegator.instance.RunFinished(testResults);
}
public void TestStarted(NUnit.Framework.Interfaces.ITest test)
{
CallbacksDelegator.instance.TestStarted(test);
}
public void TestFinished(NUnit.Framework.Interfaces.ITestResult result)
{
CallbacksDelegator.instance.TestFinished(result);
}
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: f3e1b3cbf3fac6a459b1a602167ad311
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,69 @@
using System;
using System.Collections.Generic;
using System.Linq;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal class CallbacksHolder : ScriptableSingleton<CallbacksHolder>, ICallbacksHolder
{
private List<CallbackWithPriority> m_Callbacks = new List<CallbackWithPriority>();
public void Add(ICallbacks callback, int priority)
{
m_Callbacks.Add(new CallbackWithPriority(callback, priority));
}
public void Remove(ICallbacks callback)
{
m_Callbacks.RemoveAll(callbackWithPriority => callbackWithPriority.Callback == callback);
}
public ICallbacks[] GetAll()
{
return m_Callbacks.OrderByDescending(callback => callback.Priority).Select(callback => callback.Callback).ToArray();
}
public void Clear()
{
m_Callbacks.Clear();
}
private struct CallbackWithPriority
{
public ICallbacks Callback;
public int Priority;
public CallbackWithPriority(ICallbacks callback, int priority)
{
Callback = callback;
Priority = priority;
}
}
// Sometimes - such as when we want to test the test framework itself - it's necessary to launch a test run from
// inside a test. Because callbacks are registered globally, this can cause a lot of confusion (e.g. the in-test
// run will emit UTP messages, utterly confusing UTR). In such circumstances the safest thing to do is to
// temporarily suppress all registered callbacks for the duration of the in-test run. This method can be called
// to set up a using() block which will suppress the callbacks for the scope.
public IDisposable TemporarilySuppressCallbacks()
{
return new Suppressor(this);
}
private sealed class Suppressor : IDisposable
{
private readonly CallbacksHolder _instance;
private readonly List<CallbackWithPriority> _suppressed;
public Suppressor(CallbacksHolder instance)
{
_instance = instance;
_suppressed = new List<CallbackWithPriority>(instance.m_Callbacks);
instance.m_Callbacks.Clear();
}
public void Dispose()
{
_instance.m_Callbacks.AddRange(_suppressed);
}
}
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 4884ccc3528cb2e40a0e6f0a19a2b35b
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,77 @@
using System;
using System.Linq;
using NUnit.Framework.Interfaces;
using NUnit.Framework.Internal.Filters;
using UnityEngine;
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// A set of execution settings defining how to run tests, using the <see cref="TestRunnerApi"/>.
/// </summary>
[Serializable]
public class ExecutionSettings
{
/// <summary>
/// Creates an instance with a given set of filters, if any.
/// </summary>
/// <param name="filtersToExecute">Set of filters</param>
public ExecutionSettings(params Filter[] filtersToExecute)
{
filters = filtersToExecute;
}
[SerializeField]
internal BuildTarget? targetPlatform;
/// <summary>
/// An instance of <see cref="ITestRunSettings"/> to set up before running tests on a Player.
/// </summary>
// Note: Is not available after serialization
public ITestRunSettings overloadTestRunSettings;
[SerializeField]
internal Filter filter;
///<summary>
///A collection of <see cref="Filter"/> to execute tests on.
///</summary>
[SerializeField]
public Filter[] filters;
/// <summary>
/// Note that this is only supported for EditMode tests, and that tests which take multiple frames (i.e. [UnityTest] tests, or tests with [UnitySetUp] or [UnityTearDown] scaffolding) will be filtered out.
/// </summary>
/// <returns>If true, the call to Execute() will run tests synchronously, guaranteeing that all tests have finished running by the time the call returns.</returns>
[SerializeField]
public bool runSynchronously;
/// <summary>
/// The time, in seconds, the editor should wait for heartbeats after starting a test run on a player. This defaults to 10 minutes.
/// </summary>
[SerializeField]
public int playerHeartbeatTimeout = 60*10;
internal bool EditModeIncluded()
{
return filters.Any(f => IncludesTestMode(f.testMode, TestMode.EditMode));
}
internal bool PlayModeInEditorIncluded()
{
return filters.Any(f => IncludesTestMode(f.testMode, TestMode.PlayMode) && targetPlatform == null);
}
internal bool PlayerIncluded()
{
return filters.Any(f => IncludesTestMode(f.testMode, TestMode.PlayMode) && targetPlatform != null);
}
private static bool IncludesTestMode(TestMode testMode, TestMode modeToCheckFor)
{
return (testMode & modeToCheckFor) == modeToCheckFor;
}
internal ITestFilter BuildNUnitFilter()
{
return new OrFilter(filters.Select(f => f.ToRuntimeTestRunnerFilter(runSynchronously).BuildNUnitFilter()).ToArray());
}
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: eea34a28297f9bc4c9f4c573bc8d5d1c
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,56 @@
using System;
using UnityEngine;
using UnityEngine.TestTools.TestRunner.GUI;
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// The filter class provides the <see cref="TestRunnerApi"/> with a specification of what tests to run when [running tests programmatically](https://docs.unity3d.com/Packages/com.unity.test-framework@1.1/manual/extension-run-tests.html).
/// </summary>
[Serializable]
public class Filter
{
/// <summary>
/// An enum flag that specifies if Edit Mode or Play Mode tests should run.
///</summary>
[SerializeField]
public TestMode testMode;
/// <summary>
/// The full name of the tests to match the filter. This is usually in the format FixtureName.TestName. If the test has test arguments, then include them in parenthesis. E.g. MyTestClass2.MyTestWithMultipleValues(1).
/// </summary>
[SerializeField]
public string[] testNames;
/// <summary>
/// The same as testNames, except that it allows for Regex. This is useful for running specific fixtures or namespaces. E.g. "^MyNamespace\\." Runs any tests where the top namespace is MyNamespace.
/// </summary>
[SerializeField]
public string[] groupNames;
/// <summary>
/// The name of a [Category](https://nunit.org/docs/2.2.7/category.html) to include in the run. Any test or fixtures runs that have a Category matching the string.
/// </summary>
[SerializeField]
public string[] categoryNames;
/// <summary>
/// The name of assemblies included in the run. That is the assembly name, without the .dll file extension. E.g., MyTestAssembly
/// </summary>
[SerializeField]
public string[] assemblyNames;
/// <summary>
/// The <see cref="BuildTarget"/> platform to run the test on. If set to null, then the Editor is the target for the tests.
/// </summary>
[SerializeField]
public BuildTarget? targetPlatform;
internal RuntimeTestRunnerFilter ToRuntimeTestRunnerFilter(bool synchronousOnly)
{
return new RuntimeTestRunnerFilter()
{
testNames = testNames,
categoryNames = categoryNames,
groupNames = groupNames,
assemblyNames = assemblyNames,
synchronousOnly = synchronousOnly
};
}
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 05f92e4a2414cb144a92157752dfa324
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,29 @@
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// Callbacks in the <see cref="TestRunnerApi"/> for the test stages when running tests.
/// </summary>
public interface ICallbacks
{
/// <summary>
/// A callback invoked when a test run is started.
/// </summary>
/// <param name="testsToRun">The full loaded test tree.</param>
void RunStarted(ITestAdaptor testsToRun);
/// <summary>
/// A callback invoked when a test run is finished.
/// </summary>
/// <param name="result">The result of the test run.</param>
void RunFinished(ITestResultAdaptor result);
/// <summary>
/// A callback invoked when each individual node of the test tree has started executing.
/// </summary>
/// <param name="test">The test node currently executed.</param>
void TestStarted(ITestAdaptor test);
/// <summary>
/// A callback invoked when each individual node of the test tree has finished executing.
/// </summary>
/// <param name="result">The result of the test tree node after it had been executed.</param>
void TestFinished(ITestResultAdaptor result);
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 93eea84e53d0226479c9a584f19427b5
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,18 @@
using NUnit.Framework.Interfaces;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal interface ICallbacksDelegator
{
void RunStarted(ITest testsToRun);
void RunStartedRemotely(byte[] testsToRunData);
void RunFinished(ITestResult testResults);
void RunFinishedRemotely(byte[] testResultsData);
void RunFailed(string failureMessage);
void TestStarted(ITest test);
void TestStartedRemotely(byte[] testStartedData);
void TestFinished(ITestResult result);
void TestFinishedRemotely(byte[] testResultsData);
void TestTreeRebuild(ITest test);
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 8f8f74fe8e363da42875d9cab025d3b2
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,10 @@
namespace UnityEditor.TestTools.TestRunner.Api
{
internal interface ICallbacksHolder
{
void Add(ICallbacks callback, int priority);
void Remove(ICallbacks callback);
ICallbacks[] GetAll();
void Clear();
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: d742f2caefd9f934d9f19dad07a08e6f
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,16 @@
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// An extended version of the <see cref="ICallbacks"/>, which get invoked if the test run fails due to a build error or if any <see cref="UnityEngine.TestTools.IPrebuildSetup"/> has a failure.
/// </summary>
public interface IErrorCallbacks : ICallbacks
{
/// <summary>
/// Method invoked on failure.
/// </summary>
/// <param name="message">
/// The error message detailing the reason for the run to fail.
/// </param>
void OnError(string message);
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 1a06c562b0c5eb046bcb876a29f93c98
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,100 @@
using System.Collections.Generic;
using NUnit.Framework.Interfaces;
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// ```ITestAdaptor``` is a representation of a node in the test tree implemented as a wrapper around the [NUnit](http://www.nunit.org/) [ITest](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/ITest.cs) interface.
/// </summary>
public interface ITestAdaptor
{
/// <summary>
/// The ID of the test tree node. The ID can change if you add new tests to the suite. Use UniqueName, if you want to have a more permanent point of reference.
/// </summary>
string Id { get; }
/// <summary>
/// The name of the test. E.g.,```MyTest```.
/// </summary>
string Name { get; }
/// <summary>
/// The full name of the test. E.g., ```MyNamespace.MyTestClass.MyTest```.
/// </summary>
string FullName { get; }
/// <summary>
/// The total number of test cases in the node and all sub-nodes.
/// </summary>
int TestCaseCount { get; }
/// <summary>
/// Whether the node has any children.
/// </summary>
bool HasChildren { get; }
/// <summary>
/// True if the node is a test suite/fixture, false otherwise.
/// </summary>
bool IsSuite { get; }
/// <summary>
/// The child nodes.
/// </summary>
IEnumerable<ITestAdaptor> Children { get; }
/// <summary>
/// The parent node, if any.
/// </summary>
ITestAdaptor Parent { get; }
/// <summary>
/// The test case timeout in milliseconds. Note that this value is only available on TestFinished.
/// </summary>
int TestCaseTimeout { get; }
/// <summary>
/// The type of test class as an ```NUnit``` <see cref="ITypeInfo"/>. If the node is not a test class, then the value is null.
/// </summary>
ITypeInfo TypeInfo { get; }
/// <summary>
/// The Nunit <see cref="IMethodInfo"/> of the test method. If the node is not a test method, then the value is null.
/// </summary>
IMethodInfo Method { get; }
/// <summary>
/// An array of the categories applied to the test or fixture.
/// </summary>
string[] Categories { get; }
/// <summary>
/// Returns true if the node represents a test assembly, false otherwise.
/// </summary>
bool IsTestAssembly { get; }
/// <summary>
/// The run state of the test node. Either ```NotRunnable```, ```Runnable```, ```Explicit```, ```Skipped```, or ```Ignored```.
/// </summary>
RunState RunState { get; }
/// <summary>
/// The description of the test.
/// </summary>
string Description { get; }
/// <summary>
/// The skip reason. E.g., if ignoring the test.
/// </summary>
string SkipReason { get; }
/// <summary>
/// The ID of the parent node.
/// </summary>
string ParentId { get; }
/// <summary>
/// The full name of the parent node.
/// </summary>
string ParentFullName { get; }
/// <summary>
/// A unique generated name for the test node. E.g., ```Tests.dll/MyNamespace/MyTestClass/[Tests][MyNamespace.MyTestClass.MyTest]```.
/// </summary>
string UniqueName { get; }
/// <summary>
/// A unique name of the parent node. E.g., ```Tests.dll/MyNamespace/[Tests][MyNamespace.MyTestClass][suite]```.
/// </summary>
string ParentUniqueName { get; }
/// <summary>
/// The child index of the node in its parent.
/// </summary>
int ChildIndex { get; }
/// <summary>
/// The mode of the test. Either **Edit Mode** or **Play Mode**.
/// </summary>
TestMode TestMode { get; }
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 85dd7af03f02aea4aae13a3945e3b313
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,18 @@
using System.Collections.Generic;
using NUnit.Framework.Interfaces;
using UnityEngine.TestRunner.TestLaunchers;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal interface ITestAdaptorFactory
{
ITestAdaptor Create(ITest test);
ITestAdaptor Create(RemoteTestData testData);
ITestResultAdaptor Create(ITestResult testResult);
ITestResultAdaptor Create(RemoteTestResultData testResult, RemoteTestResultDataWithTestData allData);
ITestAdaptor BuildTree(RemoteTestResultDataWithTestData data);
IEnumerator<ITestAdaptor> BuildTreeAsync(RemoteTestResultDataWithTestData data);
void ClearResultsCache();
void ClearTestsCache();
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 803abab0f7e17044db56f8760186dbd1
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,121 @@
using System;
using System.Collections.Generic;
using NUnit.Framework.Interfaces;
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// The `ITestResultAdaptor` is the representation of the test results for a node in the test tree implemented as a wrapper around the [NUnit](http://www.nunit.org/) [ITest](https://github.com/nunit/nunit/blob/master/src/NUnitFramework/framework/Interfaces/ITestResults.cs) interface.
/// </summary>
public interface ITestResultAdaptor
{
/// <summary>
/// The test details of the test result tree node as a <see cref="TestAdaptor"/>
/// </summary>
ITestAdaptor Test { get; }
///<summary>
///The name of the test node.
///</summary>
string Name { get; }
/// <summary>
/// Gets the full name of the test result
/// </summary>
///<returns>
///The name of the test result.
///</returns>
string FullName { get; }
///<summary>
///Gets the state of the result as a string.
///</summary>
///<returns>
///It returns one of these values: `Success`, `Skipped`, `Failure`, `Explicit`, `Cancelled`
///</returns>
string ResultState { get; }
///<summary>
///Gets the status of the test as an enum.
///</summary>
///<returns>
///It returns one of these values:`Inconclusive`, `Skipped`, `Passed`, or `Failed`
///</returns>
TestStatus TestStatus { get; }
/// <summary>
/// Gets the elapsed time for running the test in seconds
/// </summary>
/// <returns>
/// Time in seconds.
/// </returns>
double Duration { get; }
/// <summary>
/// Gets or sets the time the test started running.
/// </summary>
///<returns>
///A DataTime object.
///</returns>
DateTime StartTime { get; }
///<summary>
///Gets or sets the time the test finished running.
///</summary>
///<returns>
///A DataTime object.
///</returns>
DateTime EndTime { get; }
/// <summary>
/// The message associated with a test failure or with not running the test
/// </summary>
string Message { get; }
/// <summary>
/// Any stacktrace associated with an error or failure. Not available in the Compact Framework 1.0.
/// </summary>
string StackTrace { get; }
/// <summary>
/// The number of asserts executed when running the test and all its children.
/// </summary>
int AssertCount { get; }
/// <summary>
/// The number of test cases that failed when running the test and all its children.
/// </summary>
int FailCount { get; }
/// <summary>
/// The number of test cases that passed when running the test and all its children.
/// </summary>
int PassCount { get; }
/// <summary>
/// The number of test cases that were skipped when running the test and all its children.
/// </summary>
int SkipCount { get; }
/// <summary>
///The number of test cases that were inconclusive when running the test and all its children.
/// </summary>
int InconclusiveCount { get; }
/// <summary>
/// Accessing HasChildren should not force creation of the Children collection in classes implementing this interface.
/// </summary>
/// <returns>True if this result has any child results.</returns>
bool HasChildren { get; }
/// <summary>
/// Gets the the collection of child results.
/// </summary>
IEnumerable<ITestResultAdaptor> Children { get; }
/// <summary>
/// Gets any text output written to this result.
/// </summary>
string Output { get; }
/// <summary>
/// Use this to save the results to an XML file
/// </summary>
/// <returns>
/// The test results as an `NUnit` XML node.
/// </returns>
TNode ToXml();
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 4f90cfe4bf5cfb44f84a5b11387f2a42
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,16 @@
using System;
namespace UnityEditor.TestTools.TestRunner.Api
{
/// <summary>
/// ITestRunSettings lets you set any of the global settings right before building a Player for a test run and then reverts the settings afterward. ITestRunSettings implements
/// [IDisposable](https://docs.microsoft.com/en-us/dotnet/api/system.idisposable?view=netframework-4.8), and runs after building the Player with tests.
/// </summary>
public interface ITestRunSettings : IDisposable
{
/// <summary>
/// A method called before building the Player.
/// </summary>
void Apply();
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 2ae2ce6274819484fa8747a28cebdf3a
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,12 @@
using System;
namespace UnityEditor.TestTools.TestRunner.Api
{
internal interface ITestRunnerApi
{
string Execute(ExecutionSettings executionSettings);
void RegisterCallbacks<T>(T testCallbacks, int priority = 0) where T : ICallbacks;
void UnregisterCallbacks<T>(T testCallbacks) where T : ICallbacks;
void RetrieveTestList(TestMode testMode, Action<ITestAdaptor> callback);
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: a7842a837a4b13e41ae16193db753418
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

View file

@ -0,0 +1,7 @@
namespace UnityEditor.TestTools.TestRunner.Api
{
internal interface ITestTreeRebuildCallbacks : ICallbacks
{
void TestTreeRebuild(ITestAdaptor test);
}
}

View file

@ -0,0 +1,11 @@
fileFormatVersion: 2
guid: 4230e406313f1db43a4b548e7a3ad2e2
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

Some files were not shown because too many files have changed in this diff Show more