mirror of
https://gitea.invidious.io/iv-org/shard-spectator.git
synced 2024-08-15 00:53:35 +00:00
274 lines
8.8 KiB
Markdown
274 lines
8.8 KiB
Markdown
# Architecture and Design of Spectator
|
|
|
|
This document explains the structure and design decisions behind Spectator.
|
|
It is broken up into the logical components of Spectator:
|
|
|
|
- [Terms](#terms)
|
|
- [DSL](#dsl) - Domain Specific Language. Macros and methods that build a spec.
|
|
- [Matchers](#matchers)
|
|
- [Examples and groups](#examples-and-groups)
|
|
- [Runner and harness](#runner-and-harness)
|
|
- [Hooks](#hooks)
|
|
- [Mocks and doubles](#mocks-and-doubles)
|
|
- [Stubs](#stubs)
|
|
- [Doubles](#doubles)
|
|
- [Formatting](#formatting)
|
|
|
|
## Terms
|
|
|
|
The following are terms and concepts frequently used in the project.
|
|
They are listed in alphabetical order,
|
|
but you may find it useful to jump around when learning what they mean.
|
|
|
|
**Assertion**
|
|
|
|
An *assertion* is a fundamental piece of a test.
|
|
It checks that a condition is satisfied.
|
|
If that condition isn't met, then an exception is raised.
|
|
|
|
**Builder**
|
|
|
|
A *builder* is a type that incrementally constructs a complex object.
|
|
Builders are primarily used to create *specs* and *example groups*.
|
|
See: https://sourcemaking.com/design_patterns/builder
|
|
|
|
**Config**
|
|
|
|
Short for *configuration*, a *config* stores information about how to run the *spec*.
|
|
The configuration includes parsed command-line options, settings from `.spectator` and `Spectator.configure`.
|
|
|
|
**Context**
|
|
|
|
A *context* is the scope or "environment" a test runs in.
|
|
It is a part of an *example group* that provides methods, memoized values, and more to an example block.
|
|
From a technical standpoint, it is typically an instance of the class defined by an `example_group` block.
|
|
It can thought of as a closure.
|
|
|
|
**Double**
|
|
|
|
Stand-in for another type.
|
|
*Doubles* can be passed to methods under test instead of a real object.
|
|
They can be configured to respond to methods with *stubs*.
|
|
*Doubles* also track calls made to them.
|
|
An important note: a *double* is _not_ the same type (nor does it inherit) the replaced type.
|
|
|
|
**DSL**
|
|
|
|
*DSL* stands for **D**omain **S**pecific **L**anguage.
|
|
It is the human-like language that comprises a *spec*.
|
|
Keywords in the *DSL*, such as `describe`, `it`, and `expect`, are macros or methods.
|
|
Those macros and methods make calls to Spectator to describe the structure of a *spec*.
|
|
They are combined in such a way that makes it easy to read.
|
|
|
|
**Example**
|
|
|
|
An *example* is essentially a *test* and metadata.
|
|
Spectator makes a distinction between *test* and *example*.
|
|
An *example* can have a description, *group*, *context*, *result*, and *tags*.
|
|
That is to say: an *example* is the *test* and information for execution.
|
|
An *example* is a type of *node*.
|
|
|
|
In the *DSL*, an *example* is created with `example` and `it` blocks and their variants.
|
|
|
|
**Example Group**
|
|
|
|
An *example group* (or *group* for short), is a collection of *examples*.
|
|
*Groups* can be nested in other *groups*, but can only have one parent.
|
|
*Groups* can have *hooks*.
|
|
*Groups* have extra properties like a name and metadata.
|
|
A *group* is a type of *node*.
|
|
|
|
In the *DSL*, an *example group* is created with `example_group`, `describe`, and `context` blocks and their variants.
|
|
|
|
**Expectation**
|
|
|
|
An *expectation* captures a value or behavior and whether it satisfies a condition.
|
|
*Expectations* contain *match data*.
|
|
They are bubbled up from the *harness* to the runner.
|
|
*Expectations* can be thought of as wrappers for *assertions*.
|
|
|
|
In the *DSL*, an *expectation* is the code: `expect(THIS).to eq(THAT)`
|
|
An *expectation target* is just the `expect(THIS)` portion.
|
|
|
|
**Formatter**
|
|
|
|
A *formatter* takes *results* and reports them to the user in a specific format.
|
|
Examples of *formatters* are XML, HTML, JSON, dots, and documentation.
|
|
The runner will call methods on the *formatter*.
|
|
The methods called depend on the type of *result* and the state of the runner.
|
|
For instance, `#example_started` is called before a an example runs,
|
|
and `#dump_summary` is called at the end when all *results* are available.
|
|
|
|
**Harness**
|
|
|
|
A *harness* is used to safely wrap *test* code.
|
|
It captures *expectations* and creates a *result* based on the outcome.
|
|
|
|
**Filter**
|
|
|
|
A *filter* selects *nodes* to be included in a running *spec*.
|
|
There are multiple types of *filters*.
|
|
|
|
**Hook**
|
|
|
|
A *hook* is a piece of code to execute at a key time.
|
|
For instance, before a *test* starts, or after everything in a *group* completes.
|
|
*Hooks* can be run in the same *context* as a *test*.
|
|
These are known as "example hooks."
|
|
*Hooks* that don't run in a *context*, and instead run independent of *examples*, are called "example group hooks."
|
|
*Hooks* are attached to *groups*.
|
|
|
|
**Label**
|
|
|
|
A *label* is a string from the *spec* that identifies a expression.
|
|
*Labels* are captured to improve the readability of *results* and *match data*.
|
|
|
|
In the following code, the labels are: `does something useful`, `the_answer`, and `42`.
|
|
|
|
```crystal
|
|
it "does something useful" do
|
|
expect(the_answer).to eq(42)
|
|
end
|
|
```
|
|
|
|
**Matcher**
|
|
|
|
A *matcher* defines an expected value or behavior.
|
|
*Matchers* are given an "actual" value from a test and produce *match data*.
|
|
The *match data* contains information regarding whether the value or behavior was expected (satisfies a condition).
|
|
They behave similarly to an instance of a `Regex`.
|
|
|
|
In the following code, the `eq(42)` portion returns an instance of a *matcher* expecting the value 42.
|
|
|
|
```crystal
|
|
expect(the_answer).to eq(42)
|
|
```
|
|
|
|
**Match Data**
|
|
|
|
*Match data* is produced by *matchers*.
|
|
It contains information regarding whether an *expectation* is satisfied and values from the match.
|
|
The values are key-value pairs identifying things such as "expected value" and "actual value."
|
|
*Match data* is similar in concept to `Regex::MatchData`.
|
|
|
|
**Mock**
|
|
|
|
A *mock* is a type that can have its original functionality "swapped out" for a *stub*.
|
|
This allows complex types to be "mocked" so that other types can be unit tested.
|
|
*Mocks* can have any number of *stubs* defined.
|
|
They are similar to *doubles*, but use a real type.
|
|
|
|
**Node**
|
|
|
|
A *node* refers to any component in a *spec*.
|
|
*Nodes* are typically *examples* and *example groups*.
|
|
A *node* can have metadata associated with it, such as a *label*, location, and *tags*.
|
|
|
|
**Procsy**
|
|
|
|
A *procsy* is a glorified `Proc`.
|
|
It is used to wrap an underlying proc in some way.
|
|
Typically used to wrap an *example* when passed to a *hook*.
|
|
|
|
**Profile**
|
|
|
|
A *profile* includes timing information for *examples*.
|
|
It tracks how long each *example* took and sorts them.
|
|
|
|
**Report**
|
|
|
|
A *report* is a collection of *results* generated by running *examples*.
|
|
It provides easy access to various metrics and types of *results*.
|
|
|
|
**Result**
|
|
|
|
A *result* summarizes the outcome of running an *example*.
|
|
A *result* can be passing, failing, or pending.
|
|
*Results* contain timing information, *expectations* processed in the *example*, and an error for failing *results*.
|
|
|
|
**Spec**
|
|
|
|
A *spec* is a collection of *examples*.
|
|
Conceptually, a *spec* defines the behavior of a system.
|
|
A *spec* consists of a single, root *example group* that provides a tree structure of *examples*.
|
|
A *spec* also contains some *config* describing how to run it.
|
|
|
|
**Stub**
|
|
|
|
A *stub* is a method in a *double* or *mock* that replaces the original functionality.
|
|
*Stubs* can be attached to a single instance or all instances of a type.
|
|
|
|
**Tag**
|
|
|
|
A *tag* is an identifier with optional value.
|
|
*Tags* can be used to group and filter *examples* and *example groups*.
|
|
Some *tags* have special meaning, like `skip` indicating an *example* or *group* should be skipped.
|
|
|
|
**Test**
|
|
|
|
The word "test" is overused, especially when using a testing framework.
|
|
We make an effort to avoid using the word "test" for everything.
|
|
However, *test* has a technical meaning in Spectator.
|
|
It refers to the code (block) executed in an *example*.
|
|
|
|
```crystal
|
|
it "does a thing" do
|
|
# Test code starts here. Everything inside this `it` block is considered a test.
|
|
expect(the_answer).to eq(42)
|
|
# Test code ends here.
|
|
end
|
|
```
|
|
|
|
## DSL
|
|
|
|
The DSL is made up of methods and macros.
|
|
What look like keywords (`describe`, `it`, `expect`, `eq`, etc.) are just macros and methods provided by the DSL.
|
|
Those macros and methods are defined in multiple modules in the `Spectator::DSL` namespace.
|
|
They are logically grouped by their functionality.
|
|
|
|
Each module is included (as a mix-in) to the base Spectator context that all tests use.
|
|
The `SpectatorTestContext` class includes all of the DSL modules.
|
|
|
|
The DSL methods and macros should be kept simple.
|
|
Their functionality should be off-loaded to internal Spectator "APIs."
|
|
For instance, when the DSL creates an example,
|
|
it defines a method for the test code and calls another method to register it.
|
|
While Crystal macros are powerful, excessive use of them makes maintenance harder and compilation slower.
|
|
Additionally, by keeping logic out of the DSL, testing of internals becomes less dependent on the DSL.
|
|
|
|
*TODO:* Builders...
|
|
|
|
*TODO:* Tricks...
|
|
|
|
## Matchers
|
|
|
|
*TODO:* Base types
|
|
|
|
## Examples and groups
|
|
|
|
*TODO*
|
|
|
|
## Runner and harness
|
|
|
|
*TODO*
|
|
|
|
### Hooks
|
|
|
|
*TODO*
|
|
|
|
## Mocks and doubles
|
|
|
|
*TODO*
|
|
|
|
### Stubs
|
|
|
|
*TODO*
|
|
|
|
### Doubles
|
|
|
|
*TODO*
|
|
|
|
## Formatting
|
|
|
|
*TODO*
|