diff --git a/.gitignore b/.gitignore index e29dae7..c4166ba 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,6 @@ # Libraries don't need dependency lock # Dependencies will be locked in application that uses them /shard.lock + +# Ignore JUnit output +output.xml diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 630d59b..14d3678 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -14,21 +14,37 @@ before_script: spec: script: - shards - - crystal spec --error-on-warnings + - crystal spec --error-on-warnings --junit_output=. + artifacts: + when: always + paths: + - output.xml + reports: + junit: output.xml + +format: + script: + - shards + - crystal tool format --check style: script: - shards - bin/ameba - - crystal tool format --check nightly: image: "crystallang/crystal:nightly" allow_failure: true script: - shards --ignore-crystal-version - - crystal spec --error-on-warnings + - crystal spec --error-on-warnings --junit_output=. - crystal tool format --check + artifacts: + when: always + paths: + - output.xml + reports: + junit: output.xml pages: stage: deploy diff --git a/.guardian.yml b/.guardian.yml index ad67b38..f283d96 100644 --- a/.guardian.yml +++ b/.guardian.yml @@ -1,5 +1,11 @@ -files: ./**/*.cr +files: ./src/**/*.cr run: time crystal spec --error-trace --- +files: ./src/**/*.cr +run: bin/ameba %file% +--- +files: ./spec/**/*.cr +run: time crystal spec --error-trace %file% +--- files: ./shard.yml run: shards diff --git a/CHANGELOG.md b/CHANGELOG.md index 1204814..4dfbdaa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,58 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [Unreleased] +### Fixed +- Fix resolution of types with the same name in nested scopes. [#31](https://github.com/icy-arctic-fox/spectator/issues/31) +- `around_each` hooks wrap `before_all` and `after_all` hooks. [#12](https://github.com/icy-arctic-fox/spectator/issues/12) +- Hook execution order has been tweaked to match RSpec. + +### Added +- `before_each`, `after_each`, and `around_each` hooks are yielded the current example as a block argument. +- The `let` and `subject` blocks are yielded the current example as a block argument. +- Add internal logging that uses Crystal's `Log` utility. Provide the `LOG_LEVEL` environment variable to enable. +- Support dynamic creation of examples. +- Capture and log information for hooks. +- Tags can be added to examples and example groups. +- Add matcher to check compiled type of values. +- Examples can be skipped by using a `:pending` tag. A reason method can be specified: `pending: "Some excuse"` +- Examples without a test block are marked as pending. [#37](https://gitlab.com/arctic-fox/spectator/-/issues/37) +- Examples can be skipped during execution by using `skip` or `pending` in the example block. [#17](https://gitlab.com/arctic-fox/spectator/-/issues/17) +- Sample blocks can be temporarily skipped by using `xsample` or `xrandom_sample`. +- Add `before_suite` and `after_suite` hooks. [#21](https://gitlab.com/arctic-fox/spectator/-/issues/21) +- Support defining hooks in `Spectator.configure` block. [#21](https://gitlab.com/arctic-fox/spectator/-/issues/21) +- Examples with failures or skipped during execution will report the location of that result. [#57](https://gitlab.com/arctic-fox/spectator/-/issues/57) +- Support custom messages for failed expectations. [#28](https://gitlab.com/arctic-fox/spectator/-/issues/28) +- Allow named arguments and assignments for `provided` (`given`) block. +- Add `aggregate_failures` to capture and report multiple failed expectations. [#24](https://gitlab.com/arctic-fox/spectator/-/issues/24) +- Supports matching groups. [#25](https://gitlab.com/arctic-fox/spectator/-/issues/25) [#24](https://github.com/icy-arctic-fox/spectator/issues/24) +- Add `filter_run_including`, `filter_run_excluding`, and `filter_run_when_matching` to config block. +- By default, only run tests when any are marked with `focus: true`. +- Add "f-prefix" blocks for examples and groups (`fit`, `fdescribe`, etc.) as a short-hand for specifying `focus: true`. +- Add HTML formatter. Operates the same as the JUnit formatter. Specify `--html_output=DIR` to use. [#22](https://gitlab.com/arctic-fox/spectator/-/issues/22) [#3](https://github.com/icy-arctic-fox/spectator/issues/3) + +### Changed +- `given` (now `provided`) blocks changed to produce a single example. `it` can no longer be nested in a `provided` block. +- The "should" syntax no longer reports the source as inside Spectator. +- Short-hand "should" syntax must be included by using `require "spectator/should"` - `it { should eq("foo") }` +- Better error messages and detection when DSL methods are used when they shouldn't (i.e. `describe` inside `it`). +- Prevent usage of reserved keywords in DSL (such as `initialize`). +- The count argument for `sample` and `random_sample` groups must be named (use `count: 5` instead of just `5`). +- Helper methods used as arguments for `sample` and `random_sample` must be class methods. +- Simplify and reduce instanced types and generics. Should speed up compilation times. +- Overhaul example creation and handling. +- Overhaul storage of test values. +- Overhaul reporting and formatting. Cleaner output for failures and pending tests. +- Cleanup and simplify DSL implementation. +- Other minor internal improvements and cleanup. + +### Deprecated +- `pending` blocks will behave differently in v0.11.0. They will mimic RSpec in that they _compile and run_ the block expecting it to fail. Use a `skip` (or `xit`) block instead to prevent compiling the example. +- `given` has been renamed to `provided`. The `given` keyword may be reused later for memoization. + +### Removed +- Removed one-liner `it`-syntax without braces (block). + ## [0.9.40] - 2021-07-10 ### Fixed - Fix stubbing of class methods. diff --git a/README.md b/README.md index 1186e20..a366c9b 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ Add this to your application's `shard.yml`: development_dependencies: spectator: gitlab: arctic-fox/spectator - version: ~> 0.9.40 + version: ~> 0.10.0 ``` Usage @@ -277,13 +277,14 @@ For details on mocks and doubles, see the [wiki](https://gitlab.com/arctic-fox/s Spectator matches Crystal's default Spec output with some minor changes. JUnit and TAP are also supported output formats. -There is also a highly detailed JSON output. +There are also highly detailed JSON and HTML outputs. Development ----------- -This shard is still under development and is not recommended for production use (same as Crystal). -However, feel free to play around with it and use it for non-critical projects. +This shard is still in active development. +New features are being added and existing functionality improved. +Spectator is well-tested, but may have some yet-to-be-found bugs. ### Feature Progress @@ -344,20 +345,20 @@ Items not marked as completed may have partial implementations. - [ ] Message ordering - `expect().to receive().ordered` - [X] Null doubles - [X] Verifying doubles -- [ ] Runner +- [X] Runner - [X] Fail fast - - [ ] Test filtering - by name, context, and tags + - [X] Test filtering - by name, context, and tags - [X] Fail on no tests - [X] Randomize test order - [X] Dry run - for validation and checking formatted output - [X] Config block in `spec_helper.cr` - [X] Config file - `.spectator` -- [ ] Reporter and formatting +- [X] Reporter and formatting - [X] RSpec/Crystal Spec default - [X] JSON - [X] JUnit - [X] TAP - - [ ] HTML + - [X] HTML ### How it Works (in a nutshell) @@ -383,9 +384,10 @@ The CI build checks for properly formatted code. Documentation is automatically generated and published to GitLab pages. It can be found here: https://arctic-fox.gitlab.io/spectator -This project is developed on [GitLab](https://gitlab.com/arctic-fox/spectator), -and mirrored to [GitHub](https://github.com/icy-arctic-fox/spectator). -Issues and PRs/MRs are accepted on both. +This project's home is (and primarily developed) on [GitLab](https://gitlab.com/arctic-fox/spectator). +A mirror is maintained to [GitHub](https://github.com/icy-arctic-fox/spectator). +Issues, pull requests (merge requests), and discussion are welcome on both. +Maintainers will ensure your contributions make it in. ### Testing diff --git a/shard.yml b/shard.yml index bd1f405..cc50ced 100644 --- a/shard.yml +++ b/shard.yml @@ -1,5 +1,5 @@ name: spectator -version: 0.9.40 +version: 0.10.0 description: | A feature-rich spec testing framework for Crystal with similarities to RSpec. @@ -13,4 +13,4 @@ license: MIT development_dependencies: ameba: github: crystal-ameba/ameba - version: ~> 0.13.1 + version: ~> 0.14.3 diff --git a/spec/custom_message_spec.cr b/spec/custom_message_spec.cr new file mode 100644 index 0000000..64ca32b --- /dev/null +++ b/spec/custom_message_spec.cr @@ -0,0 +1,31 @@ +require "./spec_helper" + +Spectator.describe Spectator do + it "supports custom expectation messages" do + expect do + expect(false).to be_true, "paradox!" + end.to raise_error(Spectator::ExampleFailed, "paradox!") + end + + it "supports custom expectation messages with a proc" do + count = 0 + expect do + expect(false).to be_true, ->{ count += 1; "Failed #{count} times" } + end.to raise_error(Spectator::ExampleFailed, "Failed 1 times") + end + + context "not_to" do + it "supports custom expectation messages" do + expect do + expect(true).not_to be_true, "paradox!" + end.to raise_error(Spectator::ExampleFailed, "paradox!") + end + + it "supports custom expectation messages with a proc" do + count = 0 + expect do + expect(true).not_to be_true, ->{ count += 1; "Failed #{count} times" } + end.to raise_error(Spectator::ExampleFailed, "Failed 1 times") + end + end +end diff --git a/spec/helpers/example.cr b/spec/helpers/example.cr new file mode 100644 index 0000000..dca1042 --- /dev/null +++ b/spec/helpers/example.cr @@ -0,0 +1,71 @@ +require "ecr" +require "json" +require "./result" + +module Spectator::SpecHelpers + # Wrapper for compiling and running an example at runtime and getting a result. + class Example + # Creates the example. + # The *spec_helper_path* is the path to spec_helper.cr file. + # The name or ID of the example is given by *example_id*. + # Lastly, the source code for the example is given by *example_code*. + def initialize(@spec_helper_path : String, @example_id : String, @example_code : String) + end + + # Instructs the Crystal compiler to compile the test. + # Returns an instance of `JSON::Any`. + # This will be the outcome and information about the test. + # Output will be surpressed for the test. + # If an error occurs while attempting to compile and run the test, an error will be raised. + def compile + # Create a temporary file containing the test. + with_tempfile do |source_file| + args = ["run", "--no-color", source_file, "--", "--json"] + Process.run(crystal_executable, args) do |process| + JSON.parse(process.output) + rescue JSON::ParseException + raise "Compilation of example #{@example_id} failed\n\n#{process.error.gets_to_end}" + end + end + end + + # Same as `#compile`, but returns the result of the first example in the test. + # Returns a `SpectatorHelpers::Result` instance. + def result + output = compile + example = output["examples"][0] + Result.from_json_any(example) + end + + # Constructs the string representation of the example. + # This produces the Crystal source code. + # *io* is the file handle to write to. + # The *dir* is the directory of the file being written to. + # This is needed to resolve the relative path to the spec_helper.cr file. + private def write(io, dir) + spec_helper_path = Path[@spec_helper_path].relative_to(dir) # ameba:disable Lint/UselessAssign + ECR.embed(__DIR__ + "/example.ecr", io) + end + + # Creates a temporary file containing the compilable example code. + # Yields the path of the temporary file. + # Ensures the file is deleted after it is done being used. + private def with_tempfile + tempfile = File.tempfile("_#{@example_id}_spec.cr") do |file| + dir = File.dirname(file.path) + write(file, dir) + end + + begin + yield tempfile.path + ensure + tempfile.delete + end + end + + # Attempts to find the Crystal compiler on the system or raises an error. + private def crystal_executable + Process.find_executable("crystal") || raise("Could not find Crystal compiler") + end + end +end diff --git a/spec/helpers/example.ecr b/spec/helpers/example.ecr new file mode 100644 index 0000000..53355bf --- /dev/null +++ b/spec/helpers/example.ecr @@ -0,0 +1,5 @@ +require "<%= spec_helper_path %>" + +Spectator.describe "<%= @example_id %>" do + <%= @example_code %> +end diff --git a/spec/helpers/expectation.cr b/spec/helpers/expectation.cr new file mode 100644 index 0000000..fd4d84d --- /dev/null +++ b/spec/helpers/expectation.cr @@ -0,0 +1,28 @@ +module Spectator::SpecHelpers + # Information about an `expect` call in an example. + struct Expectation + # Indicates whether the expectation passed or failed. + getter? satisfied : Bool + + # Message when the expectation failed. + # Only available when `#satisfied?` is false. + getter! message : String + + # Additional information about the expectation. + # Only available when `#satisfied?` is false. + getter! values : Hash(String, String) + + # Creates the expectation outcome. + def initialize(@satisfied, @message, @values) + end + + # Extracts the expectation information from a `JSON::Any` object. + def self.from_json_any(object : JSON::Any) + satisfied = object["satisfied"].as_bool + message = object["failure"]?.try(&.as_s?) + values = object["values"]?.try(&.as_h?) + values = values.transform_values(&.as_s) if values + new(satisfied, message, values) + end + end +end diff --git a/spec/helpers/result.cr b/spec/helpers/result.cr new file mode 100644 index 0000000..edafd75 --- /dev/null +++ b/spec/helpers/result.cr @@ -0,0 +1,67 @@ +module Spectator::SpecHelpers + # Information about an example compiled and run at runtime. + struct Result + # Status of the example after running. + enum Outcome + Success + Failure + Error + Unknown + end + + # Full name and description of the example. + getter name : String + + # Status of the example after running. + getter outcome : Outcome + + # List of expectations ran in the example. + getter expectations : Array(Expectation) + + # Creates the result. + def initialize(@name, @outcome, @expectations) + end + + # Checks if the example was successful. + def success? + outcome.success? + end + + # :ditto: + def successful? + outcome.success? + end + + # Checks if the example failed, but did not error. + def failure? + outcome.failure? + end + + # Checks if the example encountered an error. + def error? + outcome.error? + end + + # Extracts the result information from a `JSON::Any` object. + def self.from_json_any(object : JSON::Any) + name = object["description"].as_s + outcome = parse_outcome_string(object["status"].as_s) + expectations = if (list = object["expectations"].as_a?) + list.map { |e| Expectation.from_json_any(e) } + else + [] of Expectation + end + new(name, outcome, expectations) + end + + # Converts a result string, such as "fail" to an enum value. + private def self.parse_outcome_string(string) + case string + when /pass/i then Outcome::Success + when /fail/i then Outcome::Failure + when /error/i then Outcome::Error + else Outcome::Unknown + end + end + end +end diff --git a/spec/line_number_spec.cr b/spec/line_number_spec.cr index ead7dd2..5439ea0 100644 --- a/spec/line_number_spec.cr +++ b/spec/line_number_spec.cr @@ -1,29 +1,29 @@ require "./spec_helper" Spectator.describe Spectator do - let(current_example) { ::Spectator::Harness.current.example } - subject(source) { current_example.source } + let(current_example) { ::Spectator::Example.current } + subject(location) { current_example.location } context "line numbers" do it "contains starting line of spec" do - expect(source.line).to eq(__LINE__ - 1) + expect(location.line).to eq(__LINE__ - 1) end it "contains ending line of spec" do - expect(source.end_line).to eq(__LINE__ + 1) + expect(location.end_line).to eq(__LINE__ + 1) end it "handles multiple lines and examples" do # Offset is important. - expect(source.line).to eq(__LINE__ - 2) + expect(location.line).to eq(__LINE__ - 2) # This line fails, refer to https://github.com/crystal-lang/crystal/issues/10562 - # expect(source.end_line).to eq(__LINE__ + 2) + # expect(location.end_line).to eq(__LINE__ + 2) # Offset is still important. end end context "file names" do - subject { source.file } + subject { location.file } it "match source code" do is_expected.to eq(__FILE__) diff --git a/spec/rspec/core/explicit_subject_spec.cr b/spec/rspec/core/explicit_subject_spec.cr index bc102e8..825f6bf 100644 --- a/spec/rspec/core/explicit_subject_spec.cr +++ b/spec/rspec/core/explicit_subject_spec.cr @@ -34,16 +34,16 @@ Spectator.describe "Explicit Subject" do subject { @@element_list.pop } - # TODO: RSpec calls the "actual" block after the "change block". - xit "is memoized across calls (i.e. the block is invoked once)" do + skip "is memoized across calls (i.e. the block is invoked once)", + reason: "RSpec calls the \"actual\" block after the \"change block\"." do expect do 3.times { subject } end.to change { @@element_list }.from([1, 2, 3]).to([1, 2]) expect(subject).to eq(3) end - # TODO: RSpec calls the "actual" block after the "change block". - xit "is not memoized across examples" do + skip "is not memoized across examples", + reason: "RSpec calls the \"actual\" block after the \"change block\"." do expect { subject }.to change { @@element_list }.from([1, 2]).to([1]) expect(subject).to eq(2) end diff --git a/spec/rspec/core/let_spec.cr b/spec/rspec/core/let_spec.cr index 815306d..a711fa9 100644 --- a/spec/rspec/core/let_spec.cr +++ b/spec/rspec/core/let_spec.cr @@ -11,7 +11,7 @@ Spectator.describe "Let and let!" do describe "let" do let(:count) { @@count += 1 } - it "memoizes thte value" do + it "memoizes the value" do expect(count).to eq(1) expect(count).to eq(1) end diff --git a/spec/rspec/expectations/all_matcher_spec.cr b/spec/rspec/expectations/all_matcher_spec.cr index 1d910e9..92d31d3 100644 --- a/spec/rspec/expectations/all_matcher_spec.cr +++ b/spec/rspec/expectations/all_matcher_spec.cr @@ -21,17 +21,15 @@ Spectator.describe "`all` matcher" do # Changed `include` to `contain` to match our own. # `include` is a keyword and can't be used as a method name in Crystal. - # TODO: Add support for compound matchers. describe ["anything", "everything", "something"] do - xit { is_expected.to all(be_a(String)) } # .and contain("thing") ) } - xit { is_expected.to all(be_a(String)) } # .and end_with("g") ) } - xit { is_expected.to all(start_with("s")) } # .or contain("y") ) } + skip reason: "Add support for compound matchers." { is_expected.to all(be_a(String).and contain("thing")) } + skip reason: "Add support for compound matchers." { is_expected.to all(be_a(String).and end_with("g")) } + skip reason: "Add support for compound matchers." { is_expected.to all(start_with("s").or contain("y")) } # deliberate failures - # TODO: Add support for compound matchers. - xit { is_expected.to all(contain("foo")) } # .and contain("bar") ) } - xit { is_expected.to all(be_a(String)) } # .and start_with("a") ) } - xit { is_expected.to all(start_with("a")) } # .or contain("z") ) } + skip reason: "Add support for compound matchers." { is_expected.to all(contain("foo").and contain("bar")) } + skip reason: "Add support for compound matchers." { is_expected.to all(be_a(String).and start_with("a")) } + skip reason: "Add support for compound matchers." { is_expected.to all(start_with("a").or contain("z")) } end end end diff --git a/spec/rspec/expectations/contain_matcher_spec.cr b/spec/rspec/expectations/contain_matcher_spec.cr index 6c8e558..b95d961 100644 --- a/spec/rspec/expectations/contain_matcher_spec.cr +++ b/spec/rspec/expectations/contain_matcher_spec.cr @@ -12,16 +12,14 @@ Spectator.describe "`contain` matcher" do it { is_expected.to contain(1, 7) } it { is_expected.to contain(1, 3, 7) } - # Utility matcher method `a_kind_of` is not supported. - # it { is_expected.to contain(a_kind_of(Int)) } + skip reason: "Utility matcher method `a_kind_of` is not supported." { is_expected.to contain(a_kind_of(Int)) } - # TODO: Compound matchers aren't supported. - # it { is_expected.to contain(be_odd.and be < 10) } + skip reason: "Compound matchers aren't supported." { is_expected.to contain(be_odd.and be < 10) } # TODO: Fix behavior and cleanup output. # This syntax is allowed, but produces a wrong result and bad output. - xit { is_expected.to contain(be_odd) } - xit { is_expected.not_to contain(be_even) } + skip reason: "Fix behavior and cleanup output." { is_expected.to contain(be_odd) } + skip reason: "Fix behavior and cleanup output." { is_expected.not_to contain(be_even) } it { is_expected.not_to contain(17) } it { is_expected.not_to contain(43, 100) } @@ -62,35 +60,31 @@ Spectator.describe "`contain` matcher" do subject { {:a => 7, :b => 5} } # Hash syntax is changed here from `:a => 7` to `a: 7`. - # it { is_expected.to contain(:a) } - # it { is_expected.to contain(:b, :a) } - - # TODO: This hash-like syntax isn't supported. - # it { is_expected.to contain(a: 7) } - # it { is_expected.to contain(b: 5, a: 7) } - # it { is_expected.not_to contain(:c) } - # it { is_expected.not_to contain(:c, :d) } - # it { is_expected.not_to contain(d: 2) } - # it { is_expected.not_to contain(a: 5) } - # it { is_expected.not_to contain(b: 7, a: 5) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(a: 7) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(b: 5, a: 7) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(:c) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(:c, :d) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(d: 2) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(a: 5) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(b: 7, a: 5) } # deliberate failures - # it { is_expected.not_to contain(:a) } - # it { is_expected.not_to contain(:b, :a) } - # it { is_expected.not_to contain(a: 7) } - # it { is_expected.not_to contain(a: 7, b: 5) } - # it { is_expected.to contain(:c) } - # it { is_expected.to contain(:c, :d) } - # it { is_expected.to contain(d: 2) } - # it { is_expected.to contain(a: 5) } - # it { is_expected.to contain(a: 5, b: 7) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(:a) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(:b, :a) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(a: 7) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(a: 7, b: 5) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(:c) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(:c, :d) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(d: 2) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(a: 5) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(a: 5, b: 7) } # Mixed cases--the hash contains one but not the other. # All 4 of these cases should fail. - # it { is_expected.to contain(:a, :d) } - # it { is_expected.not_to contain(:a, :d) } - # it { is_expected.to contain(a: 7, d: 3) } - # it { is_expected.not_to contain(a: 7, d: 3) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(:a, :d) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(:a, :d) } + skip reason: "This hash-like syntax isn't supported." { is_expected.to contain(a: 7, d: 3) } + skip reason: "This hash-like syntax isn't supported." { is_expected.not_to contain(a: 7, d: 3) } end end end diff --git a/spec/rspec/expectations/end_with_matcher_spec.cr b/spec/rspec/expectations/end_with_matcher_spec.cr index 75e6fd1..92d927a 100644 --- a/spec/rspec/expectations/end_with_matcher_spec.cr +++ b/spec/rspec/expectations/end_with_matcher_spec.cr @@ -18,10 +18,9 @@ Spectator.describe "`end_with` matcher" do context "array usage" do describe [0, 1, 2, 3, 4] do it { is_expected.to end_with 4 } - # TODO: Add support for multiple items at the end of an array. - # it { is_expected.to end_with 3, 4 } + skip reason: "Add support for multiple items at the end of an array." { is_expected.to end_with 3, 4 } it { is_expected.not_to end_with 3 } - # it { is_expected.not_to end_with 0, 1, 2, 3, 4, 5 } + skip reason: "Add support for multiple items at the end of an array." { is_expected.not_to end_with 0, 1, 2, 3, 4, 5 } # deliberate failures it_fails { is_expected.not_to end_with 4 } diff --git a/spec/rspec/expectations/have_attributes_matcher_spec.cr b/spec/rspec/expectations/have_attributes_matcher_spec.cr index 81c11dc..7bcb441 100644 --- a/spec/rspec/expectations/have_attributes_matcher_spec.cr +++ b/spec/rspec/expectations/have_attributes_matcher_spec.cr @@ -14,14 +14,14 @@ Spectator.describe "`have_attributes` matcher" do # Spectator doesn't support helper matchers like `a_string_starting_with` and `a_value <`. # But maybe in the future it will. it { is_expected.to have_attributes(name: "Jim") } - # it { is_expected.to have_attributes(name: a_string_starting_with("J") ) } + skip reason: "Add support for fuzzy matchers." { is_expected.to have_attributes(name: a_string_starting_with("J")) } it { is_expected.to have_attributes(age: 32) } - # it { is_expected.to have_attributes(age: (a_value > 30) ) } + skip reason: "Add support for fuzzy matchers." { is_expected.to have_attributes(age: (a_value > 30)) } it { is_expected.to have_attributes(name: "Jim", age: 32) } - # it { is_expected.to have_attributes(name: a_string_starting_with("J"), age: (a_value > 30) ) } + skip reason: "Add support for fuzzy matchers." { is_expected.to have_attributes(name: a_string_starting_with("J"), age: (a_value > 30)) } it { is_expected.not_to have_attributes(name: "Bob") } it { is_expected.not_to have_attributes(age: 10) } - # it { is_expected.not_to have_attributes(age: (a_value < 30) ) } + skip reason: "Add support for fuzzy matchers." { is_expected.not_to have_attributes(age: (a_value < 30)) } # deliberate failures it_fails { is_expected.to have_attributes(name: "Bob") } diff --git a/spec/rspec/expectations/raise_error_matcher_spec.cr b/spec/rspec/expectations/raise_error_matcher_spec.cr index 85678a7..1d262ce 100644 --- a/spec/rspec/expectations/raise_error_matcher_spec.cr +++ b/spec/rspec/expectations/raise_error_matcher_spec.cr @@ -75,14 +75,13 @@ Spectator.describe "`raise_error` matcher" do end end - # TODO: Support passing a block to `raise_error` matcher. - # context "set expectations on error object passed to block" do - # it "raises DivisionByZeroError" do - # expect { 42 // 0 }.to raise_error do |error| - # expect(error).to be_a(DivisionByZeroError) - # end - # end - # end + context "set expectations on error object passed to block" do + skip "raises DivisionByZeroError", reason: "Support passing a block to `raise_error` matcher." do + expect { 42 // 0 }.to raise_error do |error| + expect(error).to be_a(DivisionByZeroError) + end + end + end context "expect no error at all" do describe "#to_s" do diff --git a/spec/rspec/expectations/start_with_matcher_spec.cr b/spec/rspec/expectations/start_with_matcher_spec.cr index 471f1d6..74f8f8a 100644 --- a/spec/rspec/expectations/start_with_matcher_spec.cr +++ b/spec/rspec/expectations/start_with_matcher_spec.cr @@ -18,10 +18,9 @@ Spectator.describe "`start_with` matcher" do context "with an array" do describe [0, 1, 2, 3, 4] do it { is_expected.to start_with 0 } - # TODO: Add support for multiple items at the beginning of an array. - # it { is_expected.to start_with(0, 1) } + skip reason: "Add support for multiple items at the beginning of an array." { is_expected.to start_with(0, 1) } it { is_expected.not_to start_with(2) } - # it { is_expected.not_to start_with(0, 1, 2, 3, 4, 5) } + skip reason: "Add support for multiple items at the beginning of an array." { is_expected.not_to start_with(0, 1, 2, 3, 4, 5) } # deliberate failures it_fails { is_expected.not_to start_with 0 } diff --git a/spec/runtime_example_spec.cr b/spec/runtime_example_spec.cr new file mode 100644 index 0000000..01ae9a3 --- /dev/null +++ b/spec/runtime_example_spec.cr @@ -0,0 +1,58 @@ +require "./spec_helper" + +# This is a meta test that ensures specs can be compiled and run at runtime. +# The purpose of this is to report an error if this process fails. +# Other tests will fail, but display a different name/description of the test. +# This clearly indicates that runtime testing failed. +# +# Runtime compilation is used to get output of tests as well as check syntax. +# Some specs are too complex to be ran normally. +# Additionally, this allows examples to easily check specific failure cases. +# Plus, it makes testing user-reported issues easy. +Spectator.describe "Runtime compilation", :slow, :compile do + given_example passing_example do + it "does something" do + expect(true).to be_true + end + end + + it "can compile and retrieve the result of an example" do + expect(passing_example).to be_successful + end + + it "can retrieve expectations" do + expect(passing_example.expectations).to_not be_empty + end + + given_example failing_example do + it "does something" do + expect(true).to be_false + end + + it "doesn't run" do + expect(true).to be_false + end + end + + it "detects failed examples" do + expect(failing_example).to be_failure + end + + given_example malformed_example do + it "does something" do + asdf + end + end + + it "raises on compilation errors" do + expect { malformed_example }.to raise_error(/compilation/i) + end + + given_expectation satisfied_expectation do + expect(true).to be_true + end + + it "can compile and retrieve expectations" do + expect(satisfied_expectation).to be_satisfied + end +end diff --git a/spec/spec_helper.cr b/spec/spec_helper.cr index f8047da..6c9a806 100644 --- a/spec/spec_helper.cr +++ b/spec/spec_helper.cr @@ -1,4 +1,6 @@ require "../src/spectator" +require "../src/spectator/should" +require "./helpers/**" macro it_fails(description = nil, &block) it {{description}} do @@ -11,3 +13,35 @@ end macro specify_fails(description = nil, &block) it_fails {{description}} {{block}} end + +# Defines an example ("it" block) that is lazily compiled. +# When the example is referenced with *id*, it will be compiled and the results retrieved. +# The value returned by *id* will be a `Spectator::SpecHelpers::Result`. +# This allows the test result to be inspected. +macro given_example(id, &block) + let({{id}}) do + ::Spectator::SpecHelpers::Example.new( + {{__FILE__}}, + {{id.id.stringify}}, + {{block.body.stringify}} + ).result + end +end + +# Defines an example ("it" block) that is lazily compiled. +# The "it" block must be omitted, as the block provided to this macro will be wrapped in one. +# When the expectation is referenced with *id*, it will be compiled and the result retrieved. +# The value returned by *id* will be a `Spectator::SpecHelpers::Expectation`. +# This allows an expectation to be inspected. +# Only the last expectation performed will be returned. +# An error is raised if no expectations ran. +macro given_expectation(id, &block) + let({{id}}) do + result = ::Spectator::SpecHelpers::Example.new( + {{__FILE__}}, + {{id.id.stringify}}, + {{"it do\n" + block.body.stringify + "\nend"}} + ).result + result.expectations.last || raise("No expectations found from {{id.id}}") + end +end diff --git a/spec/spectator/aggregate_failures_spec.cr b/spec/spectator/aggregate_failures_spec.cr new file mode 100644 index 0000000..0c4506c --- /dev/null +++ b/spec/spectator/aggregate_failures_spec.cr @@ -0,0 +1,41 @@ +require "../spec_helper" + +Spectator.describe Spectator do + describe "aggregate_failures" do + it "captures multiple failed expectations" do + expect do + aggregate_failures do + expect(true).to be_false + expect(false).to be_true + end + end.to raise_error(Spectator::MultipleExpectationsFailed, /2 failures/) + end + + it "raises normally for one failed expectation" do + expect do + aggregate_failures do + expect(true).to be_false + expect(true).to be_true + end + end.to raise_error(Spectator::ExpectationFailed) + end + + it "doesn't raise when there are no failed expectations" do + expect do + aggregate_failures do + expect(false).to be_false + expect(true).to be_true + end + end.to_not raise_error(Spectator::ExpectationFailed) + end + + it "supports naming the block" do + expect do + aggregate_failures "contradiction" do + expect(true).to be_false + expect(false).to be_true + end + end.to raise_error(Spectator::MultipleExpectationsFailed, /contradiction/) + end + end +end diff --git a/spec/spectator/anything_spec.cr b/spec/spectator/anything_spec.cr new file mode 100644 index 0000000..3140f7e --- /dev/null +++ b/spec/spectator/anything_spec.cr @@ -0,0 +1,24 @@ +require "../spec_helper" + +Spectator.describe Spectator::Anything do + it "matches everything" do + expect(true).to match(subject) + expect(false).to match(subject) + expect(nil).to match(subject) + expect(42).to match(subject) + expect(42.as(Int32 | String)).to match(subject) + expect(["foo", "bar"]).to match(subject) + end + + describe "#to_s" do + subject { super.to_s } + + it { is_expected.to contain("anything") } + end + + describe "#inspect" do + subject { super.inspect } + + it { is_expected.to contain("anything") } + end +end diff --git a/spec/spectator/block_spec.cr b/spec/spectator/block_spec.cr new file mode 100644 index 0000000..a459b02 --- /dev/null +++ b/spec/spectator/block_spec.cr @@ -0,0 +1,34 @@ +require "../spec_helper" + +Spectator.describe Spectator::Block do + describe "#value" do + it "calls the block" do + called = false + block = described_class.new { called = true } + expect { block.value }.to change { called }.to(true) + end + + it "can be called multiple times (doesn't cache the value)" do + count = 0 + block = described_class.new { count += 1 } + block.value # Call once, count should be 1. + expect { block.value }.to change { count }.from(1).to(2) + end + end + + describe "#to_s" do + let(block) do + described_class.new("Test Label") { 42 } + end + + subject { block.to_s } + + it "contains the label" do + is_expected.to contain("Test Label") + end + + it "contains the value" do + is_expected.to contain("42") + end + end +end diff --git a/spec/spectator/concise_spec.cr b/spec/spectator/concise_spec.cr new file mode 100644 index 0000000..c472a6e --- /dev/null +++ b/spec/spectator/concise_spec.cr @@ -0,0 +1,55 @@ +require "../spec_helper" + +Spectator.describe Spectator do + context "consice syntax" do + describe "provided group with a single assignment" do + provided x = 42 do + expect(x).to eq(42) + end + end + + describe "provided group with multiple assignments" do + provided x = 42, y = 123 do + expect(x).to eq(42) + expect(y).to eq(123) + end + end + + describe "provided group with a single named argument" do + provided x: 42 do + expect(x).to eq(42) + end + end + + describe "provided group with multiple named arguments" do + provided x: 42, y: 123 do + expect(x).to eq(42) + expect(y).to eq(123) + end + end + + describe "provided group with mix of assignments and named arguments" do + provided x = 42, y: 123 do + expect(x).to eq(42) + expect(y).to eq(123) + end + + provided x = 42, y = 123, z: 0, foo: "bar" do + expect(x).to eq(42) + expect(y).to eq(123) + expect(z).to eq(0) + expect(foo).to eq("bar") + end + end + + describe "provided group with references to other arguments" do + let(foo) { "bar" } + + provided x = 3, y: x * 5, baz: foo.sub('r', 'z') do + expect(x).to eq(3) + expect(y).to eq(15) + expect(baz).to eq("baz") + end + end + end +end diff --git a/spec/spectator/lazy_spec.cr b/spec/spectator/lazy_spec.cr new file mode 100644 index 0000000..0f138bd --- /dev/null +++ b/spec/spectator/lazy_spec.cr @@ -0,0 +1,15 @@ +require "../spec_helper" + +Spectator.describe Spectator::Lazy do + it "returns the value of the block" do + lazy = Spectator::Lazy(Int32).new + expect { lazy.get { 42 } }.to eq(42) + end + + it "caches the value" do + lazy = Spectator::Lazy(Int32).new + count = 0 + expect { lazy.get { count += 1 } }.to change { count }.from(0).to(1) + expect { lazy.get { count += 1 } }.to_not change { count } + end +end diff --git a/spec/spectator/lazy_wrapper_spec.cr b/spec/spectator/lazy_wrapper_spec.cr new file mode 100644 index 0000000..94782b4 --- /dev/null +++ b/spec/spectator/lazy_wrapper_spec.cr @@ -0,0 +1,28 @@ +require "../spec_helper" + +Spectator.describe Spectator::LazyWrapper do + it "returns the value of the block" do + expect { subject.get { 42 } }.to eq(42) + end + + it "caches the value" do + wrapper = described_class.new + count = 0 + expect { wrapper.get { count += 1 } }.to change { count }.from(0).to(1) + expect { wrapper.get { count += 1 } }.to_not change { count } + end + + # This type of nesting is used when `super` is called in a subject block. + # ``` + # subject { super.to_s } + # ``` + it "works with nested wrappers" do + outer = described_class.new + inner = described_class.new + value = outer.get do + inner.get { 42 }.to_s + end + expect(value).to eq("42") + expect(value).to be_a(String) + end +end diff --git a/spec/spectator/value_spec.cr b/spec/spectator/value_spec.cr new file mode 100644 index 0000000..246e453 --- /dev/null +++ b/spec/spectator/value_spec.cr @@ -0,0 +1,36 @@ +require "../spec_helper" + +Spectator.describe Spectator::Value do + subject { described_class.new(42, "Test Label") } + + it "stores the value" do + # NOTE: This cast is a workaround for [issue #55](https://gitlab.com/arctic-fox/spectator/-/issues/55) + value = subject.as(Spectator::Value(Int32)).value + expect(value).to eq(42) + end + + describe "#to_s" do + subject { super.to_s } + + it "contains the label" do + is_expected.to contain("Test Label") + end + + it "contains the value" do + is_expected.to contain("42") + end + end + + describe "#inspect" do + let(value) { described_class.new([42], "Test Label") } + subject { value.inspect } + + it "contains the label" do + is_expected.to contain("Test Label") + end + + it "contains the value" do + is_expected.to contain("[42]") + end + end +end diff --git a/spec/spectator/wrapper_spec.cr b/spec/spectator/wrapper_spec.cr new file mode 100644 index 0000000..aa2351c --- /dev/null +++ b/spec/spectator/wrapper_spec.cr @@ -0,0 +1,18 @@ +require "../spec_helper" + +Spectator.describe Spectator::Wrapper do + it "stores a value" do + wrapper = described_class.new(42) + expect(wrapper.get(Int32)).to eq(42) + end + + it "retrieves a value using the block trick" do + wrapper = described_class.new(Int32) + expect(wrapper.get { Int32 }).to eq(Int32) + end + + it "raises on invalid cast" do + wrapper = described_class.new(42) + expect { wrapper.get(String) }.to raise_error(TypeCastError) + end +end diff --git a/src/spectator.cr b/src/spectator.cr index 7a94fbc..0459ae4 100644 --- a/src/spectator.cr +++ b/src/spectator.cr @@ -1,54 +1,18 @@ +require "colorize" +require "log" require "./spectator/includes" -require "./spectator_test" # Module that contains all functionality related to Spectator. module Spectator extend self + include DSL::Top # Current version of the Spectator library. - VERSION = "0.9.40" + VERSION = {{ `shards version #{__DIR__}`.stringify.chomp }} - # Top-level describe method. - # All specs in a file must be wrapped in this call. - # This takes an argument and a block. - # The argument is what your spec is describing. - # It can be any Crystal expression, - # but is typically a class name or feature string. - # The block should contain all of the specs for what is being described. - # Example: - # ``` - # Spectator.describe Foo do - # # Your specs for `Foo` go here. - # end - # ``` - # NOTE: Inside the block, the `Spectator` prefix is no longer needed. - # Actually, prefixing methods and macros with `Spectator` - # most likely won't work and can cause compiler errors. - macro describe(description, &block) - # This macro creates the foundation for all specs. - # Every group of examples is defined a separate module - `SpectatorExamples`. - # There's multiple reasons for this. - # - # The first reason is to provide namespace isolation. - # We don't want the spec code to accidentally pickup types and values from the `Spectator` module. - # Another reason is that we need a root module to put all examples and groups in. - # And lastly, the spec DSL needs to be given to the block of code somehow. - # The DSL is included in the `SpectatorTest` class. - # - # For more information on how the DSL works, see the `DSL` module. - - # Root-level class that contains all examples and example groups. - class SpectatorTest - # Pass off the description argument and block to `DSL::StructureDSL.describe`. - # That method will handle creating a new group for this spec. - describe({{description}}) {{block}} - end - end - - # :ditto: - macro context(description, &block) - describe({{description}}) {{block}} - end + # Logger for Spectator internals. + ::Log.setup_from_env + Log = ::Log.for(self) # Flag indicating whether Spectator should automatically run tests. # This should be left alone (set to true) in typical usage. @@ -79,14 +43,14 @@ module Spectator exit(1) if autorun? && !run end - @@config_builder = ConfigBuilder.new + @@config_builder = Config::Builder.new @@config : Config? # Provides a means to configure how Spectator will run and report tests. # A `ConfigBuilder` is yielded to allow changing the configuration. # NOTE: The configuration set here can be overriden # with a `.spectator` file and command-line arguments. - def configure : Nil + def configure(& : Config::Builder -> _) : Nil yield @@config_builder end @@ -98,38 +62,29 @@ module Spectator config.random end - # Trick for detecting if a constant is defined. - # Includes the block of code if the *constant* is defined. - private macro on_defined(constant) - {% if constant.resolve? %} - {{yield}} - {% end %} - end - # Builds the tests and runs the framework. private def run - # Silence default logger, only if it's used somewhere in the program. - on_defined(::Log) do - ::Log.setup_from_env(default_level: :none) - end + # Silence default logger. + ::Log.setup_from_env(default_level: :none) - # Build the test suite and run it. - suite = ::Spectator::SpecBuilder.build(config.example_filter) - Runner.new(suite, config).run + # Build the spec and run it. + spec = DSL::Builder.build + spec.run rescue ex + # Re-enable logger for fatal error. + ::Log.setup_from_env + # Catch all unhandled exceptions here. # Examples are already wrapped, so any exceptions they throw are caught. # But if an exception occurs outside an example, # it's likely the fault of the test framework (Spectator). # So we display a helpful error that could be reported and return non-zero. - display_error_stack(ex) + Log.fatal(exception: ex) { "Spectator encountered an unexpected error" } false end - # Processes and builds up a configuration to use for running tests. - private def config - @@config ||= build_config - end + # Global configuration used by Spectator for running tests. + class_getter(config) { build_config } # Builds the configuration. private def build_config @@ -155,33 +110,11 @@ module Spectator private def apply_config_file(file_path = CONFIG_FILE_PATH) : Nil return unless File.exists?(file_path) args = File.read(file_path).lines - CommandLineArgumentsConfigSource.new(args).apply_to(@@config_builder) + Config::CLIArgumentsApplicator.new(args).apply(@@config_builder) end # Applies configuration options from the command-line arguments private def apply_command_line_args : Nil - CommandLineArgumentsConfigSource.new.apply_to(@@config_builder) - end - - # Displays a complete error stack. - # Prints an error and everything that caused it. - # Stacktrace is included. - private def display_error_stack(error) : Nil - puts - puts "Encountered an unexpected error in framework" - # Loop while there's a cause for the error. - # Print each error in the stack. - loop do - display_error(error) - error = error.cause - break unless error - end - end - - # Display a single error and its stacktrace. - private def display_error(error) : Nil - puts - puts "Caused by: #{error.message}" - puts error.backtrace.join("\n") + Config::CLIArgumentsApplicator.new.apply(@@config_builder) end end diff --git a/src/spectator/abstract_expression.cr b/src/spectator/abstract_expression.cr new file mode 100644 index 0000000..7985c16 --- /dev/null +++ b/src/spectator/abstract_expression.cr @@ -0,0 +1,53 @@ +require "./label" + +module Spectator + # Represents an expression from a test. + # This is typically captured by an `expect` macro. + # It consists of a label and the value of the expression. + # The label should be a string recognizable by the user, + # or nil if one isn't available. + # + # This base class is provided so that all generic sub-classes can be stored as this one type. + # The value of the expression can be retrieved by downcasting to the expected type with `#cast`. + # + # NOTE: This is intentionally a class and not a struct. + # If it were a struct, changes made to the value held by an instance may not be kept when passing it around. + # See commit ca564619ad2ae45f832a058d514298c868fdf699. + abstract class AbstractExpression + # User recognizable string for the expression. + # This can be something like a variable name or a snippet of Crystal code. + getter label : Label + + # Creates the expression. + # The *label* is usually the Crystal code evaluating to the `#raw_value`. + # It can be nil if it isn't available. + def initialize(@label : Label) + end + + # Retrieves the evaluated value of the expression. + abstract def raw_value + + # Attempts to cast `#raw_value` to the type *T* and return it. + def cast(type : T.class) : T forall T + raw_value.as(T) + end + + # Produces a string representation of the expression. + # This consists of the label (if one is available) and the value. + def to_s(io) + if (label = @label) + io << label << ": " + end + raw_value.to_s(io) + end + + # Produces a detailed string representation of the expression. + # This consists of the label (if one is available) and the value. + def inspect(io) + if (label = @label) + io << label << ": " + end + raw_value.inspect(io) + end + end +end diff --git a/src/spectator/anything.cr b/src/spectator/anything.cr index 511a024..e4d7b34 100644 --- a/src/spectator/anything.cr +++ b/src/spectator/anything.cr @@ -1,15 +1,25 @@ module Spectator + # Type dedicated to matching everything. + # This is intended to be used as a value to compare against when the value doesn't matter. + # Can be used like so: + # ``` + # anything = Spectator::Anything.new + # expect("foo").to match(anything) + # ``` struct Anything - def ==(other) - true - end - + # Always returns true. def ===(other) true end - def =~(other) - true + # Displays "anything". + def to_s(io) + io << "anything" + end + + # Displays "". + def inspect(io) + io << "" end end end diff --git a/src/spectator/block.cr b/src/spectator/block.cr new file mode 100644 index 0000000..215d098 --- /dev/null +++ b/src/spectator/block.cr @@ -0,0 +1,34 @@ +require "./expression" +require "./label" + +module Spectator + # Represents a block from a test. + # This is typically captured by an `expect` macro. + # It consists of a label and parameterless block. + # The label should be a string recognizable by the user, + # or nil if one isn't available. + class Block(T) < Expression(T) + # Creates the block expression from a proc. + # The *proc* will be called to evaluate the value of the expression. + # The *label* is usually the Crystal code for the *proc*. + # It can be nil if it isn't available. + def initialize(@block : -> T, label : Label = nil) + super(label) + end + + # Creates the block expression by capturing a block as a proc. + # The block will be called to evaluate the value of the expression. + # The *label* is usually the Crystal code for the *block*. + # It can be nil if it isn't available. + def initialize(label : Label = nil, &@block : -> T) + super(label) + end + + # Evaluates the block and returns the value from it. + # This method _does not_ cache the resulting value like `#value` does. + # Successive calls to this method may return different values. + def value : T + @block.call + end + end +end diff --git a/src/spectator/command_line_arguments_config_source.cr b/src/spectator/command_line_arguments_config_source.cr deleted file mode 100644 index ce3c7e5..0000000 --- a/src/spectator/command_line_arguments_config_source.cr +++ /dev/null @@ -1,178 +0,0 @@ -require "option_parser" - -module Spectator - # Generates configuration from the command-line arguments. - class CommandLineArgumentsConfigSource < ConfigSource - # Creates the configuration source. - # By default, the command-line arguments (ARGV) are used. - # But custom arguments can be passed in. - def initialize(@args : Array(String) = ARGV) - end - - # Applies the specified configuration to a builder. - # Calling this method from multiple sources builds up the final configuration. - def apply_to(builder : ConfigBuilder) : Nil - OptionParser.parse(@args) do |parser| - control_parser_options(parser, builder) - filter_parser_options(parser, builder) - output_parser_options(parser, builder) - end - end - - # Adds options to the parser for controlling the test execution. - private def control_parser_options(parser, builder) - fail_fast_option(parser, builder) - fail_blank_option(parser, builder) - dry_run_option(parser, builder) - random_option(parser, builder) - seed_option(parser, builder) - order_option(parser, builder) - end - - # Adds the fail-fast option to the parser. - private def fail_fast_option(parser, builder) - parser.on("-f", "--fail-fast", "Stop testing on first failure") do - builder.fail_fast - end - end - - # Adds the fail-blank option to the parser. - private def fail_blank_option(parser, builder) - parser.on("-b", "--fail-blank", "Fail if there are no examples") do - builder.fail_blank - end - end - - # Adds the dry-run option to the parser. - private def dry_run_option(parser, builder) - parser.on("-d", "--dry-run", "Don't run any tests, output what would have run") do - builder.dry_run - end - end - - # Adds the randomize examples option to the parser. - private def random_option(parser, builder) - parser.on("-r", "--rand", "Randomize the execution order of tests") do - builder.randomize - end - end - - # Adds the random seed option to the parser. - private def seed_option(parser, builder) - parser.on("--seed INTEGER", "Set the seed for the random number generator (implies -r)") do |seed| - builder.randomize - builder.seed = seed.to_u64 - end - end - - # Adds the example order option to the parser. - private def order_option(parser, builder) - parser.on("--order ORDER", "Set the test execution order. ORDER should be one of: defined, rand, or rand:SEED") do |method| - case method.downcase - when "defined" - builder.randomize = false - when /^rand/ - builder.randomize - parts = method.split(':', 2) - builder.seed = parts[1].to_u64 if parts.size > 1 - else - nil - end - end - end - - # Adds options to the parser for filtering examples. - private def filter_parser_options(parser, builder) - example_option(parser, builder) - line_option(parser, builder) - location_option(parser, builder) - end - - # Adds the example filter option to the parser. - private def example_option(parser, builder) - parser.on("-e", "--example STRING", "Run examples whose full nested names include STRING") do |pattern| - filter = NameExampleFilter.new(pattern) - builder.add_example_filter(filter) - end - end - - # Adds the line filter option to the parser. - private def line_option(parser, builder) - parser.on("-l", "--line LINE", "Run examples whose line matches LINE") do |line| - filter = LineExampleFilter.new(line.to_i) - builder.add_example_filter(filter) - end - end - - # Adds the location filter option to the parser. - private def location_option(parser, builder) - parser.on("--location FILE:LINE", "Run the example at line 'LINE' in the file 'FILE', multiple allowed") do |location| - source = Source.parse(location) - filter = SourceExampleFilter.new(source) - builder.add_example_filter(filter) - end - end - - # Adds options to the parser for changing output. - private def output_parser_options(parser, builder) - verbose_option(parser, builder) - help_option(parser, builder) - profile_option(parser, builder) - json_option(parser, builder) - tap_option(parser, builder) - junit_option(parser, builder) - no_color_option(parser, builder) - end - - # Adds the verbose output option to the parser. - private def verbose_option(parser, builder) - parser.on("-v", "--verbose", "Verbose output using document formatter") do - builder.formatter = Formatting::DocumentFormatter.new - end - end - - # Adds the help output option to the parser. - private def help_option(parser, builder) - parser.on("-h", "--help", "Show this help") do - puts parser - exit - end - end - - # Adds the profile output option to the parser. - private def profile_option(parser, builder) - parser.on("-p", "--profile", "Display the 10 slowest specs") do - builder.profile - end - end - - # Adds the JSON output option to the parser. - private def json_option(parser, builder) - parser.on("--json", "Generate JSON output") do - builder.formatter = Formatting::JsonFormatter.new - end - end - - # Adds the TAP output option to the parser. - private def tap_option(parser, builder) - parser.on("--tap", "Generate TAP output (Test Anything Protocol)") do - builder.formatter = Formatting::TAPFormatter.new - end - end - - # Adds the JUnit output option to the parser. - private def junit_option(parser, builder) - parser.on("--junit_output OUTPUT_DIR", "Generate JUnit XML output") do |output_dir| - formatter = Formatting::JUnitFormatter.new(output_dir) - builder.add_formatter(formatter) - end - end - - # Adds the "no color" output option to the parser. - private def no_color_option(parser, builder) - parser.on("--no-color", "Disable colored output") do - Colorize.enabled = false - end - end - end -end diff --git a/src/spectator/composite_example_filter.cr b/src/spectator/composite_example_filter.cr deleted file mode 100644 index 7f776ba..0000000 --- a/src/spectator/composite_example_filter.cr +++ /dev/null @@ -1,13 +0,0 @@ -module Spectator - # Filter that combines multiple other filters. - class CompositeExampleFilter < ExampleFilter - # Creates the example filter. - def initialize(@filters : Array(ExampleFilter)) - end - - # Checks whether the example satisfies the filter. - def includes?(example) : Bool - @filters.any?(&.includes?(example)) - end - end -end diff --git a/src/spectator/composite_node_filter.cr b/src/spectator/composite_node_filter.cr new file mode 100644 index 0000000..1450bcc --- /dev/null +++ b/src/spectator/composite_node_filter.cr @@ -0,0 +1,15 @@ +require "./node_filter" + +module Spectator + # Filter that combines multiple other filters. + class CompositeNodeFilter < NodeFilter + # Creates the example filter. + def initialize(@filters : Array(NodeFilter)) + end + + # Checks whether the node satisfies the filter. + def includes?(node) : Bool + @filters.any?(&.includes?(node)) + end + end +end diff --git a/src/spectator/config.cr b/src/spectator/config.cr index a44564b..66e6501 100644 --- a/src/spectator/config.cr +++ b/src/spectator/config.cr @@ -1,51 +1,120 @@ +require "./config/*" +require "./node_filter" +require "./example_group" +require "./filtered_example_iterator" +require "./formatting/formatter" +require "./node_iterator" +require "./run_flags" + module Spectator # Provides customization and describes specifics for how Spectator will run and report tests. class Config - @formatters : Array(Formatting::Formatter) + # Primary formatter all events will be sent to. + getter formatter : Formatting::Formatter - # Indicates whether the test should abort on first failure. - getter? fail_fast : Bool + # Flags indicating how the spec should run. + getter run_flags : RunFlags - # Indicates whether the test should fail if there are no examples. - getter? fail_blank : Bool + # Seed used for random number generation. + getter random_seed : UInt64 - # Indicates whether the test should be done as a dry-run. - # Examples won't run, but the output will show that they did. - getter? dry_run : Bool + # Filter used to select which examples to run. + getter node_filter : NodeFilter - # Random number generator to use for everything. - getter random : Random + # Filter used to select which examples to _not_ run. + getter node_reject : NodeFilter - # Indicates whether tests are run in a random order. - getter? randomize : Bool + # Tags to filter on if they're present in a spec. + protected getter match_filters : Metadata - # Random seed used for number generation. - getter! random_seed : UInt64? + # List of hooks to run before all examples in the test suite. + protected getter before_suite_hooks : Deque(ExampleGroupHook) - # Indicates whether profiling information should be displayed. - getter? profile : Bool + # List of hooks to run before each top-level example group. + protected getter before_all_hooks : Deque(ExampleGroupHook) - # Filter that determines which examples to run. - getter example_filter : ExampleFilter + # List of hooks to run before every example. + protected getter before_each_hooks : Deque(ExampleHook) + + # List of hooks to run after all examples in the test suite. + protected getter after_suite_hooks : Deque(ExampleGroupHook) + + # List of hooks to run after each top-level example group. + protected getter after_all_hooks : Deque(ExampleGroupHook) + + # List of hooks to run after every example. + protected getter after_each_hooks : Deque(ExampleHook) + + # List of hooks to run around every example. + protected getter around_each_hooks : Deque(ExampleProcsyHook) # Creates a new configuration. - def initialize(builder) - @formatters = builder.formatters - @fail_fast = builder.fail_fast? - @fail_blank = builder.fail_blank? - @dry_run = builder.dry_run? - @random = builder.random - @randomize = builder.randomize? - @random_seed = builder.seed? - @profile = builder.profile? - @example_filter = builder.example_filter + # Properties are pulled from *source*. + # Typically, *source* is a `Config::Builder`. + def initialize(source) + @formatter = source.formatter + @run_flags = source.run_flags + @random_seed = source.random_seed + @node_filter = source.node_filter + @node_reject = source.node_reject + @match_filters = source.match_filters + + @before_suite_hooks = source.before_suite_hooks + @before_all_hooks = source.before_all_hooks + @before_each_hooks = source.before_each_hooks + @after_suite_hooks = source.after_suite_hooks + @after_all_hooks = source.after_all_hooks + @after_each_hooks = source.after_each_hooks + @around_each_hooks = source.around_each_hooks end - # Yields each formatter that should be reported to. - def each_formatter - @formatters.each do |formatter| - yield formatter + # Produces the default configuration. + def self.default : self + Builder.new.build + end + + # Shuffles the items in an array using the configured random settings. + # If `#randomize?` is true, the *items* are shuffled and returned as a new array. + # Otherwise, the items are left alone and returned as-is. + # The array of *items* is never modified. + def shuffle(items) + return items unless run_flags.randomize? + + items.shuffle(random) + end + + # Shuffles the items in an array using the configured random settings. + # If `#randomize?` is true, the *items* are shuffled and returned. + # Otherwise, the items are left alone and returned as-is. + # The array of *items* is modified, the items are shuffled in-place. + def shuffle!(items) + return items unless run_flags.randomize? + + items.shuffle!(random) + end + + # Creates an iterator configured to select the filtered examples. + def iterator(group : ExampleGroup) + match_filter = match_filter(group) + iterator = FilteredExampleIterator.new(group, @node_filter) + iterator = iterator.select(match_filter) if match_filter + iterator.reject(@node_reject) + end + + # Creates a node filter if any conditionally matching filters apply to an example group. + private def match_filter(group : ExampleGroup) : NodeFilter? + iterator = NodeIterator.new(group) + filters = @match_filters.compact_map do |key, value| + filter = TagNodeFilter.new(key.to_s, value) + filter.as(NodeFilter) if iterator.rewind.any?(filter) end + CompositeNodeFilter.new(filters) unless filters.empty? + end + + # Retrieves the configured random number generator. + # This will produce the same generator with the same seed every time. + def random + Random.new(random_seed) end end end diff --git a/src/spectator/config/builder.cr b/src/spectator/config/builder.cr new file mode 100644 index 0000000..7866fc6 --- /dev/null +++ b/src/spectator/config/builder.cr @@ -0,0 +1,317 @@ +require "../composite_node_filter" +require "../node_filter" +require "../formatting" +require "../metadata" +require "../null_node_filter" +require "../run_flags" +require "../tag_node_filter" + +module Spectator + class Config + # Mutable configuration used to produce a final configuration. + # Use the setters in this class to incrementally build a configuration. + # Then call `#build` to create the final configuration. + class Builder + # Seed used for random number generation. + property random_seed : UInt64 = Random.rand(100000_u64) + + # Toggles indicating how the test spec should execute. + property run_flags = RunFlags::None + + protected getter match_filters : Metadata = {:focus => nil.as(String?)} + + @primary_formatter : Formatting::Formatter? + @additional_formatters = [] of Formatting::Formatter + @filters = [] of NodeFilter + @rejects = [] of NodeFilter + + # List of hooks to run before all examples in the test suite. + protected getter before_suite_hooks = Deque(ExampleGroupHook).new + + # List of hooks to run before each top-level example group. + protected getter before_all_hooks = Deque(ExampleGroupHook).new + + # List of hooks to run before every example. + protected getter before_each_hooks = Deque(ExampleHook).new + + # List of hooks to run after all examples in the test suite. + protected getter after_suite_hooks = Deque(ExampleGroupHook).new + + # List of hooks to run after each top-level example group. + protected getter after_all_hooks = Deque(ExampleGroupHook).new + + # List of hooks to run after every example. + protected getter after_each_hooks = Deque(ExampleHook).new + + # List of hooks to run around every example. + protected getter around_each_hooks = Deque(ExampleProcsyHook).new + + # Attaches a hook to be invoked before all examples in the test suite. + def add_before_suite_hook(hook) + @before_suite_hooks.push(hook) + end + + # Defines a block of code to execute before all examples in the test suite. + def before_suite(&block) + hook = ExampleGroupHook.new(&block) + add_before_suite_hook(hook) + end + + # Attaches a hook to be invoked before each top-level example group. + def add_before_all_hook(hook) + @before_all_hooks.push(hook) + end + + # Defines a block of code to execute before each top-level example group. + def before_all(&block) + hook = ExampleGroupHook.new(&block) + add_before_all_hook(hook) + end + + # Attaches a hook to be invoked before every example. + # The current example is provided as a block argument. + def add_before_each_hook(hook) + @before_each_hooks.push(hook) + end + + # Defines a block of code to execute before every. + # The current example is provided as a block argument. + def before_each(&block : Example -> _) + hook = ExampleHook.new(&block) + add_before_each_hook(hook) + end + + # Attaches a hook to be invoked after all examples in the test suite. + def add_after_suite_hook(hook) + @after_suite_hooks.unshift(hook) + end + + # Defines a block of code to execute after all examples in the test suite. + def after_suite(&block) + hook = ExampleGroupHook.new(&block) + add_after_suite_hook(hook) + end + + # Attaches a hook to be invoked after each top-level example group. + def add_after_all_hook(hook) + @after_all_hooks.unshift(hook) + end + + # Defines a block of code to execute after each top-level example group. + def after_all(&block) + hook = ExampleGroupHook.new(&block) + add_after_all_hook(hook) + end + + # Attaches a hook to be invoked after every example. + # The current example is provided as a block argument. + def add_after_each_hook(hook) + @after_each_hooks.unshift(hook) + end + + # Defines a block of code to execute after every example. + # The current example is provided as a block argument. + def after_each(&block : Example -> _) + hook = ExampleHook.new(&block) + add_after_each_hook(hook) + end + + # Attaches a hook to be invoked around every example. + # The current example in procsy form is provided as a block argument. + def add_around_each_hook(hook) + @around_each_hooks.push(hook) + end + + # Defines a block of code to execute around every example. + # The current example in procsy form is provided as a block argument. + def around_each(&block : Example::Procsy -> _) + hook = ExampleProcsyHook.new(label: "around_each", &block) + add_around_each_hook(hook) + end + + # Creates a configuration. + def build : Config + Config.new(self) + end + + # Sets the primary formatter to use for reporting test progress and results. + def formatter=(formatter : Formatting::Formatter) + @primary_formatter = formatter + end + + # Adds an extra formatter to use for reporting test progress and results. + def add_formatter(formatter : Formatting::Formatter) + @additional_formatters << formatter + end + + # Retrieves the formatters to use. + # If one wasn't specified by the user, + # then `#default_formatter` is returned. + private def formatters + @additional_formatters + [(@primary_formatter || default_formatter)] + end + + # The formatter that should be used if one wasn't provided. + private def default_formatter + Formatting::ProgressFormatter.new + end + + # A single formatter that will satisfy the configured output. + # If one formatter was configured, then it is returned. + # Otherwise, a `Formatting::BroadcastFormatter` is returned. + protected def formatter + case (formatters = self.formatters) + when .one? then formatters.first + else Formatting::BroadcastFormatter.new(formatters) + end + end + + # Enables fail-fast mode. + def fail_fast + @run_flags |= RunFlags::FailFast + end + + # Sets the fail-fast flag. + def fail_fast=(flag) + if flag + @run_flags |= RunFlags::FailFast + else + @run_flags &= ~RunFlags::FailFast + end + end + + # Indicates whether fail-fast mode is enabled. + protected def fail_fast? + @run_flags.fail_fast? + end + + # Enables fail-blank mode (fail on no tests). + def fail_blank + @run_flags |= RunFlags::FailBlank + end + + # Enables or disables fail-blank mode. + def fail_blank=(flag) + if flag + @run_flags |= RunFlags::FailBlank + else + @run_flags &= ~RunFlags::FailBlank + end + end + + # Indicates whether fail-fast mode is enabled. + # That is, it is a failure if there are no tests. + protected def fail_blank? + @run_flags.fail_blank? + end + + # Enables dry-run mode. + def dry_run + @run_flags |= RunFlags::DryRun + end + + # Enables or disables dry-run mode. + def dry_run=(flag) + if flag + @run_flags |= RunFlags::DryRun + else + @run_flags &= ~RunFlags::DryRun + end + end + + # Indicates whether dry-run mode is enabled. + # In this mode, no tests are run, but output acts like they were. + protected def dry_run? + @run_flags.dry_run? + end + + # Randomizes test execution order. + def randomize + @run_flags |= RunFlags::Randomize + end + + # Enables or disables running tests in a random order. + def randomize=(flag) + if flag + @run_flags |= RunFlags::Randomize + else + @run_flags &= ~RunFlags::Randomize + end + end + + # Indicates whether tests are run in a random order. + protected def randomize? + @run_flags.randomize? + end + + # Displays profiling information + def profile + @run_flags |= RunFlags::Profile + end + + # Enables or disables displaying profiling information. + def profile=(flag) + if flag + @run_flags |= RunFlags::Profile + else + @run_flags &= ~RunFlags::Profile + end + end + + # Indicates whether profiling information should be displayed. + protected def profile? + @run_flags.profile? + end + + # Adds a filter to determine which examples can run. + def add_node_filter(filter : NodeFilter) + @filters << filter + end + + # Specifies one or more tags to constrain running examples to. + def filter_run_including(*tags : Symbol, **values) + tags.each { |tag| @filters << TagNodeFilter.new(tag) } + values.each { |tag, value| @filters << TagNodeFilter.new(tag, value.to_s) } + end + + # Adds a filter to prevent examples from running. + def add_node_reject(filter : NodeFilter) + @rejects << filter + end + + # Specifies one or more tags to exclude from running examples. + def filter_run_excluding(*tags : Symbol, **values) + tags.each { |tag| @rejects << TagNodeFilter.new(tag) } + values.each { |tag, value| @rejects << TagNodeFilter.new(tag, value.to_s) } + end + + # Specifies one or more tags to filter on only if they're present in the spec. + def filter_run_when_matching(*tags : Symbol, **values) + tags.each { |tag| @match_filters[tag] = nil } + values.each { |tag, value| @match_filters[tag] = value.to_s } + end + + # Retrieves a filter that determines which examples can run. + # If no filters were added with `#add_node_filter`, + # then the returned filter will allow all examples to be run. + protected def node_filter + case (filters = @filters) + when .empty? then NullNodeFilter.new + when .one? then filters.first + else CompositeNodeFilter.new(filters) + end + end + + # Retrieves a filter that prevents examples from running. + # If no filters were added with `#add_node_reject`, + # then the returned filter will allow all examples to be run. + protected def node_reject + case (filters = @rejects) + when .empty? then NullNodeFilter.new(false) + when .one? then filters.first + else CompositeNodeFilter.new(filters) + end + end + end + end +end diff --git a/src/spectator/config/cli_arguments_applicator.cr b/src/spectator/config/cli_arguments_applicator.cr new file mode 100644 index 0000000..496fc1a --- /dev/null +++ b/src/spectator/config/cli_arguments_applicator.cr @@ -0,0 +1,240 @@ +require "colorize" +require "option_parser" +require "../formatting" +require "../line_node_filter" +require "../location" +require "../location_node_filter" +require "../name_node_filter" +require "../tag_node_filter" + +module Spectator + class Config + # Applies command-line arguments to a configuration. + class CLIArgumentsApplicator + # Logger for this class. + Log = Spectator::Log.for("config") + + # Creates the configuration source. + # By default, the command-line arguments (ARGV) are used. + # But custom arguments can be passed in. + def initialize(@args : Array(String) = ARGV) + end + + # Applies the specified configuration to a builder. + # Calling this method from multiple sources builds up the final configuration. + def apply(builder) : Nil + OptionParser.parse(@args) do |parser| + control_parser_options(parser, builder) + filter_parser_options(parser, builder) + output_parser_options(parser, builder) + end + end + + # Adds options to the parser for controlling the test execution. + private def control_parser_options(parser, builder) + fail_fast_option(parser, builder) + fail_blank_option(parser, builder) + dry_run_option(parser, builder) + random_option(parser, builder) + seed_option(parser, builder) + order_option(parser, builder) + end + + # Adds the fail-fast option to the parser. + private def fail_fast_option(parser, builder) + parser.on("-f", "--fail-fast", "Stop testing on first failure") do + Log.debug { "Enabling fail-fast (-f)" } + builder.fail_fast + end + end + + # Adds the fail-blank option to the parser. + private def fail_blank_option(parser, builder) + parser.on("-b", "--fail-blank", "Fail if there are no examples") do + Log.debug { "Enabling fail-blank (-b)" } + builder.fail_blank + end + end + + # Adds the dry-run option to the parser. + private def dry_run_option(parser, builder) + parser.on("-d", "--dry-run", "Don't run any tests, output what would have run") do + Log.debug { "Enabling dry-run (-d)" } + builder.dry_run + end + end + + # Adds the randomize examples option to the parser. + private def random_option(parser, builder) + parser.on("-r", "--rand", "Randomize the execution order of tests") do + Log.debug { "Randomizing test order (-r)" } + builder.randomize + end + end + + # Adds the random seed option to the parser. + private def seed_option(parser, builder) + parser.on("--seed INTEGER", "Set the seed for the random number generator (implies -r)") do |seed| + Log.debug { "Randomizing test order and setting RNG seed to #{seed}" } + builder.randomize + builder.random_seed = seed.to_u64 + end + end + + # Adds the example order option to the parser. + private def order_option(parser, builder) + parser.on("--order ORDER", "Set the test execution order. ORDER should be one of: defined, rand, or rand:SEED") do |method| + case method.downcase + when "defined" + Log.debug { "Disabling randomized tests (--order defined)" } + builder.randomize = false + when /^rand/ + builder.randomize + parts = method.split(':', 2) + if (seed = parts[1]?) + Log.debug { "Randomizing test order and setting RNG seed to #{seed} (--order rand:#{seed})" } + builder.random_seed = seed.to_u64 + else + Log.debug { "Randomizing test order (--order rand)" } + end + end + end + end + + # Adds options to the parser for filtering examples. + private def filter_parser_options(parser, builder) + example_option(parser, builder) + line_option(parser, builder) + location_option(parser, builder) + tag_option(parser, builder) + end + + # Adds the example filter option to the parser. + private def example_option(parser, builder) + parser.on("-e", "--example STRING", "Run examples whose full nested names include STRING") do |pattern| + Log.debug { "Filtering for examples named '#{pattern}' (-e '#{pattern}')" } + filter = NameNodeFilter.new(pattern) + builder.add_node_filter(filter) + end + end + + # Adds the line filter option to the parser. + private def line_option(parser, builder) + parser.on("-l", "--line LINE", "Run examples whose line matches LINE") do |line| + Log.debug { "Filtering for examples on line #{line} (-l #{line})" } + filter = LineNodeFilter.new(line.to_i) + builder.add_node_filter(filter) + end + end + + # Adds the location filter option to the parser. + private def location_option(parser, builder) + parser.on("--location FILE:LINE", "Run the example at line 'LINE' in the file 'FILE', multiple allowed") do |location| + Log.debug { "Filtering for examples at #{location} (--location '#{location}')" } + location = Location.parse(location) + filter = LocationNodeFilter.new(location) + builder.add_node_filter(filter) + end + end + + # Adds the tag filter option to the parser. + private def tag_option(parser, builder) + parser.on("--tag TAG[:VALUE]", "Run examples with the specified TAG, or exclude examples by adding ~ before the TAG.") do |tag| + negated = tag.starts_with?('~') + tag = tag.lchop('~') + Log.debug { "Filtering for example with tag #{tag}" } + parts = tag.split(':', 2, remove_empty: true) + if parts.size > 1 + tag = parts.first + value = parts.last + end + + filter = TagNodeFilter.new(tag, value) + if negated + builder.add_node_reject(filter) + else + builder.add_node_filter(filter) + end + end + end + + # Adds options to the parser for changing output. + private def output_parser_options(parser, builder) + verbose_option(parser, builder) + help_option(parser, builder) + profile_option(parser, builder) + json_option(parser, builder) + tap_option(parser, builder) + junit_option(parser, builder) + html_option(parser, builder) + no_color_option(parser, builder) + end + + # Adds the verbose output option to the parser. + private def verbose_option(parser, builder) + parser.on("-v", "--verbose", "Verbose output using document formatter") do + Log.debug { "Setting output format to document (-v)" } + builder.formatter = Formatting::DocumentFormatter.new + end + end + + # Adds the help output option to the parser. + private def help_option(parser, builder) + parser.on("-h", "--help", "Show this help") do + puts parser + exit + end + end + + # Adds the profile output option to the parser. + private def profile_option(parser, builder) + parser.on("-p", "--profile", "Display the 10 slowest specs") do + Log.debug { "Enabling timing information (-p)" } + builder.profile + end + end + + # Adds the JSON output option to the parser. + private def json_option(parser, builder) + parser.on("--json", "Generate JSON output") do + Log.debug { "Setting output format to JSON (--json)" } + builder.formatter = Formatting::JSONFormatter.new + end + end + + # Adds the TAP output option to the parser. + private def tap_option(parser, builder) + parser.on("--tap", "Generate TAP output (Test Anything Protocol)") do + Log.debug { "Setting output format to TAP (--tap)" } + builder.formatter = Formatting::TAPFormatter.new + end + end + + # Adds the JUnit output option to the parser. + private def junit_option(parser, builder) + parser.on("--junit_output OUTPUT_DIR", "Generate JUnit XML output") do |output_dir| + Log.debug { "Setting output format to JUnit XML (--junit_output '#{output_dir}')" } + formatter = Formatting::JUnitFormatter.new(output_dir) + builder.add_formatter(formatter) + end + end + + # Adds the HTML output option to the parser. + private def html_option(parser, builder) + parser.on("--html_output OUTPUT_DIR", "Generate HTML output") do |output_dir| + Log.debug { "Setting output format to HTML (--html_output '#{output_dir}')" } + formatter = Formatting::HTMLFormatter.new(output_dir) + builder.add_formatter(formatter) + end + end + + # Adds the "no color" output option to the parser. + private def no_color_option(parser, builder) + parser.on("--no-color", "Disable colored output") do + Log.debug { "Disabling color output (--no-color)" } + Colorize.enabled = false + end + end + end + end +end diff --git a/src/spectator/config_builder.cr b/src/spectator/config_builder.cr deleted file mode 100644 index d65ffc1..0000000 --- a/src/spectator/config_builder.cr +++ /dev/null @@ -1,158 +0,0 @@ -module Spectator - # Mutable configuration used to produce a final configuration. - # Use the setters in this class to incrementally build a configuration. - # Then call `#build` to create the final configuration. - class ConfigBuilder - # Creates a default configuration. - def self.default - new.build - end - - # Random number generator to use. - protected getter random = Random::DEFAULT - - def initialize - @seed = seed = @random.rand(UInt16).to_u64 - @random.new_seed(seed) - end - - @primary_formatter : Formatting::Formatter? - @additional_formatters = [] of Formatting::Formatter - @fail_fast = false - @fail_blank = false - @dry_run = false - @randomize = false - @profile = false - @filters = [] of ExampleFilter - - # Sets the primary formatter to use for reporting test progress and results. - def formatter=(formatter : Formatting::Formatter) - @primary_formatter = formatter - end - - # Adds an extra formater to use for reporting test progress and results. - def add_formatter(formatter : Formatting::Formatter) - @additional_formatters << formatter - end - - # Retrieves the formatters to use. - # If one wasn't specified by the user, - # then `#default_formatter` is returned. - protected def formatters - @additional_formatters + [(@primary_formatter || default_formatter)] - end - - # The formatter that should be used, - # if one wasn't provided. - private def default_formatter - Formatting::DotsFormatter.new - end - - # Enables fail-fast mode. - def fail_fast - self.fail_fast = true - end - - # Sets the fail-fast flag. - def fail_fast=(flag) - @fail_fast = flag - end - - # Indicates whether fail-fast mode is enabled. - protected def fail_fast? - @fail_fast - end - - # Enables fail-blank mode (fail on no tests). - def fail_blank - self.fail_blank = true - end - - # Enables or disables fail-blank mode. - def fail_blank=(flag) - @fail_blank = flag - end - - # Indicates whether fail-fast mode is enabled. - # That is, it is a failure if there are no tests. - protected def fail_blank? - @fail_blank - end - - # Enables dry-run mode. - def dry_run - self.dry_run = true - end - - # Enables or disables dry-run mode. - def dry_run=(flag) - @dry_run = flag - end - - # Indicates whether dry-run mode is enabled. - # In this mode, no tests are run, but output acts like they were. - protected def dry_run? - @dry_run - end - - # Seed used for random number generation. - getter! seed : UInt64? - - # Sets the seed for the random number generator. - def seed=(seed) - @seed = seed - @random = Random.new(seed) - end - - # Randomizes test execution order. - def randomize - self.randomize = true - end - - # Enables or disables running tests in a random order. - def randomize=(flag) - @randomize = flag - end - - # Indicates whether tests are run in a random order. - protected def randomize? - @randomize - end - - # Displays profiling information - def profile - self.profile = true - end - - # Enables or disables displaying profiling information. - def profile=(flag) - @profile = flag - end - - # Indicates whether profiling information should be displayed. - protected def profile? - @profile - end - - # Adds a filter to determine which examples can run. - def add_example_filter(filter : ExampleFilter) - @filters << filter - end - - # Retrieves a filter that determines which examples can run. - # If no filters were added with `#add_example_filter`, - # then the returned filter will allow all examples to be run. - protected def example_filter - if @filters.empty? - NullExampleFilter.new - else - CompositeExampleFilter.new(@filters) - end - end - - # Creates a configuration. - def build : Config - Config.new(self) - end - end -end diff --git a/src/spectator/config_source.cr b/src/spectator/config_source.cr deleted file mode 100644 index 40a243f..0000000 --- a/src/spectator/config_source.cr +++ /dev/null @@ -1,8 +0,0 @@ -module Spectator - # Interface for all places that configuration can originate. - abstract class ConfigSource - # Applies the specified configuration to a builder. - # Calling this method from multiple sources builds up the final configuration. - abstract def apply_to(builder : ConfigBuilder) : Nil - end -end diff --git a/src/spectator/context.cr b/src/spectator/context.cr new file mode 100644 index 0000000..c3213e3 --- /dev/null +++ b/src/spectator/context.cr @@ -0,0 +1,32 @@ +# Base class that all test cases run in. +# This type is used to store all test case contexts as a single type. +# The instance must be downcast to the correct type before calling a context method. +# This type is intentionally outside the `Spectator` module. +# The reason for this is to prevent name collision when using the DSL to define a spec. +abstract class SpectatorContext + # Produces a dummy string to represent the context as a string. + # This prevents the default behavior, which normally stringifies instance variables. + # Due to the sheer amount of types Spectator can create + # and that the Crystal compiler instantiates a `#to_s` and/or `#inspect` for each of those types, + # an explosion in method instances can be created. + # The compile time is drastically reduced by using a dummy string instead. + def to_s(io) + io << "Context" + end + + # :ditto: + def inspect(io) + io << "Context<" << self.class << '>' + end +end + +module Spectator + # Base class that all test cases run in. + # This type is used to store all test case contexts as a single type. + # The instance must be downcast to the correct type before calling a context method. + # + # Nested contexts, such as those defined by `context` and `describe` in the DSL, can define their own methods. + # The intent is that a proc will downcast to the correct type and call one of those methods. + # This is how methods that contain test cases, hooks, and other context-specific code blocks get invoked. + alias Context = ::SpectatorContext +end diff --git a/src/spectator/context_delegate.cr b/src/spectator/context_delegate.cr new file mode 100644 index 0000000..ac927b9 --- /dev/null +++ b/src/spectator/context_delegate.cr @@ -0,0 +1,27 @@ +require "./context" +require "./context_method" +require "./null_context" + +module Spectator + # Stores a test context and a method to call within it. + struct ContextDelegate + # Creates the delegate. + # The *context* is the instance of the test context. + # The *method* is proc that downcasts *context* and calls a method on it. + def initialize(@context : Context, @method : ContextMethod) + end + + # Creates a delegate with a null context. + # The context will be ignored and the block will be executed in its original scope. + def self.null(&block : -> _) + context = NullContext.new + method = ContextMethod.new { block.call } + new(context, method) + end + + # Invokes a method in the test context. + def call + @method.call(@context) + end + end +end diff --git a/src/spectator/context_method.cr b/src/spectator/context_method.cr new file mode 100644 index 0000000..30cce6d --- /dev/null +++ b/src/spectator/context_method.cr @@ -0,0 +1,10 @@ +require "./context" + +module Spectator + # Encapsulates a method in a test context. + # This could be used to invoke a test case or hook method. + # The context is passed as an argument. + # The proc should downcast the context instance to the desired type + # and call a method on that context. + alias ContextMethod = Context -> +end diff --git a/src/spectator/dsl.cr b/src/spectator/dsl.cr index 0d13112..51d6f38 100644 --- a/src/spectator/dsl.cr +++ b/src/spectator/dsl.cr @@ -2,6 +2,13 @@ require "./dsl/*" module Spectator # Namespace containing methods representing the spec domain specific language. + # + # Note: Documentation inside macros is kept to a minimuum to reduce generated code. + # This also helps keep error traces small. + # Documentation only useful for debugging is included in generated code. module DSL + # Keywords that cannot be used in specs using the DSL. + # These are either problematic or reserved for internal use. + RESERVED_KEYWORDS = %i[initialize] end end diff --git a/src/spectator/dsl/assertions.cr b/src/spectator/dsl/assertions.cr deleted file mode 100644 index 038e45a..0000000 --- a/src/spectator/dsl/assertions.cr +++ /dev/null @@ -1,216 +0,0 @@ -require "../expectations/expectation_partial" -require "../source" -require "../test_block" -require "../test_value" - -module Spectator - module DSL - # Starts an expectation. - # This should be followed up with `Spectator::Expectations::ExpectationPartial#to` - # or `Spectator::Expectations::ExpectationPartial#to_not`. - # The value passed in will be checked - # to see if it satisfies the conditions specified. - # - # This method should be used like so: - # ``` - # expect(actual).to eq(expected) - # ``` - # Where the actual value is returned by the system-under-test, - # and the expected value is what the actual value should be to satisfy the condition. - macro expect(actual, _source_file = __FILE__, _source_line = __LINE__) - %test_value = ::Spectator::TestValue.new({{actual}}, {{actual.stringify}}) - %source = ::Spectator::Source.new({{_source_file}}, {{_source_line}}) - ::Spectator::Expectations::ExpectationPartial.new(%test_value, %source) - end - - # Starts an expectation on a block of code. - # This should be followed up with `Spectator::Expectations::ExpectationPartial#to` - # or `Spectator::Expectations::ExpectationPartial#to_not`. - # The block passed in, or its return value, will be checked - # to see if it satisfies the conditions specified. - # - # This method should be used like so: - # ``` - # expect { raise "foo" }.to raise_error - # ``` - # The block of code is passed along for validation to the matchers. - # - # The short, one argument syntax used for passing methods to blocks can be used. - # So instead of doing this: - # ``` - # expect(subject.size).to eq(5) - # ``` - # The following syntax can be used instead: - # ``` - # expect(&.size).to eq(5) - # ``` - # The method passed will always be evaluated on the subject. - macro expect(&block) - {% if block.is_a?(Nop) %} - {% raise "Argument or block must be provided to expect" %} - {% end %} - - # Check if the short-hand method syntax is used. - # This is a hack, since macros don't get this as a "literal" or something similar. - # The Crystal compiler will translate: - # ``` - # &.foo - # ``` - # to: - # ``` - # { |__arg0| __arg0.foo } - # ``` - # The hack used here is to check if it looks like a compiler-generated block. - {% if block.args.size == 1 && block.args[0] =~ /^__arg\d+$/ && block.body.is_a?(Call) && block.body.id =~ /^__arg\d+\./ %} - # Extract the method name to make it clear to the user what is tested. - # The raw block can't be used because it's not clear to the user. - {% method_name = block.body.id.split('.')[1..-1].join('.') %} - %proc = ->{ subject.{{method_name.id}} } - %test_block = ::Spectator::TestBlock.create(%proc, {{"#" + method_name}}) - {% elsif block.args.empty? %} - # In this case, it looks like the short-hand method syntax wasn't used. - # Capture the block as a proc and pass along. - %proc = ->{{block}} - %test_block = ::Spectator::TestBlock.create(%proc, {{"`" + block.body.stringify + "`"}}) - {% else %} - {% raise "Unexpected block arguments in expect call" %} - {% end %} - - %source = ::Spectator::Source.new({{block.filename}}, {{block.line_number}}) - ::Spectator::Expectations::ExpectationPartial.new(%test_block, %source) - end - - # Starts an expectation. - # This should be followed up with `Spectator::Expectations::ExpectationPartial#to` - # or `Spectator::Expectations::ExpectationPartial#to_not`. - # The value passed in will be checked - # to see if it satisfies the conditions specified. - # - # This method is identical to `#expect`, - # but is grammatically correct for the one-liner syntax. - # It can be used like so: - # ``` - # it expects(actual).to eq(expected) - # ``` - # Where the actual value is returned by the system-under-test, - # and the expected value is what the actual value should be to satisfy the condition. - macro expects(actual) - expect({{actual}}) - end - - # Starts an expectation on a block of code. - # This should be followed up with `Spectator::Expectations::ExpectationPartial#to` - # or `Spectator::Expectations::ExpectationPartial#to_not`. - # The block passed in, or its return value, will be checked - # to see if it satisfies the conditions specified. - # - # This method is identical to `#expect`, - # but is grammatically correct for the one-liner syntax. - # It can be used like so: - # ``` - # it expects { 5 / 0 }.to raise_error - # ``` - # The block of code is passed along for validation to the matchers. - # - # The short, one argument syntax used for passing methods to blocks can be used. - # So instead of doing this: - # ``` - # it expects(subject.size).to eq(5) - # ``` - # The following syntax can be used instead: - # ``` - # it expects(&.size).to eq(5) - # ``` - # The method passed will always be evaluated on the subject. - macro expects(&block) - expect {{block}} - end - - # Short-hand for expecting something of the subject. - # These two are functionally equivalent: - # ``` - # expect(subject).to eq("foo") - # is_expected.to eq("foo") - # ``` - macro is_expected - expect(subject) - end - - # Short-hand form of `#is_expected` that can be used for one-liner syntax. - # For instance: - # ``` - # it "is 42" do - # expect(subject).to eq(42) - # end - # ``` - # Can be shortened to: - # ``` - # it is(42) - # ``` - # - # These three are functionally equivalent: - # ``` - # expect(subject).to eq("foo") - # is_expected.to eq("foo") - # is("foo") - # ``` - # - # See also: `#is_not` - macro is(expected) - is_expected.to eq({{expected}}) - end - - # Short-hand, negated form of `#is_expected` that can be used for one-liner syntax. - # For instance: - # ``` - # it "is not 42" do - # expect(subject).to_not eq(42) - # end - # ``` - # Can be shortened to: - # ``` - # it is_not(42) - # ``` - # - # These three are functionally equivalent: - # ``` - # expect(subject).to_not eq("foo") - # is_expected.to_not eq("foo") - # is_not("foo") - # ``` - # - # See also: `#is` - macro is_not(expected) - is_expected.to_not eq({{expected}}) - end - - macro should(matcher) - is_expected.to({{matcher}}) - end - - macro should_not(matcher) - is_expected.to_not({{matcher}}) - end - - macro should_eventually(matcher) - is_expected.to_eventually({{matcher}}) - end - - macro should_never(matcher) - is_expected.to_never({{matcher}}) - end - - # Immediately fail the current test. - # A reason can be passed, - # which is reported in the output. - def fail(reason : String) - raise ExampleFailed.new(reason) - end - - # :ditto: - @[AlwaysInline] - def fail - fail("Example failed") - end - end -end diff --git a/src/spectator/dsl/builder.cr b/src/spectator/dsl/builder.cr new file mode 100644 index 0000000..4054906 --- /dev/null +++ b/src/spectator/dsl/builder.cr @@ -0,0 +1,108 @@ +require "../example_group_hook" +require "../example_hook" +require "../example_procsy_hook" +require "../spec_builder" + +module Spectator::DSL + # Incrementally builds up a test spec from the DSL. + # This is intended to be used only by the Spectator DSL. + module Builder + extend self + + # Underlying spec builder. + private class_getter(builder) { SpecBuilder.new(Spectator.config) } + + # Defines a new example group and pushes it onto the group stack. + # Examples and groups defined after calling this method will be nested under the new group. + # The group will be finished and popped off the stack when `#end_example` is called. + # + # See `Spec::Builder#start_group` for usage details. + def start_group(*args) + builder.start_group(*args) + end + + # Defines a new iterative example group and pushes it onto the group stack. + # Examples and groups defined after calling this method will be nested under the new group. + # The group will be finished and popped off the stack when `#end_example` is called. + # + # See `Spec::Builder#start_iterative_group` for usage details. + def start_iterative_group(*args) + builder.start_iterative_group(*args) + end + + # Completes a previously defined example group and pops it off the group stack. + # Be sure to call `#start_group` and `#end_group` symmetically. + # + # See `Spec::Builder#end_group` for usage details. + def end_group(*args) + builder.end_group(*args) + end + + # Defines a new example. + # The example is added to the group currently on the top of the stack. + # + # See `Spec::Builder#add_example` for usage details. + def add_example(*args, &block : Example ->) + builder.add_example(*args, &block) + end + + # Defines a new pending example. + # The example is added to the group currently on the top of the stack. + # + # See `Spec::Builder#add_pending_example` for usage details. + def add_pending_example(*args) + builder.add_pending_example(*args) + end + + # Defines a block of code to execute before any and all examples in the test suite. + def before_suite(location = nil, label = "before_suite", &block) + hook = ExampleGroupHook.new(location: location, label: label, &block) + builder.before_suite(hook) + end + + # Defines a block of code to execute before any and all examples in the current group. + def before_all(location = nil, label = "before_all", &block) + hook = ExampleGroupHook.new(location: location, label: label, &block) + builder.before_all(hook) + end + + # Defines a block of code to execute before every example in the current group + def before_each(location = nil, label = "before_each", &block : Example -> _) + hook = ExampleHook.new(location: location, label: label, &block) + builder.before_each(hook) + end + + # Defines a block of code to execute after any and all examples in the test suite. + def after_suite(location = nil, label = "after_suite", &block) + hook = ExampleGroupHook.new(location: location, label: label, &block) + builder.after_suite(hook) + end + + # Defines a block of code to execute after any and all examples in the current group. + def after_all(location = nil, label = "after_all", &block) + hook = ExampleGroupHook.new(location: location, label: label, &block) + builder.after_all(hook) + end + + # Defines a block of code to execute after every example in the current group. + def after_each(location = nil, label = "after_each", &block : Example ->) + hook = ExampleHook.new(location: location, label: label, &block) + builder.after_each(hook) + end + + # Defines a block of code to execute around every example in the current group. + def around_each(location = nil, label = "around_each", &block : Example::Procsy ->) + hook = ExampleProcsyHook.new(location: location, label: label, &block) + builder.around_each(hook) + end + + # Constructs the test spec. + # Returns the spec instance. + # + # Raises an error if there were not symmetrical calls to `#start_group` and `#end_group`. + # This would indicate a logical error somewhere in Spectator or an extension of it. + def build : Spec + builder.build + end + end +end diff --git a/src/spectator/dsl/concise.cr b/src/spectator/dsl/concise.cr new file mode 100644 index 0000000..a7222ae --- /dev/null +++ b/src/spectator/dsl/concise.cr @@ -0,0 +1,47 @@ +require "./examples" +require "./groups" +require "./memoize" + +module Spectator::DSL + # DSL methods and macros for shorter syntax. + module Concise + # Defines an example and input values in a shorter syntax. + # The only arguments given to this macro are one or more assignments. + # The names in the assigments will be available in the example code. + # + # If the code block is omitted, then the example is skipped (marked as not implemented). + # + # Tags and metadata cannot be used with this macro. + # + # ``` + # given x = 42 do + # expect(x).to eq(42) + # end + # ``` + macro provided(*assignments, **kwargs, &block) + {% raise "Cannot use 'provided' inside of a test block" if @def %} + + class Given%given < {{@type.id}} + {% for assignment in assignments %} + let({{assignment.target}}) { {{assignment.value}} } + {% end %} + {% for name, value in kwargs %} + let({{name}}) { {{value}} } + {% end %} + + {% if block %} + example {{block}} + {% else %} + example {{assignments.splat.stringify}} + {% end %} + end + end + + # :ditto: + @[Deprecated("Use `provided` instead.")] + macro given(*assignments, **kwargs, &block) + {% raise "Cannot use 'given' inside of a test block" if @def %} + provided({{assignments.splat(",")}} {{kwargs.double_splat}}) {{block}} + end + end +end diff --git a/src/spectator/dsl/examples.cr b/src/spectator/dsl/examples.cr index 06232e1..03484ae 100644 --- a/src/spectator/dsl/examples.cr +++ b/src/spectator/dsl/examples.cr @@ -1,72 +1,151 @@ -require "../source" -require "../spec_builder" +require "../context" +require "../location" +require "./builder" +require "./metadata" -module Spectator - module DSL - macro it(description = nil, &block) - {% if block.is_a?(Nop) %} - {% if description.is_a?(Call) %} - def %run - {{description}} +module Spectator::DSL + # DSL methods for defining examples and test code. + module Examples + include Metadata + + # Defines a macro to generate code for an example. + # The *name* is the name given to the macro. + # + # In addition, another macro is defined that marks the example as pending. + # The pending macro is prefixed with 'x'. + # For instance, `define_example :it` defines `it` and `xit`. + # + # Default tags can be provided with *tags* and *metadata*. + # The tags are merged with parent groups. + # Any items with falsey values from *metadata* remove the corresponding tag. + macro define_example(name, *tags, **metadata) + # Defines an example. + # + # If a block is given, it is treated as the code to test. + # The block is provided the current example instance as an argument. + # + # The first argument names the example (test). + # Typically, this specifies what is being tested. + # It has no effect on the test and is purely used for output. + # If omitted, a name is generated from the first assertion in the test. + # + # The example will be marked as pending if the block is omitted. + # A block or name must be provided. + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # Any falsey items will remove a previously defined tag. + macro {{name.id}}(what = nil, *tags, **metadata, &block) + \{% raise "Cannot use '{{name.id}}' inside of a test block" if @def %} + \{% raise "A description or block must be provided. Cannot use '{{name.id}}' alone." unless what || block %} + + _spectator_metadata(%metadata, :metadata, {{tags.splat(",")}} {{metadata.double_splat}}) + _spectator_metadata(\%metadata, %metadata, \{{tags.splat(",")}} \{{metadata.double_splat}}) + + \{% if block %} + \{% raise "Block argument count '{{name.id}}' must be 0..1" if block.args.size > 1 %} + + private def \%test(\{{block.args.splat}}) : Nil + \{{block.body}} end - {% else %} - {% raise "Unrecognized syntax: `it #{description}` at #{_source_file}:#{_source_line}" %} - {% end %} - {% else %} - def %run - {{block.body}} - end - {% end %} - {% if block.is_a?(Nop) %} - %source = ::Spectator::Source.new({{description.filename}}, line: {{description.line_number}}, end_line: {{description.end_line_number}}) - {% else %} - %source = ::Spectator::Source.new({{block.filename}}, line: {{block.line_number}}, end_line: {{block.end_line_number}}) - {% end %} - ::Spectator::SpecBuilder.add_example( - {{description.is_a?(StringLiteral) || description.is_a?(StringInterpolation) || description.is_a?(NilLiteral) ? description : description.stringify}}, - %source, - {{@type.name}} - ) { |test| test.as({{@type.name}}).%run } - end - - macro specify(description = nil, &block) - it({{description}}) {{block}} - end - - macro pending(description = nil, &block) - {% if block.is_a?(Nop) %} - {% if description.is_a?(Call) %} - def %run - {{description}} + ::Spectator::DSL::Builder.add_example( + _spectator_example_name(\{{what}}), + ::Spectator::Location.new(\{{block.filename}}, \{{block.line_number}}, \{{block.end_line_number}}), + -> { new.as(::Spectator::Context) }, + \%metadata + ) do |example| + example.with_context(\{{@type.name}}) do + \{% if block.args.empty? %} + \%test + \{% else %} + \%test(example) + \{% end %} + end end - {% else %} - {% raise "Unrecognized syntax: `pending #{description}` at #{_source_file}:#{_source_line}" %} - {% end %} + + \{% else %} + ::Spectator::DSL::Builder.add_pending_example( + _spectator_example_name(\{{what}}), + ::Spectator::Location.new(\{{what.filename}}, \{{what.line_number}}), + \%metadata, + "Not yet implemented" + ) + \{% end %} + end + + define_pending_example :x{{name.id}}, skip: "Temporarily skipped with x{{name.id}}" + end + + # Defines a macro to generate code for a pending example. + # The *name* is the name given to the macro. + # + # The block for the example's content is discarded at compilation time. + # This prevents issues with undefined methods, signature differences, etc. + # + # Default tags can be provided with *tags* and *metadata*. + # The tags are merged with parent groups. + # Any items with falsey values from *metadata* remove the corresponding tag. + macro define_pending_example(name, *tags, **metadata) + # Defines a pending example. + # + # If a block is given, it is treated as the code to test. + # The block is provided the current example instance as an argument. + # + # The first argument names the example (test). + # Typically, this specifies what is being tested. + # It has no effect on the test and is purely used for output. + # If omitted, a name is generated from the first assertion in the test. + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # Any falsey items will remove a previously defined tag. + macro {{name.id}}(what = nil, *tags, **metadata, &block) + \{% raise "Cannot use '{{name.id}}' inside of a test block" if @def %} + \{% raise "A description or block must be provided. Cannot use '{{name.id}}' alone." unless what || block %} + \{% raise "Block argument count '{{name.id}}' must be 0..1" if block && block.args.size > 1 %} + + _spectator_metadata(%metadata, :metadata, {{tags.splat(",")}} {{metadata.double_splat}}) + _spectator_metadata(\%metadata, %metadata, \{{tags.splat(",")}} \{{metadata.double_splat}}) + + ::Spectator::DSL::Builder.add_pending_example( + _spectator_example_name(\{{what}}), + ::Spectator::Location.new(\{{(what || block).filename}}, \{{(what || block).line_number}}, \{{(what || block).end_line_number}}), + \%metadata, + \{% if !block %}"Not yet implemented"\{% end %} + ) + end + end + + # Inserts the correct representation of a example's name. + # If *what* is a string, then it is dropped in as-is. + # For anything else, it is stringified. + # This is intended to be used to convert a description from the spec DSL to `Node#name`. + private macro _spectator_example_name(what) + {% if what.is_a?(StringLiteral) || + what.is_a?(StringInterpolation) || + what.is_a?(NilLiteral) %} + {{what}} {% else %} - def %run - {{block.body}} - end + {{what.stringify}} {% end %} - - {% if block.is_a?(Nop) %} - %source = ::Spectator::Source.new({{description.filename}}, line: {{description.line_number}}, end_line: {{description.end_line_number}}) - {% else %} - %source = ::Spectator::Source.new({{block.filename}}, line: {{block.line_number}}, end_line: {{block.end_line_number}}) - {% end %} - ::Spectator::SpecBuilder.add_pending_example( - {{description.is_a?(StringLiteral) || description.is_a?(StringInterpolation) || description.is_a?(NilLiteral) ? description : description.stringify}}, - %source, - {{@type.name}} - ) { |test| test.as({{@type.name}}).%run } end - macro skip(description = nil, &block) - pending({{description}}) {{block}} - end + define_example :example - macro xit(description = nil, &block) - pending({{description}}) {{block}} - end + define_example :it + + define_example :specify + + define_example :fexample, focus: true + + define_example :fit, focus: true + + define_example :fspecify, focus: true + + @[Deprecated("Behavior of pending blocks will change in Spectator v0.11.0. Use `skip` instead.")] + define_pending_example :pending + + define_pending_example :skip end end diff --git a/src/spectator/dsl/expectations.cr b/src/spectator/dsl/expectations.cr new file mode 100644 index 0000000..0878dda --- /dev/null +++ b/src/spectator/dsl/expectations.cr @@ -0,0 +1,181 @@ +require "../block" +require "../example_pending" +require "../expectation" +require "../expectation_failed" +require "../location" +require "../pending_result" +require "../value" + +module Spectator::DSL + # Methods and macros for asserting that conditions are met. + module Expectations + # Immediately fail the current test. + # A reason can be specified with *message*. + def fail(message = "Example failed", *, _file = __FILE__, _line = __LINE__) + raise ExampleFailed.new(Location.new(_file, _line), message) + end + + # Mark the current test as pending and immediately abort. + # A reason can be specified with *message*. + def pending(message = PendingResult::DEFAULT_REASON, *, _file = __FILE__, _line = __LINE__) + raise ExamplePending.new(Location.new(_file, _line), message) + end + + # Mark the current test as skipped and immediately abort. + # A reason can be specified with *message*. + def skip(message = PendingResult::DEFAULT_REASON, *, _file = __FILE__, _line = __LINE__) + raise ExamplePending.new(Location.new(_file, _line), message) + end + + # Starts an expectation. + # This should be followed up with `Assertion::Target#to` or `Assertion::Target#to_not`. + # The value passed in will be checked to see if it satisfies the conditions of the specified matcher. + # + # This macro should be used like so: + # ``` + # expect(actual).to eq(expected) + # ``` + # + # Where the actual value is returned by the system under test, + # and the expected value is what the actual value should be to satisfy the condition. + macro expect(actual) + %actual = begin + {{actual}} + end + + %expression = ::Spectator::Value.new(%actual, {{actual.stringify}}) + %location = ::Spectator::Location.new({{actual.filename}}, {{actual.line_number}}) + ::Spectator::Expectation::Target.new(%expression, %location) + end + + # Starts an expectation. + # This should be followed up with `Assertion::Target#to` or `Assertion::Target#not_to`. + # The value passed in will be checked to see if it satisfies the conditions of the specified matcher. + # + # This macro should be used like so: + # ``` + # expect { raise "foo" }.to raise_error + # ``` + # + # The block of code is passed along for validation to the matchers. + # + # The short, one argument syntax used for passing methods to blocks can be used. + # So instead of doing this: + # ``` + # expect(subject.size).to eq(5) + # ``` + # + # The following syntax can be used instead: + # ``` + # expect(&.size).to eq(5) + # ``` + # + # The method passed will always be evaluated on the subject. + # + # TECHNICAL NOTE: + # This macro uses an ugly hack to detect the short-hand syntax. + # + # The Crystal compiler will translate: + # ``` + # &.foo + # ``` + # + # effectively to: + # ``` + # { |__arg0| __arg0.foo } + # ``` + macro expect(&block) + {% if block.args.size == 1 && block.args[0] =~ /^__arg\d+$/ && block.body.is_a?(Call) && block.body.id =~ /^__arg\d+\./ %} + {% method_name = block.body.id.split('.')[1..-1].join('.') %} + %block = ::Spectator::Block.new({{"#" + method_name}}) do + subject.{{method_name.id}} + end + {% elsif block.args.empty? %} + %block = ::Spectator::Block.new({{"`" + block.body.stringify + "`"}}) {{block}} + {% else %} + {% raise "Unexpected block arguments in 'expect' call" %} + {% end %} + + %location = ::Spectator::Location.new({{block.filename}}, {{block.line_number}}) + ::Spectator::Expectation::Target.new(%block, %location) + end + + # Short-hand for expecting something of the subject. + # + # These two are functionally equivalent: + # ``` + # expect(subject).to eq("foo") + # is_expected.to eq("foo") + # ``` + macro is_expected + expect(subject) + end + + # Short-hand form of `#is_expected` that can be used for one-liner syntax. + # + # For instance: + # ``` + # it "is 42" do + # expect(subject).to eq(42) + # end + # ``` + # + # Can be shortened to: + # ``` + # it { is(42) } + # ``` + # + # These three are functionally equivalent: + # ``` + # expect(subject).to eq("foo") + # is_expected.to eq("foo") + # is("foo") + # ``` + # + # See also: `#is_not` + macro is(expected) + expect(subject).to(eq({{expected}})) + end + + # Short-hand, negated form of `#is_expected` that can be used for one-liner syntax. + # + # For instance: + # ``` + # it "is not 42" do + # expect(subject).to_not eq(42) + # end + # ``` + # + # Can be shortened to: + # ``` + # it { is_not(42) } + # ``` + # + # These three are functionally equivalent: + # ``` + # expect(subject).not_to eq("foo") + # is_expected.not_to eq("foo") + # is_not("foo") + # ``` + # + # See also: `#is` + macro is_not(expected) + expect(subject).not_to(eq({{expected}})) + end + + # Captures multiple possible failures. + # Aborts after the block completes if there were any failed expectations in the block. + # + # ``` + # aggregate_failures do + # expect(true).to be_false + # expect(false).to be_true + # end + # ``` + def aggregate_failures(label = nil) + ::Spectator::Harness.current.aggregate_failures(label) do + yield + end + end + end +end diff --git a/src/spectator/dsl/groups.cr b/src/spectator/dsl/groups.cr index 492cbfb..8f47b27 100644 --- a/src/spectator/dsl/groups.cr +++ b/src/spectator/dsl/groups.cr @@ -1,154 +1,235 @@ -require "../spec_builder" +require "../location" +require "./builder" +require "./memoize" +require "./metadata" -module Spectator - module DSL - macro context(what, &block) - class Context%context < {{@type.id}} - {% - description = if what.is_a?(StringLiteral) || what.is_a?(StringInterpolation) - if what.starts_with?("#") || what.starts_with?(".") - what.id.symbolize - else - what - end - else - what.symbolize - end - %} +module Spectator::DSL + # DSL methods and macros for creating example groups. + # This module should be included as a mix-in. + module Groups + include Metadata - %source = ::Spectator::Source.new({{block.filename}}, {{block.line_number}}) - ::Spectator::SpecBuilder.start_group({{description}}, %source) + # Defines a macro to generate code for an example group. + # The *name* is the name given to the macro. + # + # Default tags can be provided with *tags* and *metadata*. + # The tags are merged with parent groups. + # Any items with falsey values from *metadata* remove the corresponding tag. + macro define_example_group(name, *tags, **metadata) + # Defines a new example group. + # The *what* argument is a name or description of the group. + # + # The first argument names the example (test). + # Typically, this specifies what is being tested. + # This argument is also used as the subject. + # When it is a type name, it becomes an explicit, which overrides any previous subjects. + # Otherwise it becomes an implicit subject, which doesn't override explicitly defined subjects. + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # Any falsey items will remove a previously defined tag. + # + # TODO: Handle string interpolation in example and group names. + macro {{name.id}}(what, *tags, **metadata, &block) + \{% raise "Cannot use '{{name.id}}' inside of a test block" if @def %} - # Oddly, `#resolve?` can return a constant's value, which isn't a TypeNode. - # Ensure `described_class` and `subject` are only set for real types (is a `TypeNode`). - {% if (what.is_a?(Path) || what.is_a?(Generic)) && (described_type = what.resolve?).is_a?(TypeNode) %} - macro described_class - {{what}} - end + class Group\%group < \{{@type.id}} + _spectator_group_subject(\{{what}}) - subject do - {% if described_type < Reference || described_type < Value %} - described_class.new - {% else %} - described_class - {% end %} - end - {% else %} - def _spectator_implicit_subject(*args) - {{what}} - end - {% end %} + _spectator_metadata(:metadata, :super, {{tags.splat(", ")}} {{metadata.double_splat}}) + _spectator_metadata(:metadata, :previous_def, \{{tags.splat(", ")}} \{{metadata.double_splat}}) - {{block.body}} + ::Spectator::DSL::Builder.start_group( + _spectator_group_name(\{{what}}), + ::Spectator::Location.new(\{{block.filename}}, \{{block.line_number}}, \{{block.end_line_number}}), + metadata + ) - ::Spectator::SpecBuilder.end_group + \{{block.body if block}} + + ::Spectator::DSL::Builder.end_group + end end end - macro describe(what, &block) - context({{what}}) {{block}} + # Defines a macro to generate code for an iterative example group. + # The *name* is the name given to the macro. + # + # Default tags can be provided with *tags* and *metadata*. + # The tags are merged with parent groups. + # Any items with falsey values from *metadata* remove the corresponding tag. + # + # If provided, a block can be used to modify collection that will be iterated. + # It takes a single argument - the original collection from the user. + # The modified collection should be returned. + # + # TODO: Handle string interpolation in example and group names. + macro define_iterative_group(name, *tags, **metadata, &block) + macro {{name.id}}(collection, *tags, count = nil, **metadata, &block) + \{% raise "Cannot use 'sample' inside of a test block" if @def %} + + class Group\%group < \{{@type.id}} + _spectator_metadata(:metadata, :super, {{tags.splat(", ")}} {{metadata.double_splat}}) + _spectator_metadata(:metadata, :previous_def, \{{tags.splat(", ")}} \{{metadata.double_splat}}) + + def self.\%collection + \{{collection}} + end + + {% if block %} + def self.%mutate({{block.args.splat}}) + {{block.body}} + end + + def self.\%collection + %mutate(previous_def) + end + {% end %} + + \{% if count %} + def self.\%collection + previous_def.first(\{{count}}) + end + \{% end %} + + ::Spectator::DSL::Builder.start_iterative_group( + \%collection, + \{{collection.stringify}}, + \{{block.args.empty? ? :nil.id : block.args.first.stringify}}, + ::Spectator::Location.new(\{{block.filename}}, \{{block.line_number}}, \{{block.end_line_number}}), + metadata + ) + + \{% if block %} + \{% if block.args.size == 1 %} + let(\{{block.args.first}}) do |example| + example.group.as(::Spectator::ExampleGroupIteration(typeof(Group\%group.\%collection.first))).item + end + \{% elsif block.args.size > 1 %} + \{% raise "Expected 1 argument for 'sample' block, but got #{block.args.size}" %} + \{% end %} + + \{{block.body}} + \{% end %} + + ::Spectator::DSL::Builder.end_group + end + end end - macro sample(collection, count = nil, &block) - {% name = block.args.empty? ? :value.id : block.args.first.id %} + # Inserts the correct representation of a group's name. + # If *what* appears to be a type name, it will be symbolized. + # If it's a string, then it is dropped in as-is. + # For anything else, it is stringified. + # This is intended to be used to convert a description from the spec DSL to `Node#name`. + private macro _spectator_group_name(what) + {% if (what.is_a?(Generic) || + what.is_a?(Path) || + what.is_a?(TypeNode) || + what.is_a?(Union)) && + what.resolve?.is_a?(TypeNode) %} + {{what.symbolize}} + {% elsif what.is_a?(StringLiteral) || + what.is_a?(StringInterpolation) || + what.is_a?(NilLiteral) %} + {{what}} + {% else %} + {{what.stringify}} + {% end %} + end - def %collection - {{collection}} - end - - def %to_a - {% if count %} - %collection.first({{count}}) - {% else %} - %collection.to_a - {% end %} - end - - class Context%sample < {{@type.id}} - %source = ::Spectator::Source.new({{block.filename}}, {{block.line_number}}) - ::Spectator::SpecBuilder.start_sample_group({{collection.stringify}}, %source, :%sample, {{name.stringify}}) do |values| - sample = {{@type.id}}.new(values) - sample.%to_a + # Defines the implicit subject for the test context. + # If *what* is a type, then the `described_class` method will be defined. + # Additionally, the implicit subject is set to an instance of *what* if it's not a module. + # + # There is no common macro type that has the `#resolve?` method. + # Also, `#responds_to?` can't be used in macros. + # So the large if statement in this macro is used to look for type signatures. + private macro _spectator_group_subject(what) + {% if (what.is_a?(Generic) || + what.is_a?(Path) || + what.is_a?(TypeNode) || + what.is_a?(Union)) && + (described_type = what.resolve?).is_a?(TypeNode) %} + private macro described_class + {{what}} end - def {{name}} - @spectator_test_values.get_value(:%sample, typeof(%to_a.first)) - end - - {{block.body}} - - ::Spectator::SpecBuilder.end_group - end - end - - macro random_sample(collection, count = nil, &block) - {% name = block.args.empty? ? :value.id : block.args.first.id %} - - def %collection - {{collection}} - end - - def %to_a - {% if count %} - %collection.first({{count}}) - {% else %} - %collection.to_a - {% end %} - end - - class Context%sample < {{@type.id}} - %source = ::Spectator::Source.new({{block.filename}}, {{block.line_number}}) - ::Spectator::SpecBuilder.start_sample_group({{collection.stringify}}, %source, :%sample, {{name.stringify}}) do |values| - sample = {{@type.id}}.new(values) - collection = sample.%to_a - {% if count %} - collection.sample({{count}}, ::Spectator.random) + subject do + {% if described_type.class? || described_type.struct? %} + described_class.new {% else %} - collection.shuffle(::Spectator.random) + described_class {% end %} end - - def {{name}} - @spectator_test_values.get_value(:%sample, typeof(%to_a.first)) + {% else %} + private def _spectator_implicit_subject + {{what}} end - - {{block.body}} - - ::Spectator::SpecBuilder.end_group - end + {% end %} end - macro given(*assignments, &block) - context({{assignments.splat.stringify}}) do - {% for assignment in assignments %} - let({{assignment.target}}) { {{assignment.value}} } - {% end %} + define_example_group :example_group - {% # Trick to get the contents of the block as an array of nodes. -# If there are multiple expressions/statements in the block, -# then the body will be a `Expressions` type. -# If there's only one expression, then the body is just that. - body = if block.is_a?(Nop) - raise "Missing block for 'given'" - elsif block.body.is_a?(Expressions) - # Get the expressions, which is already an array. - block.body.expressions - else - # Wrap the expression in an array. - [block.body] - end %} + define_example_group :describe - {% for item in body %} - # If the item starts with "it", then leave it as-is. - # Otherwise, prefix it with "it" - # and treat it as the one-liner "it" syntax. - {% if item.is_a?(Call) && item.name == :it.id %} - {{item}} - {% else %} - it {{item}} - {% end %} - {% end %} - end + define_example_group :context + + define_example_group :xexample_group, skip: "Temporarily skipped with xexample_group" + + define_example_group :xdescribe, skip: "Temporarily skipped with xdescribe" + + define_example_group :xcontext, skip: "Temporarily skipped with xcontext" + + define_example_group :fexample_group, focus: true + + define_example_group :fdescribe, focus: true + + define_example_group :fcontext, focus: true + + # Defines a new iterative example group. + # This type of group duplicates its contents for each element in *collection*. + # + # The first argument is the collection of elements to iterate over. + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # Any falsey items will remove a previously defined tag. + # + # The number of items iterated can be restricted by specifying a *count* argument. + # The first *count* items will be used if specified, otherwise all items will be used. + define_iterative_group :sample + + # :ditto: + define_iterative_group :xsample, skip: "Temporarily skipped with xsample" + + define_iterative_group :fsample, focus: true + + # Defines a new iterative example group. + # This type of group duplicates its contents for each element in *collection*. + # This is the same as `#sample` except that the items are shuffled. + # The items are selected with a RNG based on the seed. + # + # The first argument is the collection of elements to iterate over. + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # Any falsey items will remove a previously defined tag. + # + # The number of items iterated can be restricted by specifying a *count* argument. + # The first *count* items will be used if specified, otherwise all items will be used. + define_iterative_group :random_sample do |collection| + collection.to_a.shuffle(::Spectator.random) + end + + # :ditto: + define_iterative_group :xrandom_sample, skip: "Temporarily skipped with xrandom_sample" do |collection| + collection.to_a.shuffle(::Spectator.random) + end + + # :ditto: + define_iterative_group :frandom_sample, focus: true do |collection| + collection.to_a.shuffle(::Spectator.random) end end end diff --git a/src/spectator/dsl/hooks.cr b/src/spectator/dsl/hooks.cr index bd65db3..928b3ee 100644 --- a/src/spectator/dsl/hooks.cr +++ b/src/spectator/dsl/hooks.cr @@ -1,79 +1,142 @@ -module Spectator - module DSL - macro before_each(&block) - def %hook({{block.args.splat}}) : Nil - {{block.body}} - end +require "../location" +require "./builder" - ::Spectator::SpecBuilder.add_before_each_hook do |test, example| - cast_test = test.as({{@type.id}}) - {% if block.args.empty? %} - cast_test.%hook - {% else %} - cast_test.%hook(example) +module Spectator::DSL + # DSL methods for adding custom logic to key times of the spec execution. + module Hooks + # Defines a macro to create an example group hook. + # The *type* indicates when the hook runs and must be a method on `Spectator::DSL::Builder`. + # A custom *name* can be used for the hook method. + # If not provided, *type* will be used instead. + # Additionally, a block can be provided. + # The block can perform any operations necessary and yield to invoke the end-user hook. + macro define_example_group_hook(type, name = nil, &block) + macro {{(name ||= type).id}}(&block) + \{% raise "Missing block for '{{name.id}}' hook" unless block %} + \{% raise "Cannot use '{{name.id}}' inside of a test block" if @def %} + + private def self.\%hook : Nil + \{{block.body}} + end + + {% if block %} + private def self.%wrapper : Nil + {{block.body}} + end {% end %} + + ::Spectator::DSL::Builder.{{type.id}}( + ::Spectator::Location.new(\{{block.filename}}, \{{block.line_number}}, \{{block.end_line_number}}) + ) do + {% if block %} + %wrapper do |*args| + \{% if block.args.empty? %} + \%hook + \{% else %} + \%hook(*args) + \{% end %} + end + {% else %} + \%hook + {% end %} + end end end - macro after_each(&block) - def %hook({{block.args.splat}}) : Nil - {{block.body}} - end + # Defines a macro to create an example hook. + # The *type* indicates when the hook runs and must be a method on `Spectator::DSL::Builder`. + # A custom *name* can be used for the hook method. + # If not provided, *type* will be used instead. + # Additionally, a block can be provided that takes the current example as an argument. + # The block can perform any operations necessary and yield to invoke the end-user hook. + macro define_example_hook(type, name = nil, &block) + macro {{(name ||= type).id}}(&block) + \{% raise "Missing block for '{{name.id}}' hook" unless block %} + \{% raise "Block argument count '{{name.id}}' hook must be 0..1" if block.args.size > 1 %} + \{% raise "Cannot use '{{name.id}}' inside of a test block" if @def %} - ::Spectator::SpecBuilder.add_after_each_hook do |test, example| - cast_test = test.as({{@type.id}}) - {% if block.args.empty? %} - cast_test.%hook - {% else %} - cast_test.%hook(example) + private def \%hook(\{{block.args.splat}}) : Nil + \{{block.body}} + end + + {% if block %} + private def %wrapper({{block.args.splat}}) : Nil + {{block.body}} + end {% end %} + + ::Spectator::DSL::Builder.{{type.id}}( + ::Spectator::Location.new(\{{block.filename}}, \{{block.line_number}}) + ) do |example| + example.with_context(\{{@type.name}}) do + {% if block %} + {% if block.args.empty? %} + %wrapper do |*args| + \{% if block.args.empty? %} + \%hook + \{% else %} + \%hook(*args) + \{% end %} + end + {% else %} + %wrapper(example) do |*args| + \{% if block.args.empty? %} + \%hook + \{% else %} + \%hook(*args) + \{% end %} + end + {% end %} + {% else %} + \{% if block.args.empty? %} + \%hook + \{% else %} + \%hook(example) + \{% end %} + {% end %} + end + end end end - macro before_all(&block) - ::Spectator::SpecBuilder.add_before_all_hook {{block}} - end + # Defines a block of code that will be invoked once before any examples in the suite. + # The block will not run in the context of the current running example. + # This means that values defined by `let` and `subject` are not available. + define_example_group_hook :before_suite - macro after_all(&block) - ::Spectator::SpecBuilder.add_after_all_hook {{block}} - end + # Defines a block of code that will be invoked once after all examples in the suite. + # The block will not run in the context of the current running example. + # This means that values defined by `let` and `subject` are not available. + define_example_group_hook :after_suite - macro around_each(&block) - def %hook({{block.args.first || :example.id}}) : Nil - {{block.body}} - end + # Defines a block of code that will be invoked once before any examples in the group. + # The block will not run in the context of the current running example. + # This means that values defined by `let` and `subject` are not available. + define_example_group_hook :before_all - ::Spectator::SpecBuilder.add_around_each_hook { |test, proc| test.as({{@type.id}}).%hook(proc) } - end + # Defines a block of code that will be invoked once after all examples in the group. + # The block will not run in the context of the current running example. + # This means that values defined by `let` and `subject` are not available. + define_example_group_hook :after_all - macro pre_condition(&block) - def %hook({{block.args.splat}}) : Nil - {{block.body}} - end + # Defines a block of code that will be invoked before every example in the group. + # The block will be run in the context of the current running example. + # This means that values defined by `let` and `subject` are available. + define_example_hook :before_each - ::Spectator::SpecBuilder.add_pre_condition do |test, example| - cast_test = test.as({{@type.id}}) - {% if block.args.empty? %} - cast_test.%hook - {% else %} - cast_test.%hook(example) - {% end %} - end - end + # Defines a block of code that will be invoked after every example in the group. + # The block will be run in the context of the current running example. + # This means that values defined by `let` and `subject` are available. + define_example_hook :after_each - macro post_condition(&block) - def %hook({{block.args.splat}}) : Nil - {{block.body}} - end - - ::Spectator::SpecBuilder.add_post_condition do |test, example| - cast_test = test.as({{@type.id}}) - {% if block.args.empty? %} - cast_test.%hook - {% else %} - cast_test.%hook(example) - {% end %} - end - end + # Defines a block of code that will be invoked around every example in the group. + # The block will be run in the context of the current running example. + # This means that values defined by `let` and `subject` are available. + # + # The block will execute before the example. + # An `Example::Procsy` is passed to the block. + # The `Example::Procsy#run` method should be called to ensure the example runs. + # More code can run afterwards (in the block). + define_example_hook :around_each end end diff --git a/src/spectator/dsl/matchers.cr b/src/spectator/dsl/matchers.cr index 8332c44..34adf26 100644 --- a/src/spectator/dsl/matchers.cr +++ b/src/spectator/dsl/matchers.cr @@ -1,9 +1,9 @@ +require "../block" require "../matchers" -require "../test_block" -require "../test_value" +require "../value" -module Spectator - module DSL +module Spectator::DSL + module Matchers # Indicates that some value should equal another. # The == operator is used for this check. # The value passed to this method is the expected value. @@ -13,8 +13,8 @@ module Spectator # expect(1 + 2).to eq(3) # ``` macro eq(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::EqualityMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::EqualityMatcher.new(%value) end # Indicates that some value should not equal another. @@ -26,8 +26,8 @@ module Spectator # expect(1 + 2).to ne(5) # ``` macro ne(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::InequalityMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::InequalityMatcher.new(%value) end # Indicates that some value when compared to another satisfies an operator. @@ -61,8 +61,8 @@ module Spectator # expect(obj.dup).to_not be(obj) # ``` macro be(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::ReferenceMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::ReferenceMatcher.new(%value) end # Indicates that some value should be of a specified type. @@ -123,7 +123,7 @@ module Spectator end # Indicates that some value should be of a specified type. - # The value's runtime class is checked. + # The value's runtime type is checked. # A type name or type union should be used for *expected*. # # Examples: @@ -135,7 +135,7 @@ module Spectator end # Indicates that some value should be of a specified type. - # The value's runtime class is checked. + # The value's runtime type is checked. # A type name or type union should be used for *expected*. # This method is identical to `#be_an_instance_of`, # and exists just to improve grammar. @@ -148,6 +148,19 @@ module Spectator be_instance_of({{expected}}) end + # Indicates that some value should be of a specified type at compile time. + # The value's compile time type is checked. + # This can test is a variable or value returned by a method is inferred to the expected type. + # + # Examples: + # ``` + # value = 42 || "foobar" + # expect(value).to compile_as(Int32 | String) + # ``` + macro compile_as(expected) + ::Spectator::Matchers::CompiledTypeMatcher({{expected}}).new + end + # Indicates that some value should respond to a method call. # One or more method names can be provided. # @@ -173,8 +186,8 @@ module Spectator # expect(3 - 1).to be_lt(3) # ``` macro be_lt(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::LessThanMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::LessThanMatcher.new(%value) end # Indicates that some value should be less than or equal to another. @@ -186,8 +199,8 @@ module Spectator # expect(3 - 1).to be_le(3) # ``` macro be_le(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::LessThanEqualMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::LessThanEqualMatcher.new(%value) end # Indicates that some value should be greater than another. @@ -199,8 +212,8 @@ module Spectator # expect(3 + 1).to be_gt(3) # ``` macro be_gt(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::GreaterThanMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::GreaterThanMatcher.new(%value) end # Indicates that some value should be greater than or equal to another. @@ -212,8 +225,8 @@ module Spectator # expect(3 + 1).to be_ge(3) # ``` macro be_ge(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::GreaterThanEqualMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::GreaterThanEqualMatcher.new(%value) end # Indicates that some value should match another. @@ -230,8 +243,8 @@ module Spectator # expect({:foo, 5}).to match({Symbol, Int32}) # ``` macro match(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::CaseMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::CaseMatcher.new(%value) end # Indicates that some value should be true. @@ -321,8 +334,8 @@ module Spectator # NOTE: Do not attempt to mix the two use cases. # It likely won't work and will result in a compilation error. macro be_within(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::CollectionMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::CollectionMatcher.new(%value) end # Indicates that some value should be between a lower and upper-bound. @@ -344,8 +357,8 @@ module Spectator macro be_between(min, max) %range = Range.new({{min}}, {{max}}) %label = [{{min.stringify}}, {{max.stringify}}].join(" to ") - %test_value = ::Spectator::TestValue.new(%range, %label) - ::Spectator::Matchers::RangeMatcher.new(%test_value) + %value = ::Spectator::Value.new(%range, %label) + ::Spectator::Matchers::RangeMatcher.new(%value) end # Indicates that some value should be within a delta of an expected value. @@ -403,8 +416,8 @@ module Spectator # expect(%w[foo bar]).to start_with(/foo/) # ``` macro start_with(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::StartWithMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::StartWithMatcher.new(%value) end # Indicates that some value or set should end with another value. @@ -426,8 +439,8 @@ module Spectator # expect(%w[foo bar]).to end_with(/bar/) # ``` macro end_with(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::EndWithMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::EndWithMatcher.new(%value) end # Indicates that some value or set should contain another value. @@ -451,11 +464,11 @@ module Spectator # ``` macro contain(*expected) {% if expected.id.starts_with?("{*") %} - %test_value = ::Spectator::TestValue.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::ContainMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::ContainMatcher.new(%value) {% else %} - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::ContainMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::ContainMatcher.new(%value) {% end %} end @@ -475,8 +488,8 @@ module Spectator # expect(%i[a b c]).to contain_elements(%i[a b]) # ``` macro contain_elements(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::ContainMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::ContainMatcher.new(%value) end # Indicates that some range (or collection) should contain another value. @@ -497,11 +510,11 @@ module Spectator # ``` macro cover(*expected) {% if expected.id.starts_with?("{*") %} - %test_value = ::Spectator::TestValue.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::ContainMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::ContainMatcher.new(%value) {% else %} - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::ContainMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::ContainMatcher.new(%value) {% end %} end @@ -532,11 +545,11 @@ module Spectator # ``` macro have(*expected) {% if expected.id.starts_with?("{*") %} - %test_value = ::Spectator::TestValue.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::HaveMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected.id[2...-1]}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::HaveMatcher.new(%value) {% else %} - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.splat.stringify}}) - ::Spectator::Matchers::HaveMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.splat.stringify}}) + ::Spectator::Matchers::HaveMatcher.new(%value) {% end %} end @@ -559,8 +572,8 @@ module Spectator # expect([1, 2, 3, :a, :b, :c]).to have_elements([Int32, Symbol]) # ``` macro have_elements(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::HaveMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::HaveMatcher.new(%value) end # Indicates that some set, such as a `Hash`, has a given key. @@ -572,8 +585,8 @@ module Spectator # expect({"lucky" => 7}).to have_key("lucky") # ``` macro have_key(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::HaveKeyMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::HaveKeyMatcher.new(%value) end # :ditto: @@ -590,8 +603,8 @@ module Spectator # expect({"lucky" => 7}).to have_value(7) # ``` macro have_value(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::HaveValueMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::HaveValueMatcher.new(%value) end # :ditto: @@ -607,11 +620,11 @@ module Spectator # ``` macro contain_exactly(*expected) {% if expected.id.starts_with?("{*") %} - %test_value = ::Spectator::TestValue.new(({{expected.id[2...-1]}}).to_a, {{expected.stringify}}) - ::Spectator::Matchers::ArrayMatcher.new(%test_value) + %value = ::Spectator::Value.new(({{expected.id[2...-1]}}).to_a, {{expected.stringify}}) + ::Spectator::Matchers::ArrayMatcher.new(%value) {% else %} - %test_value = ::Spectator::TestValue.new(({{expected}}).to_a, {{expected.stringify}}) - ::Spectator::Matchers::ArrayMatcher.new(%test_value) + %value = ::Spectator::Value.new(({{expected}}).to_a, {{expected.stringify}}) + ::Spectator::Matchers::ArrayMatcher.new(%value) {% end %} end @@ -623,8 +636,8 @@ module Spectator # expect([1, 2, 3]).to match_array([3, 2, 1]) # ``` macro match_array(expected) - %test_value = ::Spectator::TestValue.new(({{expected}}).to_a, {{expected.stringify}}) - ::Spectator::Matchers::ArrayMatcher.new(%test_value) + %value = ::Spectator::Value.new(({{expected}}).to_a, {{expected.stringify}}) + ::Spectator::Matchers::ArrayMatcher.new(%value) end # Indicates that some set should have a specified size. @@ -634,8 +647,8 @@ module Spectator # expect([1, 2, 3]).to have_size(3) # ``` macro have_size(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::SizeMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::SizeMatcher.new(%value) end # Indicates that some set should have the same size (number of elements) as another set. @@ -645,8 +658,8 @@ module Spectator # expect([1, 2, 3]).to have_size_of(%i[x y z]) # ``` macro have_size_of(expected) - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.stringify}}) - ::Spectator::Matchers::SizeOfMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.stringify}}) + ::Spectator::Matchers::SizeOfMatcher.new(%value) end # Indicates that some value should have a set of attributes matching some conditions. @@ -661,11 +674,11 @@ module Spectator # ``` macro have_attributes(**expected) {% if expected.id.starts_with?("{**") %} - %test_value = ::Spectator::TestValue.new({{expected.id[3...-1]}}, {{expected.double_splat.stringify}}) - ::Spectator::Matchers::AttributesMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected.id[3...-1]}}, {{expected.double_splat.stringify}}) + ::Spectator::Matchers::AttributesMatcher.new(%value) {% else %} - %test_value = ::Spectator::TestValue.new({{expected}}, {{expected.double_splat.stringify}}) - ::Spectator::Matchers::AttributesMatcher.new(%test_value) + %value = ::Spectator::Value.new({{expected}}, {{expected.double_splat.stringify}}) + ::Spectator::Matchers::AttributesMatcher.new(%value) {% end %} end @@ -718,37 +731,18 @@ module Spectator # expect { subject << :foo }.to change(&.size).by(1) # ``` macro change(&expression) - {% if expression.is_a?(Nop) %} - {% raise "Block must be provided to change matcher" %} - {% end %} - - # Check if the short-hand method syntax is used. - # This is a hack, since macros don't get this as a "literal" or something similar. - # The Crystal compiler will translate: - # ``` - # &.foo - # ``` - # to: - # ``` - # { |__arg0| __arg0.foo } - # ``` - # The hack used here is to check if it looks like a compiler-generated block. - {% if expression.args == ["__arg0".id] && expression.body.is_a?(Call) && expression.body.id =~ /^__arg0\./ %} - # Extract the method name to make it clear to the user what is tested. - # The raw block can't be used because it's not clear to the user. + {% if expression.args.size == 1 && expression.args[0] =~ /^__arg\d+$/ && expression.body.is_a?(Call) && expression.body.id =~ /^__arg\d+\./ %} {% method_name = expression.body.id.split('.')[1..-1].join('.') %} - %proc = ->{ subject.{{method_name.id}} } - %test_block = ::Spectator::TestBlock.create(%proc, {{"#" + method_name}}) + %block = ::Spectator::Block.new({{"#" + method_name}}) do + subject.{{method_name.id}} + end {% elsif expression.args.empty? %} - # In this case, it looks like the short-hand method syntax wasn't used. - # Capture the block as a proc and pass along. - %proc = ->{{expression}} - %test_block = ::Spectator::TestBlock.create(%proc, {{"`" + expression.body.stringify + "`"}}) + %block = ::Spectator::Block.new({{"`" + expression.body.stringify + "`"}}) {{expression}} {% else %} - {% raise "Unexpected block arguments in change matcher" %} + {% raise "Unexpected block arguments in 'expect' call" %} {% end %} - ::Spectator::Matchers::ChangeMatcher.new(%test_block) + ::Spectator::Matchers::ChangeMatcher.new(%block) end # Indicates that some block should raise an error. @@ -828,8 +822,8 @@ module Spectator end macro have_received(method) - %test_value = ::Spectator::TestValue.new(({{method.id.symbolize}}), {{method.id.stringify}}) - ::Spectator::Matchers::ReceiveMatcher.new(%test_value) + %value = ::Spectator::Value.new(({{method.id.symbolize}}), {{method.id.stringify}}) + ::Spectator::Matchers::ReceiveMatcher.new(%value) end # Used to create predicate matchers. @@ -872,8 +866,8 @@ module Spectator {% end %} label << ')' {% end %} - test_value = ::Spectator::TestValue.new(descriptor, label.to_s) - ::Spectator::Matchers::{{matcher.id}}.new(test_value) + value = ::Spectator::Value.new(descriptor, label.to_s) + ::Spectator::Matchers::{{matcher.id}}.new(value) end end end diff --git a/src/spectator/dsl/memoize.cr b/src/spectator/dsl/memoize.cr new file mode 100644 index 0000000..6ba3c9d --- /dev/null +++ b/src/spectator/dsl/memoize.cr @@ -0,0 +1,107 @@ +require "../lazy_wrapper" + +module Spectator::DSL + # DSL methods for defining test values (subjects). + # These values are stored and reused throughout the test. + module Memoize + # Defines a memoized getter. + # The *name* is the name of the getter method. + # The block is evaluated only on the first time the getter is used + # and the return value is saved for subsequent calls. + macro let(name, &block) + {% raise "Missing block for 'let'" unless block %} + {% raise "Expected zero or one arguments for 'let', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'let' inside of an example block" if @def %} + {% raise "Cannot use '#{name.id}' for 'let'" if ::Spectator::DSL::RESERVED_KEYWORDS.includes?(name.id.symbolize) %} + + @%value = ::Spectator::LazyWrapper.new + + private def {{name.id}} + {% if block.args.size > 0 %} + {{block.args.first}} = ::Spectator::Example.current + {% end %} + @%value.get do + {{block.body}} + end + end + end + + # Defines a memoized getter. + # The *name* is the name of the getter method. + # The block is evaluated once before the example runs + # and the return value is saved. + macro let!(name, &block) + {% raise "Missing block for 'let!'" unless block %} + {% raise "Expected zero or one arguments for 'let!', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'let!' inside of an example block" if @def %} + {% raise "Cannot use '#{name.id}' for 'let!'" if ::Spectator::DSL::RESERVED_KEYWORDS.includes?(name.id.symbolize) %} + + let({{name}}) {{block}} + before_each { {{name.id}} } + end + + # Explicitly defines the subject of the tests. + # Creates a memoized getter for the subject. + # The block is evaluated only the first time the subject is referenced + # and the return value is saved for subsequent calls. + macro subject(&block) + {% raise "Missing block for 'subject'" unless block %} + {% raise "Expected zero or one arguments for 'let!', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'subject' inside of an example block" if @def %} + + let(subject) {{block}} + end + + # Explicitly defines the subject of the tests. + # Creates a memoized getter for the subject. + # The subject can be referenced by using `subject` or *name*. + # The block is evaluated only the first time the subject is referenced + # and the return value is saved for subsequent calls. + macro subject(name, &block) + {% raise "Missing block for 'subject'" unless block %} + {% raise "Expected zero or one arguments for 'subject', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'subject' inside of an example block" if @def %} + {% raise "Cannot use '#{name.id}' for 'subject'" if ::Spectator::DSL::RESERVED_KEYWORDS.includes?(name.id.symbolize) %} + + let({{name.id}}) {{block}} + + {% if name.id != :subject.id %} + private def subject + {{name.id}} + end + {% end %} + end + + # Explicitly defines the subject of the tests. + # Creates a memoized getter for the subject. + # The block is evaluated once before the example runs + # and the return value is saved for subsequent calls. + macro subject!(&block) + {% raise "Missing block for 'subject'" unless block %} + {% raise "Expected zero or one arguments for 'subject!', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'subject!' inside of an example block" if @def %} + + let!(subject) {{block}} + end + + # Explicitly defines the subject of the tests. + # Creates a memoized getter for the subject. + # The subject can be referenced by using `subject` or *name*. + # The block is evaluated once before the example runs + # and the return value is saved for subsequent calls. + macro subject!(name, &block) + {% raise "Missing block for 'subject'" unless block %} + {% raise "Expected zero or one arguments for 'subject!', but got #{block.args.size}" if block.args.size > 1 %} + {% raise "Cannot use 'subject!' inside of an example block" if @def %} + {% raise "Cannot use '#{name.id}' for 'subject!'" if ::Spectator::DSL::RESERVED_KEYWORDS.includes?(name.id.symbolize) %} + + let!({{name.id}}) {{block}} + + {% if name.id != :subject.id %} + private def subject + {{name.id}} + end + {% end %} + end + end +end diff --git a/src/spectator/dsl/metadata.cr b/src/spectator/dsl/metadata.cr new file mode 100644 index 0000000..308bcbd --- /dev/null +++ b/src/spectator/dsl/metadata.cr @@ -0,0 +1,26 @@ +module Spectator::DSL + module Metadata + # Defines a class method named *name* that combines metadata + # returned by *source* with *tags* and *metadata*. + # Any falsey items from *metadata* are removed. + private macro _spectator_metadata(name, source, *tags, **metadata) + private def self.{{name.id}} + %metadata = {{source.id}}.dup + {% for k in tags %} + %metadata[{{k.id.symbolize}}] = nil + {% end %} + {% for k, v in metadata %} + %cond = begin + {{v}} + end + if %cond + %metadata[{{k.id.symbolize}}] = %cond.to_s + else + %metadata.delete({{k.id.symbolize}}) + end + {% end %} + %metadata + end + end + end +end diff --git a/src/spectator/dsl/mocks.cr b/src/spectator/dsl/mocks.cr index 640c054..7e75738 100644 --- a/src/spectator/dsl/mocks.cr +++ b/src/spectator/dsl/mocks.cr @@ -1,185 +1,183 @@ require "../mocks" module Spectator::DSL - macro double(name = "Anonymous", **stubs, &block) - {% if name.is_a?(StringLiteral) || name.is_a?(StringInterpolation) %} - anonymous_double({{name}}, {{stubs.double_splat}}) - {% else %} - {% - safe_name = name.id.symbolize.gsub(/\W/, "_").id - type_name = "Double#{safe_name}".id - %} - - {% if block %} - define_double({{type_name}}, {{name}}, {{stubs.double_splat}}) {{block}} + module Mocks + macro double(name = "Anonymous", **stubs, &block) + {% if name.is_a?(StringLiteral) || name.is_a?(StringInterpolation) %} + anonymous_double({{name}}, {{stubs.double_splat}}) {% else %} - create_double({{type_name}}, {{name}}, {{stubs.double_splat}}) - {% end %} - {% end %} - end + {% + safe_name = name.id.symbolize.gsub(/\W/, "_").id + type_name = "Double#{safe_name}".id + %} - macro create_double(type_name, name, **stubs) - {% if type_name.resolve? %} - {{type_name}}.new.tap do |%double| - {% for name, value in stubs %} - allow(%double).to receive({{name.id}}).and_return({{value}}) + {% if block %} + define_double({{type_name}}, {{name}}, {{stubs.double_splat}}) {{block}} + {% else %} + create_double({{type_name}}, {{name}}, {{stubs.double_splat}}) {% end %} - end - {% elsif @def %} - anonymous_double({{name ? name.stringify : "Anonymous"}}, {{stubs.double_splat}}) - {% else %} - {% raise "Block required for double definition" %} - {% end %} - end - - macro define_double(type_name, name, **stubs, &block) - {% begin %} - {% if (name.is_a?(Path) || name.is_a?(Generic)) && (resolved = name.resolve?) %} - verify_double({{name}}) - class {{type_name}} < ::Spectator::Mocks::VerifyingDouble(::{{resolved.id}}) - {% else %} - class {{type_name}} < ::Spectator::Mocks::Double - def initialize(null = false) - super({{name.id.stringify}}, null) - end {% end %} - - def as_null_object - {{type_name}}.new(true) - end - - # TODO: Do something with **stubs? - - {{block.body}} end - {% end %} - end - def anonymous_double(name = "Anonymous", **stubs) - Mocks::AnonymousDouble.new(name, stubs) - end - - macro null_double(name, **stubs, &block) - {% if name.is_a?(StringLiteral) || name.is_a?(StringInterpolation) %} - anonymous_null_double({{name}}, {{stubs.double_splat}}) - {% else %} - {% - safe_name = name.id.symbolize.gsub(/\W/, "_").id - type_name = "Double#{safe_name}".id - %} - - {% if block %} - define_null_double({{type_name}}, {{name}}, {{stubs.double_splat}}) {{block}} + macro create_double(type_name, name, **stubs) + {% if type_name.resolve? %} + {{type_name}}.new.tap do |%double| + {% for name, value in stubs %} + allow(%double).to receive({{name.id}}).and_return({{value}}) + {% end %} + end + {% elsif @def %} + anonymous_double({{name ? name.stringify : "Anonymous"}}, {{stubs.double_splat}}) {% else %} - create_null_double({{type_name}}, {{name}}, {{stubs.double_splat}}) + {% raise "Block required for double definition" %} {% end %} - {% end %} - end + end + + macro define_double(type_name, name, **stubs, &block) + {% begin %} + {% if (name.is_a?(Path) || name.is_a?(Generic)) && (resolved = name.resolve?) %} + verify_double({{name}}) + class {{type_name}} < ::Spectator::Mocks::VerifyingDouble(::{{resolved.id}}) + {% else %} + class {{type_name}} < ::Spectator::Mocks::Double + def initialize(null = false) + super({{name.id.stringify}}, null) + end + {% end %} + + def as_null_object + {{type_name}}.new(true) + end + + # TODO: Do something with **stubs? + + {{block.body}} + end + {% end %} + end + + def anonymous_double(name = "Anonymous", **stubs) + ::Spectator::Mocks::AnonymousDouble.new(name, stubs) + end + + macro null_double(name, **stubs, &block) + {% if name.is_a?(StringLiteral) || name.is_a?(StringInterpolation) %} + anonymous_null_double({{name}}, {{stubs.double_splat}}) + {% else %} + {% + safe_name = name.id.symbolize.gsub(/\W/, "_").id + type_name = "Double#{safe_name}".id + %} + + {% if block.is_a?(Nop) %} + create_null_double({{type_name}}, {{name}}, {{stubs.double_splat}}) + {% else %} + define_null_double({{type_name}}, {{name}}, {{stubs.double_splat}}) {{block}} + {% end %} + {% end %} + end + + macro create_null_double(type_name, name, **stubs) + {% type_name.resolve? || raise("Could not find a double labeled #{name}") %} - macro create_null_double(type_name, name, **stubs) - {% if type_name.resolve? %} {{type_name}}.new(true).tap do |%double| {% for name, value in stubs %} allow(%double).to receive({{name.id}}).and_return({{value}}) {% end %} end - {% elsif @def %} - anonymous_null_double({{name ? name.stringify : "Anonymous"}}, {{stubs.double_splat}}) - {% else %} - {% raise "Block required for double definition" %} - {% end %} - end - - macro define_null_double(type_name, name, **stubs, &block) - class {{type_name}} < ::Spectator::Mocks::Double - def initialize(null = true) - super({{name.id.stringify}}, null) - end - - def as_null_object - {{type_name}}.new(true) - end - - # TODO: Do something with **stubs? - - {{block.body}} end - end - def anonymous_null_double(name = "Anonymous", **stubs) - AnonymousNullDouble.new(name, stubs) - end + macro define_null_double(type_name, name, **stubs, &block) + class {{type_name}} < ::Spectator::Mocks::Double + def initialize(null = true) + super({{name.id.stringify}}, null) + end - macro mock(name, &block) - {% resolved = name.resolve - type = if resolved < Reference - :class - elsif resolved < Value - :struct - else - :module - end %} - {% begin %} - {{type.id}} ::{{resolved.id}} - include ::Spectator::Mocks::Stubs + def as_null_object + {{type_name}}.new(true) + end + + # TODO: Do something with **stubs? {{block.body}} end - {% end %} - end + end - macro verify_double(name, &block) - {% resolved = name.resolve - type = if resolved < Reference - :class - elsif resolved < Value - :struct - else - :module - end %} - {% begin %} - {{type.id}} ::{{resolved.id}} - include ::Spectator::Mocks::Reflection + def anonymous_null_double(name = "Anonymous", **stubs) + ::Spectator::Mocks::AnonymousNullDouble.new(name, stubs) + end - macro finished - _spectator_reflect + macro mock(name, &block) + {% resolved = name.resolve + type = if resolved < Reference + :class + elsif resolved < Value + :struct + else + :module + end %} + {% begin %} + {{type.id}} ::{{resolved.id}} + include ::Spectator::Mocks::Stubs + + {{block.body}} end - end - {% end %} - end + {% end %} + end - def allow(thing) - Mocks::Allow.new(thing) - end + macro verify_double(name, &block) + {% resolved = name.resolve + type = if resolved < Reference + :class + elsif resolved < Value + :struct + else + :module + end %} + {% begin %} + {{type.id}} ::{{resolved.id}} + include ::Spectator::Mocks::Reflection - def allow_any_instance_of(type : T.class) forall T - Mocks::AllowAnyInstance(T).new - end + macro finished + _spectator_reflect + end + end + {% end %} + end - macro expect_any_instance_of(type, _source_file = __FILE__, _source_line = __LINE__) - %source = ::Spectator::Source.new({{_source_file}}, {{_source_line}}) - ::Spectator::Mocks::ExpectAnyInstance({{type}}).new(%source) - end + def allow(thing) + ::Spectator::Mocks::Allow.new(thing) + end - macro receive(method_name, _source_file = __FILE__, _source_line = __LINE__, &block) - %source = ::Spectator::Source.new({{_source_file}}, {{_source_line}}) - {% if block %} - ::Spectator::Mocks::ProcMethodStub.create({{method_name.id.symbolize}}, %source) { {{block.body}} } - {% else %} - ::Spectator::Mocks::NilMethodStub.new({{method_name.id.symbolize}}, %source) - {% end %} - end + def allow_any_instance_of(type : T.class) forall T + ::Spectator::Mocks::AllowAnyInstance(T).new + end - macro receive_messages(_source_file = __FILE__, _source_line = __LINE__, **stubs) - %source = ::Spectator::Source.new({{_source_file}}, {{_source_line}}) - %stubs = [] of ::Spectator::Mocks::MethodStub - {% for name, value in stubs %} - %stubs << ::Spectator::Mocks::ValueMethodStub.new({{name.id.symbolize}}, %source, {{value}}) - {% end %} - %stubs - end + macro expect_any_instance_of(type, _source_file = __FILE__, _source_line = __LINE__) + %location = ::Spectator::Location.new({{_source_file}}, {{_source_line}}) + ::Spectator::Mocks::ExpectAnyInstance({{type}}).new(%location) + end - def no_args - ::Spectator::Mocks::NoArguments.new + macro receive(method_name, _source_file = __FILE__, _source_line = __LINE__, &block) + %location = ::Spectator::Location.new({{_source_file}}, {{_source_line}}) + {% if block %} + ::Spectator::Mocks::ProcMethodStub.create({{method_name.id.symbolize}}, %location) { {{block.body}} } + {% else %} + ::Spectator::Mocks::NilMethodStub.new({{method_name.id.symbolize}}, %location) + {% end %} + end + + macro receive_messages(_source_file = __FILE__, _source_line = __LINE__, **stubs) + %location = ::Spectator::Location.new({{_source_file}}, {{_source_line}}) + %stubs = [] of ::Spectator::Mocks::MethodStub + {% for name, value in stubs %} + %stubs << ::Spectator::Mocks::ValueMethodStub.new({{name.id.symbolize}}, %location, {{value}}) + {% end %} + %stubs + end + + def no_args + ::Spectator::Mocks::NoArguments.new + end end end diff --git a/src/spectator/dsl/top.cr b/src/spectator/dsl/top.cr new file mode 100644 index 0000000..e70dd8f --- /dev/null +++ b/src/spectator/dsl/top.cr @@ -0,0 +1,32 @@ +require "./groups" + +module Spectator::DSL + module Top + {% for method in %i[example_group describe context] %} + # Top-level describe method. + # All specs in a file must be wrapped in this call. + # This takes an argument and a block. + # The argument is what your spec is describing. + # It can be any Crystal expression, + # but is typically a class name or feature string. + # The block should contain all of the examples for what is being described. + # + # Example: + # ``` + # Spectator.describe Foo do + # # Your examples for `Foo` go here. + # end + # ``` + # + # Tags can be specified by adding symbols (keywords) after the first argument. + # Key-value pairs can also be specified. + # + # NOTE: Inside the block, the `Spectator` prefix _should not_ be used. + macro {{method.id}}(description, *tags, **metadata, &block) + class ::SpectatorTestContext + {{method.id}}(\{{description}}, \{{tags.splat(", ")}} \{{metadata.double_splat}}) \{{block}} + end + end + {% end %} + end +end diff --git a/src/spectator/dsl/values.cr b/src/spectator/dsl/values.cr deleted file mode 100644 index 6b1ba51..0000000 --- a/src/spectator/dsl/values.cr +++ /dev/null @@ -1,65 +0,0 @@ -module Spectator - module DSL - macro let(name, &block) - @%wrapper : ::Spectator::ValueWrapper? - - def {{name.id}} - {{block.body}} - end - - def {{name.id}} - if (wrapper = @%wrapper) - wrapper.as(::Spectator::TypedValueWrapper(typeof(previous_def))).value - else - previous_def.tap do |value| - @%wrapper = ::Spectator::TypedValueWrapper.new(value) - end - end - end - end - - macro let!(name, &block) - @%wrapper : ::Spectator::ValueWrapper? - - def %wrapper - {{block.body}} - end - - before_each do - @%wrapper = ::Spectator::TypedValueWrapper.new(%wrapper) - end - - def {{name.id}} - @%wrapper.as(::Spectator::TypedValueWrapper(typeof(%wrapper))).value - end - end - - macro subject(&block) - {% if block.is_a?(Nop) %} - self.subject - {% else %} - let(:subject) {{block}} - {% end %} - end - - macro subject(name, &block) - let({{name.id}}) {{block}} - - def subject - {{name.id}} - end - end - - macro subject!(&block) - let!(:subject) {{block}} - end - - macro subject!(name, &block) - let!({{name.id}}) {{block}} - - def subject - {{name.id}} - end - end - end -end diff --git a/src/spectator/error_result.cr b/src/spectator/error_result.cr new file mode 100644 index 0000000..4babc2a --- /dev/null +++ b/src/spectator/error_result.cr @@ -0,0 +1,28 @@ +require "./fail_result" + +module Spectator + # Outcome that indicates running an example generated an error. + # This occurs when an unexpected exception was raised while running an example. + # This is different from a "failed" result in that the error was not from a failed assertion. + class ErrorResult < FailResult + # Calls the `error` method on *visitor*. + def accept(visitor) + visitor.error(self) + end + + # Calls the `error` method on *visitor*. + def accept(visitor) + visitor.error(yield self) + end + + # One-word description of the result. + def to_s(io) + io << "error" + end + + # String used for the JSON status field. + private def json_status + "error" + end + end +end diff --git a/src/spectator/errored_result.cr b/src/spectator/errored_result.cr deleted file mode 100644 index 00c73ff..0000000 --- a/src/spectator/errored_result.cr +++ /dev/null @@ -1,50 +0,0 @@ -require "./failed_result" - -module Spectator - # Outcome that indicates running an example generated an error. - # This type of result occurs when an exception was raised. - # This is different from a "failed" result - # in that the error was not from a failed expectation. - class ErroredResult < FailedResult - # Calls the `error` method on *interface*. - def call(interface) - interface.error - end - - # Calls the `error` method on *interface* - # and passes the yielded value. - def call(interface) - value = yield self - interface.error(value) - end - - # One-word descriptor of the result. - def to_s(io) - io << "error" - end - - # Adds the common JSON fields for all result types - # and fields specific to errored results. - private def add_json_fields(json : ::JSON::Builder) - super - json.field("exceptions") do - exception = error - json.array do - while exception - error_to_json(exception, json) if exception - exception = error.cause - end - end - end - end - - # Adds a single exception to a JSON builder. - private def error_to_json(error : Exception, json : ::JSON::Builder) - json.object do - json.field("type", error.class.to_s) - json.field("message", error.message) - json.field("backtrace", error.backtrace) - end - end - end -end diff --git a/src/spectator/example.cr b/src/spectator/example.cr index cc955d1..f9efc42 100644 --- a/src/spectator/example.cr +++ b/src/spectator/example.cr @@ -1,83 +1,259 @@ -require "./example_component" -require "./test_wrapper" +require "./example_context_delegate" +require "./example_group" +require "./harness" +require "./location" +require "./node" +require "./pending_result" +require "./result" +require "./metadata" module Spectator - # Base class for all types of examples. - # Concrete types must implement the `#run_impl` method. - abstract class Example < ExampleComponent - @finished = false - @description : String? = nil + # Standard example that runs a test case. + class Example < Node + # Currently running example. + class_getter! current : Example - protected setter description + # Group the node belongs to. + getter! group : ExampleGroup - # Indicates whether the example has already been run. - def finished? : Bool - @finished + # Assigns the node to the specified *group*. + # This is an internal method and should only be called from `ExampleGroup`. + # `ExampleGroup` manages the association of nodes to groups. + protected setter group : ExampleGroup? + + # Indicates whether the example already ran. + getter? finished : Bool = false + + # Result of the last time the example ran. + # Is pending if the example hasn't run. + getter result : Result = PendingResult.new("Example not run") + + # Creates the example. + # An instance to run the test code in is given by *context*. + # The *entrypoint* defines the test code (typically inside *context*). + # The *name* describes the purpose of the example. + # It can be a `Symbol` to describe a type. + # The *location* tracks where the example exists in source code. + # The example will be assigned to *group* if it is provided. + # A set of *metadata* can be used for filtering and modifying example behavior. + # Note: The metadata will not be merged with the parent metadata. + def initialize(@context : Context, @entrypoint : self ->, + name : String? = nil, location : Location? = nil, + @group : ExampleGroup? = nil, metadata = Metadata.new) + super(name, location, metadata) + + # Ensure group is linked. + group << self if group end - # Group that the example belongs to. - getter group : ExampleGroup + # Creates a dynamic example. + # A block provided to this method will be called as-if it were the test code for the example. + # The block will be given this example instance as an argument. + # The *name* describes the purpose of the example. + # It can be a `Symbol` to describe a type. + # The *location* tracks where the example exists in source code. + # The example will be assigned to *group* if it is provided. + # A set of *metadata* can be used for filtering and modifying example behavior. + # Note: The metadata will not be merged with the parent metadata. + def initialize(name : String? = nil, location : Location? = nil, + @group : ExampleGroup? = nil, metadata = Metadata.new, &block : self ->) + super(name, location, metadata) - # Retrieves the internal wrapped instance. - protected getter test_wrapper : TestWrapper + @context = NullContext.new + @entrypoint = block - # Source where the example originated from. - def source : Source - @test_wrapper.source + # Ensure group is linked. + group << self if group end - def description : String | Symbol - @description || @test_wrapper.description + # Creates a pending example. + # The *name* describes the purpose of the example. + # It can be a `Symbol` to describe a type. + # The *location* tracks where the example exists in source code. + # The example will be assigned to *group* if it is provided. + # A set of *metadata* can be used for filtering and modifying example behavior. + # Note: The metadata will not be merged with the parent metadata. + def self.pending(name : String? = nil, location : Location? = nil, + group : ExampleGroup? = nil, metadata = Metadata.new, reason = nil) + # Add pending tag and reason if they don't exist. + metadata = metadata.merge({:pending => nil, :reason => reason}) { |_, v, _| v } + new(name, location, group, metadata) { nil } end - def symbolic? : Bool - return false unless @test_wrapper.description? - - description = @test_wrapper.description - description.starts_with?('#') || description.starts_with?('.') - end - - abstract def run_impl - - # Runs the example code. - # A result is returned, which represents the outcome of the test. - # An example can be run only once. - # An exception is raised if an attempt is made to run it more than once. + # Executes the test case. + # Returns the result of the execution. + # The result will also be stored in `#result`. def run : Result - raise "Attempted to run example more than once (#{self})" if finished? - run_impl - ensure + Log.debug { "Running example #{self}" } + Log.warn { "Example #{self} already ran" } if @finished + + if pending? + Log.debug { "Skipping example #{self} - marked pending" } + @finished = true + return @result = PendingResult.new(pending_reason) + end + + previous_example = @@current + @@current = self + + begin + @result = Harness.run do + @group.try(&.call_before_all) + if (parent = @group) + parent.call_around_each(procsy).call + else + run_internal + end + if (parent = @group) + parent.call_after_all if parent.finished? + end + end + ensure + @@current = previous_example + @finished = true + end + end + + private def run_internal + @group.try(&.call_before_each(self)) + @entrypoint.call(self) @finished = true + @group.try(&.call_after_each(self)) end - # Creates the base of the example. - # The group should be the example group the example belongs to. - def initialize(@group, @test_wrapper) + # Executes code within the example's test context. + # This is an advanced method intended for internal usage only. + # + # The *klass* defines the type of the test context. + # This is typically only known by the code constructing the example. + # An error will be raised if *klass* doesn't match the test context's type. + # The block given to this method will be executed within the test context. + # + # The context casted to an instance of *klass* is provided as a block argument. + # + # TODO: Benchmark compiler performance using this method versus client-side casting in a proc. + protected def with_context(klass) + context = klass.cast(@context) + with context yield end - # Indicates there is only one example to run. - def example_count : Int - 1 + # Casts the example's test context to a specific type. + # This is an advanced method intended for internal usage only. + # + # The *klass* defines the type of the test context. + # This is typically only known by the code constructing the example. + # An error will be raised if *klass* doesn't match the test context's type. + # + # The context casted to an instance of *klass* is returned. + # + # TODO: Benchmark compiler performance using this method versus client-side casting in a proc. + protected def cast_context(klass) + klass.cast(@context) end - # Retrieve the current example. - def [](index : Int) : Example - self + # Yields this example and all parent groups. + def ascend + node = self + while node + yield node + node = node.group? + end end - # String representation of the example. - # This consists of the groups the example is in and the description. - # The string can be given to end-users to identify the example. + # Constructs the full name or description of the example. + # This prepends names of groups this example is part of. def to_s(io) - @group.to_s(io) - io << ' ' unless symbolic? && @group.symbolic? - io << description + name = @name + + # Prefix with group's full name if the node belongs to a group. + if (parent = @group) + parent.to_s(io) + + # Add padding between the node names + # only if the names don't appear to be symbolic. + # Skip blank group names (like the root group). + io << ' ' unless !parent.name? || # ameba:disable Style/NegatedConditionsInUnless + (parent.name?.is_a?(Symbol) && name.is_a?(String) && + (name.starts_with?('#') || name.starts_with?('.'))) + end + + super + end + + # Exposes information about the example useful for debugging. + def inspect(io) + super + io << ' ' << result end # Creates the JSON representation of the example, # which is just its name. - def to_json(json : ::JSON::Builder) - json.string(to_s) + def to_json(json : JSON::Builder) + json.object do + json.field("description", name? || "") + json.field("full_description", to_s) + if location = location? + json.field("file_path", location.path) + json.field("line_number", location.line) + end + @result.to_json(json) if @finished + end + end + + # Creates a procsy from this example that runs the example. + def procsy + Procsy.new(self) { run_internal } + end + + # Creates a procsy from this example and the provided block. + def procsy(&block : ->) + Procsy.new(self, &block) + end + + # Wraps an example to behave like a `Proc`. + # This is typically used for an *around_each* hook. + # Invoking `#call` or `#run` will run the example. + struct Procsy + # Underlying example that will run. + getter example : Example + + # Creates the example proxy. + # The *example* should be run eventually. + # The *proc* defines the block of code to run when `#call` or `#run` is invoked. + def initialize(@example : Example, &@proc : ->) + end + + # Invokes the proc. + def call : Nil + @proc.call + end + + # Invokes the proc. + def run : Nil + @proc.call + end + + # Creates a new procsy for a block and the example from this instance. + def wrap(&block : ->) : self + self.class.new(@example, &block) + end + + # Executes code within the example's test context. + # This is an advanced method intended for internal usage only. + # + # The *klass* defines the type of the test context. + # This is typically only known by the code constructing the example. + # An error will be raised if *klass* doesn't match the test context's type. + # The block given to this method will be executed within the test context. + # + # TODO: Benchmark compiler performance using this method versus client-side casting in a proc. + protected def with_context(klass) + context = @example.cast_context(klass) + with context yield + end + + # Allow instance to behave like an example. + forward_missing_to @example end end end diff --git a/src/spectator/example_builder.cr b/src/spectator/example_builder.cr new file mode 100644 index 0000000..c3cc01e --- /dev/null +++ b/src/spectator/example_builder.cr @@ -0,0 +1,27 @@ +require "./context" +require "./example" +require "./location" +require "./metadata" +require "./node_builder" + +module Spectator + # Constructs examples. + # Call `#build` to produce an `Example`. + class ExampleBuilder < NodeBuilder + # Creates the builder. + # A proc provided by *context_builder* is used to create a unique `Context` for each example produced by `#build`. + # The *entrypoint* indicates the proc used to invoke the test code in the example. + # The *name*, *location*, and *metadata* will be applied to the `Example` produced by `#build`. + def initialize(@context_builder : -> Context, @entrypoint : Example ->, + @name : String? = nil, @location : Location? = nil, @metadata : Metadata = Metadata.new) + end + + # Constructs an example with previously defined attributes and context. + # The *parent* is an already constructed example group to nest the new example under. + # It can be nil if the new example won't have a parent. + def build(parent = nil) + context = @context_builder.call + Example.new(context, @entrypoint, @name, @location, parent, @metadata) + end + end +end diff --git a/src/spectator/example_component.cr b/src/spectator/example_component.cr deleted file mode 100644 index 9260f95..0000000 --- a/src/spectator/example_component.cr +++ /dev/null @@ -1,26 +0,0 @@ -module Spectator - # Abstract base for all examples and collections of examples. - # This is used as the base node type for the composite design pattern. - abstract class ExampleComponent - # Text that describes the context or test. - abstract def description : Symbol | String - - def full_description - to_s - end - - abstract def source : Source - - # Indicates whether the example (or group) has been completely run. - abstract def finished? : Bool - - # The number of examples in this instance. - abstract def example_count : Int - - # Lookup the example with the specified index. - abstract def [](index : Int) : Example - - # Indicates that the component references a type or method. - abstract def symbolic? : Bool - end -end diff --git a/src/spectator/example_conditions.cr b/src/spectator/example_conditions.cr deleted file mode 100644 index 0fbd03d..0000000 --- a/src/spectator/example_conditions.cr +++ /dev/null @@ -1,41 +0,0 @@ -module Spectator - # Collection of checks that run before and after tests. - # The pre-conditions can be used to verify - # that the SUT is in an expected state prior to testing. - # The post-conditions can be used to verify - # that the SUT is in an expected state after tests have finished. - # Each check is just a `Proc` (code block) that runs when invoked. - class ExampleConditions - # Creates an empty set of conditions. - # This will effectively run nothing extra while running a test. - def self.empty - new( - [] of TestMetaMethod, - [] of TestMetaMethod - ) - end - - # Creates a new set of conditions. - def initialize( - @pre_conditions : Array(TestMetaMethod), - @post_conditions : Array(TestMetaMethod) - ) - end - - # Runs all pre-condition checks. - # These should be run before every test. - def run_pre_conditions(wrapper : TestWrapper, example : Example) - @pre_conditions.each do |hook| - wrapper.call(hook, example) - end - end - - # Runs all post-condition checks. - # These should be run after every test. - def run_post_conditions(wrapper : TestWrapper, example : Example) - @post_conditions.each do |hook| - wrapper.call(hook, example) - end - end - end -end diff --git a/src/spectator/example_context_delegate.cr b/src/spectator/example_context_delegate.cr new file mode 100644 index 0000000..0b61f57 --- /dev/null +++ b/src/spectator/example_context_delegate.cr @@ -0,0 +1,33 @@ +require "./context" +require "./example_context_method" +require "./null_context" + +module Spectator + # Stores a test context and a method to call within it. + # This is a variant of `ContextDelegate` that accepts the current running example. + struct ExampleContextDelegate + # Retrieves the underlying context. + protected getter context : Context + + # Creates the delegate. + # The *context* is the instance of the test context. + # The *method* is proc that downcasts *context* and calls a method on it. + def initialize(@context : Context, @method : ExampleContextMethod) + end + + # Creates a delegate with a null context. + # The context will be ignored and the block will be executed in its original scope. + # The example instance will be provided as an argument to the block. + def self.null(&block : Example -> _) + context = NullContext.new + method = ExampleContextMethod.new { |example| block.call(example) } + new(context, method) + end + + # Invokes a method in the test context. + # The *example* is the current running example. + def call(example : Example) + @method.call(example, @context) + end + end +end diff --git a/src/spectator/example_context_method.cr b/src/spectator/example_context_method.cr new file mode 100644 index 0000000..c14d254 --- /dev/null +++ b/src/spectator/example_context_method.cr @@ -0,0 +1,10 @@ +require "./context" + +module Spectator + # Encapsulates a method in a test context. + # This could be used to invoke a test case or hook method. + # The context is passed as an argument. + # The proc should downcast the context instance to the desired type and call a method on that context. + # The current example is also passed as an argument. + alias ExampleContextMethod = Example, Context -> +end diff --git a/src/spectator/example_failed.cr b/src/spectator/example_failed.cr index a1b74f7..2170604 100644 --- a/src/spectator/example_failed.cr +++ b/src/spectator/example_failed.cr @@ -1,5 +1,14 @@ +require "./location" + module Spectator - # Exception that indicates an example failed and should abort. + # Exception that indicates an example failed. + # When raised within a test, the test should abort. class ExampleFailed < Exception + getter! location : Location + + # Creates the exception. + def initialize(@location : Location?, message : String? = nil, cause : Exception? = nil) + super(message, cause) + end end end diff --git a/src/spectator/example_filter.cr b/src/spectator/example_filter.cr deleted file mode 100644 index 0aa3ab2..0000000 --- a/src/spectator/example_filter.cr +++ /dev/null @@ -1,9 +0,0 @@ -module Spectator - # Base class for all example filters. - # Checks whether an example should be run. - # Sub-classes must implement the `#includes?` method. - abstract class ExampleFilter - # Checks if an example is in the filter, and should be run. - abstract def includes?(example : Example) : Bool - end -end diff --git a/src/spectator/example_group.cr b/src/spectator/example_group.cr index ae8a5a8..e57b2df 100644 --- a/src/spectator/example_group.cr +++ b/src/spectator/example_group.cr @@ -1,131 +1,133 @@ -require "./example_component" +require "./example_procsy_hook" +require "./hooks" +require "./node" module Spectator - # Shared base class for groups of examples. - # - # Represents a collection of examples and other groups. - # Use the `#each` methods to iterate through each child. - # However, these methods do not recurse into sub-groups. - # If you need that functionality, see `ExampleIterator`. - # Additionally, the indexer method (`#[]`) will index into sub-groups. - # - # This class also stores hooks to be associated with all examples in the group. - # The hooks can be invoked by running the `#run_before_hooks` and `#run_after_hooks` methods. - abstract class ExampleGroup < ExampleComponent - include Enumerable(ExampleComponent) - include Iterable(ExampleComponent) + # Collection of examples and sub-groups. + class ExampleGroup < Node + include Hooks + include Indexable(Node) - @example_count = 0 + @nodes = [] of Node - # Retrieves the children in the group. - # This only returns the direct descends (non-recursive). - # The children must be set (with `#children=`) prior to calling this method. - getter! children : Array(ExampleComponent) + # Parent group this group belongs to. + getter! group : ExampleGroup - # Sets the children of the group. - # This should be called only from a builder in the `DSL` namespace. - # The children can be set only once - - # attempting to set more than once will raise an error. - # All sub-groups' children should be set before setting this group's children. - def children=(children : Array(ExampleComponent)) - raise "Attempted to reset example group children" if @children - @children = children - # Recursively count the number of examples. - # This won't work if a sub-group hasn't had their children set (is still nil). - @example_count = children.sum(&.example_count) + # Assigns this group to the specified *group*. + # This is an internal method and should only be called from `ExampleGroup`. + # `ExampleGroup` manages the association of nodes to groups. + protected setter group : ExampleGroup? + + define_hook before_all : ExampleGroupHook do + Log.trace { "Processing before_all hooks for #{self}" } + + @group.try &.call_before_all + before_all_hooks.each &.call_once end - def double(id, sample_values) - @doubles[id].build(sample_values) - end + define_hook after_all : ExampleGroupHook, :prepend do + Log.trace { "Processing after_all hooks for #{self}" } - getter context - - def initialize(@context : TestContext) - end - - # Yields each direct descendant. - def each - children.each do |child| - yield child + after_all_hooks.each &.call_once if finished? + if group = @group + group.call_after_all if group.finished? end end - # Returns an iterator for each direct descendant. - def each : Iterator(ExampleComponent) - children.each + define_hook before_each : ExampleHook do |example| + Log.trace { "Processing before_each hooks for #{self}" } + + @group.try &.call_before_each(example) + before_each_hooks.each &.call(example) end - # Number of examples in this group and all sub-groups. - def example_count : Int - @example_count + define_hook after_each : ExampleHook, :prepend do |example| + Log.trace { "Processing after_each hooks for #{self}" } + + after_each_hooks.each &.call(example) + @group.try &.call_after_each(example) end - # Retrieves an example by its index. - # This recursively searches for an example. - # - # Positive and negative indices can be used. - # Any value out of range will raise an `IndexError`. - # - # Examples are indexed as if they are in a flattened tree. - # For instance: - # ``` - # examples = [0, 1, [2, 3, 4], [5, [6, 7], 8], 9, [10]].flatten - # ``` - # The arrays symbolize groups, - # and the numbers are the index of the example in that slot. - def [](index : Int) : Example - offset = check_bounds(index) - find_nested(offset) + define_hook around_each : ExampleProcsyHook do |procsy| + Log.trace { "Processing around_each hooks for #{self}" } + + around_each_hooks.reverse_each { |hook| procsy = hook.wrap(procsy) } + if group = @group + procsy = group.call_around_each(procsy) + end + procsy end - # Checks whether an index is within acceptable bounds. - # If the index is negative, - # it will be converted to its positive equivalent. - # If the index is out of bounds, an `IndexError` is raised. - # If the index is in bounds, - # the positive index is returned. - private def check_bounds(index) - if index < 0 - raise IndexError.new if index < -example_count - index + example_count - else - raise IndexError.new if index >= example_count - index + # Creates the example group. + # The *name* describes the purpose of the group. + # It can be a `Symbol` to describe a type. + # The *location* tracks where the group exists in source code. + # This group will be assigned to the parent *group* if it is provided. + # A set of *metadata* can be used for filtering and modifying example behavior. + def initialize(@name : Label = nil, @location : Location? = nil, + @group : ExampleGroup? = nil, @metadata : Metadata = Metadata.new) + # Ensure group is linked. + group << self if group + end + + delegate size, unsafe_fetch, to: @nodes + + # Yields this group and all parent groups. + def ascend + group = self + while group + yield group + group = group.group? end end - # Finds the example with the specified index in the children. - # The *index* must be positive and within bounds (use `#check_bounds`). - private def find_nested(index) - offset = index - # Loop through each child - # until one is found to contain the index. - found = children.each do |child| - count = child.example_count - # Example groups consider their range to be [0, example_count). - # Each child is offset by the total example count of the previous children. - # The group exposes them in this way: - # 1. [0, example_count of group 1) - # 2. [example_count of group 1, example_count of group 2) - # 3. [example_count of group n, example_count of group n + 1) - # To iterate through children, the offset is tracked. - # Each iteration removes the previous child's count. - # This way the child receives the expected range. - break child if offset < count - offset -= count - end - # The remaining offset is passed along to the child. - # If it's an `Example`, it returns itself. - # Otherwise, the indexer repeats the process for the next child. - # It should be impossible to get nil here, - # provided the bounds check and example counts are correct. - found.not_nil![offset] + # Removes the specified *node* from the group. + # The node will be unassigned from this group. + def delete(node : Node) + # Only remove from the group if it is associated with this group. + return unless node.group == self + + node.group = nil + @nodes.delete(node) end - # Checks whether all examples in the group have been run. + # Checks if all examples and sub-groups have finished. def finished? : Bool - children.all?(&.finished?) + @nodes.all?(&.finished?) + end + + # Constructs the full name or description of the example group. + # This prepends names of groups this group is part of. + def to_s(io) + # Prefix with group's full name if the node belongs to a group. + return unless parent = @group + + parent.to_s(io) + name = @name + + # Add padding between the node names + # only if the names don't appear to be symbolic. + # Skip blank group names (like the root group). + io << ' ' unless !parent.name? || # ameba:disable Style/NegatedConditionsInUnless + (parent.name?.is_a?(Symbol) && name.is_a?(String) && + (name.starts_with?('#') || name.starts_with?('.'))) + + super + end + + # Adds the specified *node* to the group. + # Assigns the node to this group. + # If the node already belongs to a group, + # it will be removed from the previous group before adding it to this group. + def <<(node : Node) + # Remove from existing group if the node is part of one. + if (previous = node.group?) + previous.delete(node) + end + + # Add the node to this group and associate with it. + @nodes << node + node.group = self end end end diff --git a/src/spectator/example_group_builder.cr b/src/spectator/example_group_builder.cr new file mode 100644 index 0000000..b7fd79b --- /dev/null +++ b/src/spectator/example_group_builder.cr @@ -0,0 +1,58 @@ +require "./example_group" +require "./example_group_hook" +require "./example_hook" +require "./example_procsy_hook" +require "./hooks" +require "./label" +require "./location" +require "./metadata" +require "./node_builder" + +module Spectator + # Progressively constructs an example group. + # Hooks and builders for child nodes can be added over time to this builder. + # When done, call `#build` to produce an `ExampleGroup`. + class ExampleGroupBuilder < NodeBuilder + include Hooks + + define_hook before_all : ExampleGroupHook + define_hook after_all : ExampleGroupHook, :prepend + define_hook before_each : ExampleHook + define_hook after_each : ExampleHook, :prepend + define_hook around_each : ExampleProcsyHook + + @children = [] of NodeBuilder + + # Creates the builder. + # Initially, the builder will have no children and no hooks. + # The *name*, *location*, and *metadata* will be applied to the `ExampleGroup` produced by `#build`. + def initialize(@name : Label = nil, @location : Location? = nil, @metadata : Metadata = Metadata.new) + end + + # Constructs an example group with previously defined attributes, children, and hooks. + # The *parent* is an already constructed example group to nest the new example group under. + # It can be nil if the new example group won't have a parent. + def build(parent = nil) + ExampleGroup.new(@name, @location, parent, @metadata).tap do |group| + apply_hooks(group) + @children.each(&.build(group)) + end + end + + # Adds a child builder to the group. + # The *builder* will have `NodeBuilder#build` called on it from within `#build`. + # The new example group will be passed to it. + def <<(builder) + @children << builder + end + + # Adds all previously configured hooks to an example group. + private def apply_hooks(group) + before_all_hooks.each { |hook| group.before_all(hook) } + before_each_hooks.each { |hook| group.before_each(hook) } + after_all_hooks.reverse_each { |hook| group.after_all(hook) } + after_each_hooks.reverse_each { |hook| group.after_each(hook) } + around_each_hooks.each { |hook| group.around_each(hook) } + end + end +end diff --git a/src/spectator/example_group_hook.cr b/src/spectator/example_group_hook.cr new file mode 100644 index 0000000..bd6bac8 --- /dev/null +++ b/src/spectator/example_group_hook.cr @@ -0,0 +1,57 @@ +require "./label" +require "./location" + +module Spectator + # Information about a hook tied to an example group and a proc to invoke it. + class ExampleGroupHook + # Location of the hook in source code. + getter! location : Location + + # User-defined description of the hook. + getter! label : Label + + @proc : -> + @called = Atomic::Flag.new + + # Creates the hook with a proc. + # The *proc* will be called when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(@proc : (->), *, @location : Location? = nil, @label : Label = nil) + end + + # Creates the hook with a block. + # The block will be executed when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(*, @location : Location? = nil, @label : Label = nil, &block : -> _) + @proc = block + end + + # Invokes the hook. + def call : Nil + @called.test_and_set + @proc.call + end + + # Invokes the hook if it hasn't already been invoked. + # Returns true if the hook was invoked (first time being called). + def call_once : Bool + first = @called.test_and_set + @proc.call if first + first + end + + # Produces the string representation of the hook. + # Includes the location and label if they're not nil. + def to_s(io) + io << "example group hook" + + if (label = @label) + io << ' ' << label + end + + if (location = @location) + io << " @ " << location + end + end + end +end diff --git a/src/spectator/example_group_iteration.cr b/src/spectator/example_group_iteration.cr new file mode 100644 index 0000000..d6576d2 --- /dev/null +++ b/src/spectator/example_group_iteration.cr @@ -0,0 +1,25 @@ +require "./example_group" +require "./label" +require "./location" +require "./metadata" + +module Spectator + # Collection of examples and sub-groups for a single iteration of an iterative example group. + class ExampleGroupIteration(T) < ExampleGroup + # Item for this iteration of the example groups. + getter item : T + + # Creates the example group iteration. + # The element for the current iteration is provided by *item*. + # The *name* describes the purpose of the group. + # It can be a `Symbol` to describe a type. + # This is typically a stringified form of *item*. + # The *location* tracks where the group exists in source code. + # This group will be assigned to the parent *group* if it is provided. + # A set of *metadata* can be used for filtering and modifying example behavior. + def initialize(@item : T, name : Label = nil, location : Location? = nil, + group : ExampleGroup? = nil, metadata : Metadata = Metadata.new) + super(name, location, group, metadata) + end + end +end diff --git a/src/spectator/example_hook.cr b/src/spectator/example_hook.cr new file mode 100644 index 0000000..edebf26 --- /dev/null +++ b/src/spectator/example_hook.cr @@ -0,0 +1,52 @@ +require "./label" +require "./location" + +module Spectator + # Information about a hook tied to an example and a proc to invoke it. + class ExampleHook + # Method signature for example hooks. + alias Proc = Example -> + + # Location of the hook in source code. + getter! location : Location + + # User-defined description of the hook. + getter! label : Label + + @proc : Proc + + # Creates the hook with a proc. + # The *proc* will be called when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(@proc : Proc, *, @location : Location? = nil, @label : Label = nil) + end + + # Creates the hook with a block. + # The block must take a single argument - the current example. + # The block will be executed when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(*, @location : Location? = nil, @label : Label = nil, &block : Proc) + @proc = block + end + + # Invokes the hook. + # The *example* refers to the current example. + def call(example : Example) : Nil + @proc.call(example) + end + + # Produces the string representation of the hook. + # Includes the location and label if they're not nil. + def to_s(io) + io << "example hook" + + if (label = @label) + io << ' ' << label + end + + if (location = @location) + io << " @ " << location + end + end + end +end diff --git a/src/spectator/example_hooks.cr b/src/spectator/example_hooks.cr deleted file mode 100644 index d345bf5..0000000 --- a/src/spectator/example_hooks.cr +++ /dev/null @@ -1,75 +0,0 @@ -module Spectator - alias TestMetaMethod = ::SpectatorTest, Example -> - - # Collection of hooks that run at various times throughout testing. - # A hook is just a `Proc` (code block) that runs at a specified time. - class ExampleHooks - # Creates an empty set of hooks. - # This will effectively run nothing extra while running a test. - def self.empty - new( - [] of ->, - [] of TestMetaMethod, - [] of ->, - [] of TestMetaMethod, - [] of ::SpectatorTest, Proc(Nil) -> - ) - end - - # Creates a new set of hooks. - def initialize( - @before_all : Array(->), - @before_each : Array(TestMetaMethod), - @after_all : Array(->), - @after_each : Array(TestMetaMethod), - @around_each : Array(::SpectatorTest, Proc(Nil) ->) - ) - end - - # Runs all "before-all" hooks. - # These hooks should be run once before all examples in the group start. - def run_before_all - @before_all.each &.call - end - - # Runs all "before-each" hooks. - # These hooks should be run every time before each example in a group. - def run_before_each(wrapper : TestWrapper, example : Example) - @before_each.each do |hook| - wrapper.call(hook, example) - end - end - - # Runs all "after-all" hooks. - # These hooks should be run once after all examples in group finish. - def run_after_all - @after_all.each &.call - end - - # Runs all "after-all" hooks. - # These hooks should be run every time after each example in a group. - def run_after_each(wrapper : TestWrapper, example : Example) - @after_each.each do |hook| - wrapper.call(hook, example) - end - end - - # Creates a proc that runs the "around-each" hooks - # in addition to a block passed to this method. - # To call the block and all "around-each" hooks, - # just invoke `Proc#call` on the returned proc. - def wrap_around_each(test, block : ->) - wrapper = block - # Must wrap in reverse order, - # otherwise hooks will run in the wrong order. - @around_each.reverse_each do |hook| - wrapper = wrap_foo(test, hook, wrapper) - end - wrapper - end - - private def wrap_foo(test, hook, wrapper) - ->{ hook.call(test, wrapper) } - end - end -end diff --git a/src/spectator/example_iterator.cr b/src/spectator/example_iterator.cr index b5dd3f3..3a4ba30 100644 --- a/src/spectator/example_iterator.cr +++ b/src/spectator/example_iterator.cr @@ -1,17 +1,19 @@ +require "./example" +require "./node" + module Spectator # Iterates through all examples in a group and its nested groups. + # Nodes are iterated in pre-order. class ExampleIterator include Iterator(Example) - # Stack that contains the iterators for each group. # A stack is used to track where in the tree this iterator is. - @stack : Array(Iterator(ExampleComponent)) + @stack = Deque(Node).new(1) # Creates a new iterator. # The *group* is the example group to iterate through. - def initialize(@group : Iterable(ExampleComponent)) - iter = @group.each.as(Iterator(ExampleComponent)) - @stack = [iter] + def initialize(@group : Node) + @stack.push(@group) end # Retrieves the next `Example`. @@ -21,51 +23,30 @@ module Spectator # a. an example is found. # b. the stack is empty. until @stack.empty? - # Retrieve the next "thing". - # This could be an `Example`, - # or a group. - item = advance - # Return the item if it's an example. + # Retrieve the next node. + # This could be an `Example` or a group. + node = @stack.pop + + # If the node is a group, add its direct children to the queue + # in reverse order so that the tree is traversed in pre-order. + if node.is_a?(Indexable(Node)) + node.reverse_each { |child| @stack.push(child) } + end + + # Return the node if it's an example. # Otherwise, advance and check the next one. - return item if item.is_a?(Example) + return node if node.is_a?(Example) end + # Nothing left to iterate. stop end # Restart the iterator at the beginning. def rewind - # Same code as `#initialize`, but return self. - iter = @group.each.as(Iterator(ExampleComponent)) - @stack = [iter] + @stack.clear + @stack.push(@group) self end - - # Retrieves the top of the stack. - private def top - @stack.last - end - - # Retrieves the next "thing" from the tree. - # This method will return an `Example` or "something else." - private def advance - # Get the iterator from the top of the stack. - # Advance the iterator and check what the next item is. - case (item = top.next) - when ExampleGroup - # If the next thing is a group, - # we need to traverse its branch. - # Push its iterator onto the stack and return. - @stack.push(item.each) - when Iterator::Stop - # If a stop instance is encountered, - # then the current group is done. - # Pop its iterator from the stack and return. - @stack.pop - else - # Found an example, return it. - item - end - end end end diff --git a/src/spectator/example_pending.cr b/src/spectator/example_pending.cr new file mode 100644 index 0000000..16c0709 --- /dev/null +++ b/src/spectator/example_pending.cr @@ -0,0 +1,14 @@ +module Spectator + # Exception that indicates an example is pending and should be skipped. + # When raised within a test, the test should abort. + class ExamplePending < Exception + # Location of where the example was aborted. + getter location : Location? + + # Creates the exception. + # Specify *location* to where the example was aborted. + def initialize(@location : Location? = nil, message : String? = nil, cause : Exception? = nil) + super(message, cause) + end + end +end diff --git a/src/spectator/example_procsy_hook.cr b/src/spectator/example_procsy_hook.cr new file mode 100644 index 0000000..8a64f17 --- /dev/null +++ b/src/spectator/example_procsy_hook.cr @@ -0,0 +1,54 @@ +require "./label" +require "./location" + +module Spectator + # Information about a hook tied to an example and a proc to invoke it. + class ExampleProcsyHook + # Location of the hook in source code. + getter! location : Location + + # User-defined description of the hook. + getter! label : Label + + @proc : Example::Procsy -> + + # Creates the hook with a proc. + # The *proc* will be called when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(@proc : (Example::Procsy ->), *, @location : Location? = nil, @label : Label = nil) + end + + # Creates the hook with a block. + # The block must take a single argument - the current example wrapped in a procsy. + # The block will be executed when the hook is invoked. + # A *location* and *label* can be provided for debugging. + def initialize(*, @location : Location? = nil, @label : Label = nil, &block : Example::Procsy -> _) + @proc = block + end + + # Invokes the hook. + # The *example* refers to the current example. + def call(procsy : Example::Procsy) : Nil + @proc.call(procsy) + end + + # Creates an example procsy that invokes this hook. + def wrap(procsy : Example::Procsy) : Example::Procsy + procsy.wrap { call(procsy) } + end + + # Produces the string representation of the hook. + # Includes the location and label if they're not nil. + def to_s(io) + io << "example hook" + + if (label = @label) + io << ' ' << label + end + + if (location = @location) + io << " @ " << location + end + end + end +end diff --git a/src/spectator/expectation.cr b/src/spectator/expectation.cr new file mode 100644 index 0000000..ff9dc57 --- /dev/null +++ b/src/spectator/expectation.cr @@ -0,0 +1,187 @@ +require "json" +require "./expression" +require "./location" + +module Spectator + # Result of evaluating a matcher on a target. + # Contains information about the match, + # such as whether it was successful and a description of the operation. + struct Expectation + # Location of the expectation in source code. + # This can be nil if the location isn't capturable, + # for instance using the *should* syntax or dynamically created expectations. + getter! location : Location + + # Indicates whether the expectation was met. + def satisfied? + @match_data.matched? + end + + # Indicates whether the expectation was not met. + def failed? + !satisfied? + end + + # If nil, then the match was successful. + def failure_message? + return unless match_data = @match_data.as?(Matchers::FailedMatchData) + + case message = @message + when String then message + when Proc(String) then @message = message.call # Cache result of call. + else match_data.failure_message + end + end + + # Description of why the match failed. + def failure_message + failure_message?.not_nil! + end + + # Additional information about the match, useful for debug. + # If nil, then the match was successful. + def values? + @match_data.as?(Matchers::FailedMatchData).try(&.values) + end + + # Additional information about the match, useful for debug. + def values + values?.not_nil! + end + + def description + @match_data.description + end + + # Creates the expectation. + # The *match_data* comes from the result of calling `Matcher#match`. + # The *location* is the location of the expectation in source code, if available. + # A custom *message* can be used in case of a failure. + def initialize(@match_data : Matchers::MatchData, @location : Location? = nil, + @message : String? | Proc(String) = nil) + end + + # Creates the JSON representation of the expectation. + def to_json(json : JSON::Builder) + json.object do + if location = @location + json.field("file_path", location.path) + json.field("line_number", location.line) + end + json.field("satisfied", satisfied?) + if (failed = @match_data.as?(Matchers::FailedMatchData)) + failed_to_json(failed, json) + end + end + end + + # Adds failure information to a JSON structure. + private def failed_to_json(failed : Matchers::FailedMatchData, json : JSON::Builder) + json.field("failure", failed.failure_message) + json.field("values") do + json.object do + failed.values.each do |pair| + json.field(pair.first, pair.last) + end + end + end + end + + # Stores part of an expectation. + # This covers the actual value (or block) being inspected and its location. + # This is the type returned by an `expect` block in the DSL. + # It is not intended to be used directly, but instead by chaining methods. + # Typically `#to` and `#not_to` are used. + struct Target(T) + # Creates the expectation target. + # The *expression* is the actual value being tested and its label. + # The *location* is the location of where this expectation was defined. + def initialize(@expression : Expression(T), @location : Location) + end + + # Asserts that some criteria defined by the matcher is satisfied. + # Allows a custom message to be used. + def to(matcher, message = nil) : Nil + match_data = matcher.match(@expression) + report(match_data, message) + end + + def to(stub : Mocks::MethodStub) : Nil + Harness.current.mocks.expect(@expression.value, stub) + value = Value.new(stub.name, stub.to_s) + matcher = Matchers::ReceiveMatcher.new(value, stub.arguments?) + to_eventually(matcher) + end + + def to(stubs : Enumerable(Mocks::MethodStub)) : Nil + stubs.each { |stub| to(stub) } + end + + # Asserts that some criteria defined by the matcher is not satisfied. + # This is effectively the opposite of `#to`. + # Allows a custom message to be used. + def to_not(matcher, message = nil) : Nil + match_data = matcher.negated_match(@expression) + report(match_data, message) + end + + # :ditto: + @[AlwaysInline] + def not_to(matcher, message = nil) : Nil + to_not(matcher, message) + end + + def to_not(stub : Mocks::MethodStub) : Nil + value = Value.new(stub.name, stub.to_s) + matcher = Matchers::ReceiveMatcher.new(value, stub.arguments?) + to_never(matcher) + end + + def to_not(stubs : Enumerable(Mocks::MethodStub)) : Nil + stubs.each { |stub| to_not(stub) } + end + + # Asserts that some criteria defined by the matcher is eventually satisfied. + # The expectation is checked after the example finishes and all hooks have run. + # Allows a custom message to be used. + def to_eventually(matcher, message = nil) : Nil + Harness.current.defer { to(matcher, message) } + end + + def to_eventually(stub : Mocks::MethodStub) : Nil + to(stub) + end + + def to_eventually(stubs : Enumerable(Mocks::MethodStub)) : Nil + to(stub) + end + + # Asserts that some criteria defined by the matcher is never satisfied. + # The expectation is checked after the example finishes and all hooks have run. + # Allows a custom message to be used. + def to_never(matcher, message = nil) : Nil + Harness.current.defer { to_not(matcher, message) } + end + + # :ditto: + @[AlwaysInline] + def never_to(matcher, message = nil) : Nil + to_never(matcher, message) + end + + def to_never(stub : Mocks::MethodStub) : Nil + to_not(stub) + end + + def to_never(stub : Enumerable(Mocks::MethodStub)) : Nil + to_not(stub) + end + + # Reports an expectation to the current harness. + private def report(match_data : Matchers::MatchData, message : String? | Proc(String) = nil) + expectation = Expectation.new(match_data, @location, message) + Harness.current.report(expectation) + end + end + end +end diff --git a/src/spectator/expectation_failed.cr b/src/spectator/expectation_failed.cr index 05246da..2fe3941 100644 --- a/src/spectator/expectation_failed.cr +++ b/src/spectator/expectation_failed.cr @@ -1,15 +1,16 @@ require "./example_failed" +require "./expectation" module Spectator - # Exception that indicates a required expectation was not met in an example. + # Exception that indicates an expectation from a test failed. + # When raised within a test, the test should abort. class ExpectationFailed < ExampleFailed # Expectation that failed. - getter expectation : Expectations::Expectation + getter expectation : Expectation # Creates the exception. - # The exception string is generated from the expecation message. - def initialize(@expectation) - super(@expectation.failure_message) + def initialize(@expectation : Expectation, message : String? = nil, cause : Exception? = nil) + super(expectation.location?, message, cause) end end end diff --git a/src/spectator/expectations.cr b/src/spectator/expectations.cr deleted file mode 100644 index b835f10..0000000 --- a/src/spectator/expectations.cr +++ /dev/null @@ -1,7 +0,0 @@ -require "./expectations/*" - -module Spectator - # Namespace that contains all expectations, partials, and handling of them. - module Expectations - end -end diff --git a/src/spectator/expectations/example_expectations.cr b/src/spectator/expectations/example_expectations.cr deleted file mode 100644 index 31f1f98..0000000 --- a/src/spectator/expectations/example_expectations.cr +++ /dev/null @@ -1,62 +0,0 @@ -require "./expectation" - -module Spectator::Expectations - # Collection of expectations from an example. - class ExampleExpectations - include Enumerable(Expectation) - - # Creates the collection. - def initialize(@expectations : Array(Expectation)) - end - - # Iterates through all expectations. - def each - @expectations.each do |expectation| - yield expectation - end - end - - # Returns a collection of only the satisfied expectations. - def satisfied : Enumerable(Expectation) - @expectations.select(&.satisfied?) - end - - # Iterates over only the satisfied expectations. - def each_satisfied - @expectations.each do |expectation| - yield expectation if expectation.satisfied? - end - end - - # Returns a collection of only the unsatisfied expectations. - def unsatisfied : Enumerable(Expectation) - @expectations.reject(&.satisfied?) - end - - # Iterates over only the unsatisfied expectations. - def each_unsatisfied - @expectations.each do |expectation| - yield expectation unless expectation.satisfied? - end - end - - # Determines whether the example was successful - # based on if all expectations were satisfied. - def successful? - @expectations.all?(&.satisfied?) - end - - # Determines whether the example failed - # based on if any expectations were not satisfied. - def failed? - !successful? - end - - # Creates the JSON representation of the expectations. - def to_json(json : ::JSON::Builder) - json.array do - each &.to_json(json) - end - end - end -end diff --git a/src/spectator/expectations/expectation.cr b/src/spectator/expectations/expectation.cr deleted file mode 100644 index 72d3dbd..0000000 --- a/src/spectator/expectations/expectation.cr +++ /dev/null @@ -1,74 +0,0 @@ -require "../matchers/failed_match_data" -require "../matchers/match_data" -require "../source" - -module Spectator::Expectations - # Result of evaluating a matcher on an expectation partial. - struct Expectation - # Location where this expectation was defined. - getter source : Source - - # Creates the expectation. - def initialize(@match_data : Matchers::MatchData, @source : Source) - end - - # Indicates whether the matcher was satisified. - def satisfied? - @match_data.matched? - end - - # Indicates that the expectation was not satisified. - def failure? - !satisfied? - end - - # Description of why the match failed. - # If nil, then the match was successful. - def failure_message? - @match_data.as?(Matchers::FailedMatchData).try(&.failure_message) - end - - # Description of why the match failed. - def failure_message - failure_message?.not_nil! - end - - # Additional information about the match, useful for debug. - # If nil, then the match was successful. - def values? - @match_data.as?(Matchers::FailedMatchData).try(&.values) - end - - # Additional information about the match, useful for debug. - def values - values?.not_nil! - end - - def description - @match_data.description - end - - # Creates the JSON representation of the expectation. - def to_json(json : ::JSON::Builder) - json.object do - json.field("source") { @source.to_json(json) } - json.field("satisfied", satisfied?) - if (failed = @match_data.as?(Matchers::FailedMatchData)) - failed_to_json(failed, json) - end - end - end - - # Adds failure information to a JSON structure. - private def failed_to_json(failed : Matchers::FailedMatchData, json : ::JSON::Builder) - json.field("failure", failed.failure_message) - json.field("values") do - json.object do - failed.values.each do |pair| - json.field(pair.first, pair.last) - end - end - end - end - end -end diff --git a/src/spectator/expectations/expectation_partial.cr b/src/spectator/expectations/expectation_partial.cr deleted file mode 100644 index c837c86..0000000 --- a/src/spectator/expectations/expectation_partial.cr +++ /dev/null @@ -1,101 +0,0 @@ -require "../matchers/match_data" -require "../source" -require "../test_expression" - -module Spectator::Expectations - # Stores part of an expectation (obviously). - # The part of the expectation this type covers is the actual value and source. - # This can also cover a block's behavior. - struct ExpectationPartial(T) - # The actual value being tested. - # This also contains its label. - getter actual : TestExpression(T) - - # Location where this expectation was defined. - getter source : Source - - # Creates the partial. - def initialize(@actual : TestExpression(T), @source : Source) - end - - # Asserts that some criteria defined by the matcher is satisfied. - def to(matcher) : Nil - match_data = matcher.match(@actual) - report(match_data) - end - - def to(stub : Mocks::MethodStub) : Nil - Harness.current.mocks.expect(@actual.value, stub) - value = TestValue.new(stub.name, stub.to_s) - matcher = Matchers::ReceiveMatcher.new(value, stub.arguments?) - to_eventually(matcher) - end - - def to(stubs : Enumerable(Mocks::MethodStub)) : Nil - stubs.each { |stub| to(stub) } - end - - # Asserts that some criteria defined by the matcher is not satisfied. - # This is effectively the opposite of `#to`. - def to_not(matcher) : Nil - match_data = matcher.negated_match(@actual) - report(match_data) - end - - def to_not(stub : Mocks::MethodStub) : Nil - value = TestValue.new(stub.name, stub.to_s) - matcher = Matchers::ReceiveMatcher.new(value, stub.arguments?) - to_never(matcher) - end - - def to_not(stubs : Enumerable(Mocks::MethodStub)) : Nil - stubs.each { |stub| to_not(stub) } - end - - # :ditto: - @[AlwaysInline] - def not_to(matcher) : Nil - to_not(matcher) - end - - # Asserts that some criteria defined by the matcher is eventually satisfied. - # The expectation is checked after the example finishes and all hooks have run. - def to_eventually(matcher) : Nil - Harness.current.defer { to(matcher) } - end - - def to_eventually(stub : Mocks::MethodStub) : Nil - to(stub) - end - - def to_eventually(stubs : Enumerable(Mocks::MethodStub)) : Nil - to(stub) - end - - # Asserts that some criteria defined by the matcher is never satisfied. - # The expectation is checked after the example finishes and all hooks have run. - def to_never(matcher) : Nil - Harness.current.defer { to_not(matcher) } - end - - def to_never(stub : Mocks::MethodStub) : Nil - to_not(stub) - end - - def to_never(stub : Enumerable(Mocks::MethodStub)) : Nil - to_not(stub) - end - - # :ditto: - @[AlwaysInline] - def never_to(matcher) : Nil - to_never(matcher) - end - - # Reports an expectation to the current harness. - private def report(match_data : Matchers::MatchData) - expectation = Expectation.new(match_data, @source) - Harness.current.report_expectation(expectation) - end - end -end diff --git a/src/spectator/expectations/expectation_reporter.cr b/src/spectator/expectations/expectation_reporter.cr deleted file mode 100644 index 2830b8e..0000000 --- a/src/spectator/expectations/expectation_reporter.cr +++ /dev/null @@ -1,34 +0,0 @@ -module Spectator::Expectations - # Tracks the expectations and their outcomes in an example. - # A single instance of this class should be associated with one example. - class ExpectationReporter - # All expectations are stored in this array. - # The initial capacity is set to one, - # as that is the typical (and recommended) - # number of expectations per example. - @expectations = Array(Expectation).new(1) - - # Creates the reporter. - # When the *raise_on_failure* flag is set to true, - # which is the default, an exception will be raised - # on the first failure that is reported. - # To store failures and continue, set the flag to false. - def initialize(@raise_on_failure = true) - end - - # Stores the outcome of an expectation. - # If the raise on failure flag is set to true, - # then this method will raise an exception - # when a failing result is given. - def report(expectation : Expectation) : Nil - @expectations << expectation - raise ExpectationFailed.new(expectation) if !expectation.satisfied? && @raise_on_failure - end - - # Returns the reported expectations from the example. - # This should be run after the example has finished. - def expectations : ExampleExpectations - ExampleExpectations.new(@expectations) - end - end -end diff --git a/src/spectator/expression.cr b/src/spectator/expression.cr new file mode 100644 index 0000000..0937b8e --- /dev/null +++ b/src/spectator/expression.cr @@ -0,0 +1,18 @@ +require "./abstract_expression" + +module Spectator + # Represents an expression from a test. + # This is typically captured by an `expect` macro. + # It consists of a label and a typed expression. + # The label should be a string recognizable by the user, + # or nil if one isn't available. + abstract class Expression(T) < AbstractExpression + # Retrieves the underlying value of the expression. + abstract def value : T + + # Retrieves the evaluated value of the expression. + def raw_value + value + end + end +end diff --git a/src/spectator/fail_result.cr b/src/spectator/fail_result.cr new file mode 100644 index 0000000..36ea8fb --- /dev/null +++ b/src/spectator/fail_result.cr @@ -0,0 +1,81 @@ +require "json" +require "./example_failed" +require "./location" +require "./result" + +module Spectator + # Outcome that indicates an example failed. + # This typically means an assertion did not pass. + class FailResult < Result + # Error that occurred while running the example. + # This describes the primary reason for the failure. + getter error : Exception + + # Creates a failure result. + # The *elapsed* argument is the length of time it took to run the example. + # The *error* is the exception raised that caused the failure. + def initialize(elapsed, @error, expectations = [] of Expectation) + super(elapsed, expectations) + end + + # Calls the `failure` method on *visitor*. + def accept(visitor) + visitor.fail(self) + end + + # Calls the `failure` method on *visitor*. + def accept(visitor) + visitor.fail(yield self) + end + + # Indicates whether the example passed. + def pass? : Bool + false + end + + # Indicates whether the example failed. + def fail? : Bool + true + end + + # Attempts to retrieve the location where the example failed. + # This only works if the location of the failed expectation was reported. + # If available, returns a `Location`, otherwise `nil`. + def location? : Location? + return unless error = @error.as?(ExampleFailed) + + error.location? + end + + # Attempts to retrieve the location where the example failed. + # This only works if the location of the failed expectation was reported. + # If available, returns a `Location`, otherwise raises `NilAssertionError`. + def location : Location + location? || raise(NilAssertionError.new("Source location of failure unavailable")) + end + + # One-word description of the result. + def to_s(io) + io << "fail" + end + + # Creates a JSON object from the result information. + def to_json(json : JSON::Builder) + super + json.field("status", json_status) + json.field("exception") do + json.object do + json.field("class", @error.class.name) + json.field("message", @error.message) + json.field("backtrace", @error.backtrace) + end + end + end + + # String used for the JSON status field. + # Necessary for the error result to override the status, but nothing else from `#to_json`. + private def json_status + "failed" + end + end +end diff --git a/src/spectator/failed_result.cr b/src/spectator/failed_result.cr deleted file mode 100644 index 65f5c4e..0000000 --- a/src/spectator/failed_result.cr +++ /dev/null @@ -1,42 +0,0 @@ -require "./result" - -module Spectator - # Outcome that indicates running an example was a failure. - class FailedResult < FinishedResult - # Error that occurred while running the example. - getter error : Exception - - # Creates a failed result. - # The *example* should refer to the example that was run - # and that this result is for. - # The *elapsed* argument is the length of time it took to run the example. - # The *expectations* references the expectations that were checked in the example. - # The *error* is the exception that was raised to cause the failure. - def initialize(example, elapsed, expectations, @error) - super(example, elapsed, expectations) - end - - # Calls the `failure` method on *interface*. - def call(interface) - interface.failure - end - - # Calls the `failure` method on *interface* - # and passes the yielded value. - def call(interface) - value = yield self - interface.failure(value) - end - - # One-word descriptor of the result. - def to_s(io) - io << "fail" - end - - # Adds all of the JSON fields for finished results and failed results. - private def add_json_fields(json : ::JSON::Builder) - super - json.field("error", error.message) - end - end -end diff --git a/src/spectator/filtered_example_iterator.cr b/src/spectator/filtered_example_iterator.cr new file mode 100644 index 0000000..3286f3b --- /dev/null +++ b/src/spectator/filtered_example_iterator.cr @@ -0,0 +1,85 @@ +require "./example" +require "./node" +require "./node_filter" +require "./node_iterator" + +module Spectator + # Iterates through selected nodes in a group and its nested groups. + # Nodes are iterated in pre-order. + class FilteredExampleIterator + include Iterator(Example) + + # A stack is used to track where in the tree this iterator is. + @stack = Deque(Node).new(1) + + # A queue stores forced examples that have been matched by the a parent group. + @queue = Deque(Example).new + + # Creates a new iterator. + # The *group* is the example group to iterate through. + # The *filter* selects which examples (and groups) to iterate through. + def initialize(@group : Node, @filter : NodeFilter) + @stack.push(@group) + end + + # Retrieves the next selected `Example`. + # If there are none left, then `Iterator::Stop` is returned. + def next + # Return items from the queue first before continuing to the stack. + return @queue.shift unless @queue.empty? + + # Keep going until either: + # a. a suitable example is found. + # b. the stack is empty. + until @stack.empty? + # Retrieve the next node. + node = @stack.pop + + # If the node is a group, conditionally traverse it. + if node.is_a?(Indexable(Node)) + # To traverse, a child node or the group itself must match the filter. + return node if node = next_group_match(node) + elsif node.is_a?(Example) && @filter.includes?(node) + return node + end + end + + # Nothing left to iterate. + stop + end + + # Restart the iterator at the beginning. + def rewind + @stack.clear + @stack.push(@group) + @queue.clear + self + end + + # Attempts to find the next matching example in a group. + # If any child in the group matches, then traversal on the stack (down the tree) continues. + # However, if no children match, but the group itself does, then all examples in the group match. + # In the latter scenario, the examples are added to the queue, and the next item from the queue returned. + # Stack iteration should continue if nil is returned. + private def next_group_match(group : Indexable(Node)) : Example? + # Look for any children that match. + iterator = NodeIterator.new(group) + + # Check if any children match. + # Skip first node because its the group being checked. + if iterator.skip(1).any?(@filter) + # Add the group's direct children to the queue + # in reverse order so that the tree is traversed in pre-order. + group.reverse_each { |node| @stack.push(node) } + + # Check if the group matches, but no children match. + elsif @filter.includes?(group) + # Add examples from the group to the queue. + # Return the next example from the queue. + iterator.rewind.select(Example).each { |node| @queue.push(node) } + @queue.shift unless @queue.empty? + # If the queue is empty (group has no examples), go to next loop iteration of the stack. + end + end + end +end diff --git a/src/spectator/finished_result.cr b/src/spectator/finished_result.cr deleted file mode 100644 index 87ec298..0000000 --- a/src/spectator/finished_result.cr +++ /dev/null @@ -1,27 +0,0 @@ -module Spectator - # Abstract class for all results by examples - abstract class FinishedResult < Result - # Length of time it took to run the example. - getter elapsed : Time::Span - - # The expectations that were run in the example. - getter expectations : Expectations::ExampleExpectations - - # Creates a successful result. - # The *example* should refer to the example that was run - # and that this result is for. - # The *elapsed* argument is the length of time it took to run the example. - # The *expectations* references the expectations that were checked in the example. - def initialize(example, @elapsed, @expectations) - super(example) - end - - # Adds the common JSON fields for all result types - # and fields specific to finished results. - private def add_json_fields(json : ::JSON::Builder) - super - json.field("time", elapsed.total_seconds) - json.field("expectations", expectations) - end - end -end diff --git a/src/spectator/formatting/broadcast_formatter.cr b/src/spectator/formatting/broadcast_formatter.cr new file mode 100644 index 0000000..76ac814 --- /dev/null +++ b/src/spectator/formatting/broadcast_formatter.cr @@ -0,0 +1,87 @@ +require "./formatter" + +module Spectator::Formatting + # Reports events to multiple other formatters. + # Events received by this formatter will be sent to others. + class BroadcastFormatter < Formatter + # Creates the broadcast formatter. + # Takes a collection of formatters to pass events along to. + def initialize(@formatters : Enumerable(Formatter)) + end + + # Forwards the event to other formatters. + def start(notification) + @formatters.each(&.start(notification)) + end + + # :ditto: + def example_started(notification) + @formatters.each(&.example_started(notification)) + end + + # :ditto: + def example_finished(notification) + @formatters.each(&.example_finished(notification)) + end + + # :ditto: + def example_passed(notification) + @formatters.each(&.example_passed(notification)) + end + + # :ditto: + def example_pending(notification) + @formatters.each(&.example_pending(notification)) + end + + # :ditto: + def example_failed(notification) + @formatters.each(&.example_failed(notification)) + end + + # :ditto: + def example_error(notification) + @formatters.each(&.example_error(notification)) + end + + # :ditto: + def message(notification) + @formatters.each(&.message(notification)) + end + + # :ditto: + def stop + @formatters.each(&.stop) + end + + # :ditto: + def start_dump + @formatters.each(&.start_dump) + end + + # :ditto: + def dump_pending(notification) + @formatters.each(&.dump_pending(notification)) + end + + # :ditto: + def dump_failures(notification) + @formatters.each(&.dump_failures(notification)) + end + + # :ditto: + def dump_summary(notification) + @formatters.each(&.dump_summary(notification)) + end + + # :ditto: + def dump_profile(notification) + @formatters.each(&.dump_profile(notification)) + end + + # :ditto: + def close + @formatters.each(&.close) + end + end +end diff --git a/src/spectator/formatting/color.cr b/src/spectator/formatting/color.cr deleted file mode 100644 index ed02ddc..0000000 --- a/src/spectator/formatting/color.cr +++ /dev/null @@ -1,42 +0,0 @@ -require "colorize" - -module Spectator::Formatting - # Method for colorizing output. - module Color - extend self - - # Symbols in `Colorize` representing result types and formatting types. - private COLORS = { - success: :green, - failure: :red, - error: :magenta, - pending: :yellow, - comment: :cyan, - } - - # Colorizes some text with the success color. - def success(text) - text.colorize(COLORS[:success]) - end - - # Colorizes some text with the failure color. - def failure(text) - text.colorize(COLORS[:failure]) - end - - # Colorizes some text with the error color. - def error(text) - text.colorize(COLORS[:error]) - end - - # Colorizes some text with the pending/skipped color. - def pending(text) - text.colorize(COLORS[:pending]) - end - - # Colorizes some text with the comment color. - def comment(text) - text.colorize(COLORS[:comment]) - end - end -end diff --git a/src/spectator/formatting/comment.cr b/src/spectator/formatting/comment.cr deleted file mode 100644 index 6a42685..0000000 --- a/src/spectator/formatting/comment.cr +++ /dev/null @@ -1,20 +0,0 @@ -module Spectator::Formatting - # Produces a stringified comment for output. - private struct Comment(T) - # Creates the comment. - def initialize(@text : T) - end - - # Appends the comment to the output. - def to_s(io) - io << '#' - io << ' ' - io << @text - end - - # Creates a colorized version of the comment. - def self.color(text) - Color.comment(new(text)) - end - end -end diff --git a/src/spectator/formatting/components.cr b/src/spectator/formatting/components.cr new file mode 100644 index 0000000..9d888f3 --- /dev/null +++ b/src/spectator/formatting/components.cr @@ -0,0 +1,8 @@ +require "./components/**" + +module Spectator::Formatting + # Namespace for snippets of text displayed in console output. + # These types are typically constructed and have `#to_s` called. + module Components + end +end diff --git a/src/spectator/formatting/components/block.cr b/src/spectator/formatting/components/block.cr new file mode 100644 index 0000000..40cd5a8 --- /dev/null +++ b/src/spectator/formatting/components/block.cr @@ -0,0 +1,32 @@ +module Spectator::Formatting::Components + # Base type for handling indented output. + # Indents are tracked and automatically printed. + # Use `#indent` to increase the indent for the duration of a block. + # Use `#line` to produce a line with an indentation prefixing it. + abstract struct Block + # Default indent amount. + private INDENT = 2 + + # Creates the block. + # A default *indent* size can be specified. + def initialize(*, @indent : Int32 = INDENT) + end + + # Increases the indent by the a specific *amount* for the duration of the block. + private def indent(amount = INDENT) + @indent += amount + yield + @indent -= amount + end + + # Produces a line of output with an indent before it. + # The contents of the line should be generated by a block provided to this method. + # Ensure that _only_ one line is produced by the block, + # otherwise the indent will be lost. + private def line(io) + @indent.times { io << ' ' } + yield + io.puts + end + end +end diff --git a/src/spectator/formatting/components/comment.cr b/src/spectator/formatting/components/comment.cr new file mode 100644 index 0000000..b398840 --- /dev/null +++ b/src/spectator/formatting/components/comment.cr @@ -0,0 +1,23 @@ +require "colorize" + +module Spectator::Formatting::Components + # Object that can be stringified pre-pended with a comment mark (#). + struct Comment(T) + # Default color for a comment. + private COLOR = :cyan + + # Creates a comment with the specified content. + def initialize(@content : T) + end + + # Creates a colored comment. + def self.colorize(content) + new(content).colorize(COLOR) + end + + # Writes the comment to the output. + def to_s(io) + io << "# " << @content + end + end +end diff --git a/src/spectator/formatting/components/error_result_block.cr b/src/spectator/formatting/components/error_result_block.cr new file mode 100644 index 0000000..353c096 --- /dev/null +++ b/src/spectator/formatting/components/error_result_block.cr @@ -0,0 +1,81 @@ +require "colorize" +require "../../example" +require "../../error_result" +require "./result_block" + +module Spectator::Formatting::Components + # Displays information about an error result. + struct ErrorResultBlock < ResultBlock + # Creates the component. + def initialize(example : Example, index : Int32, @result : ErrorResult, subindex = 0) + super(example, index, subindex) + end + + # Content displayed on the second line of the block after the label. + private def subtitle + @result.error.message.try(&.each_line.first) + end + + # Prefix for the second line of the block. + private def subtitle_label + "Error: ".colorize(:red) + end + + # Display error information. + private def content(io) + # Fetch the error and message. + error = @result.error + lines = error.message.try(&.lines) + + # Write the error and message if available. + case + when lines.nil? then write_error_class(io, error) + when lines.size == 1 then write_error_message(io, error, lines.first) + when lines.size > 1 then write_multiline_error_message(io, error, lines) + else write_error_class(io, error) + end + + # Display the backtrace if it's available. + if backtrace = error.backtrace? + indent { write_backtrace(io, backtrace) } + end + + io.puts + end + + # Display just the error type. + private def write_error_class(io, error) + line(io) do + io << error.class.colorize(:red) + end + end + + # Display the error type and first line of the message. + private def write_error_message(io, error, message) + line(io) do + io << "#{error.class}: ".colorize(:red) + io << message + end + end + + # Display the error type and its multi-line message. + private def write_multiline_error_message(io, error, lines) + # Use the normal formatting for the first line. + write_error_message(io, error, lines.first) + + # Display additional lines after the first. + lines.skip(1).each do |entry| + line(io) { io << entry } + end + end + + # Writes the backtrace entries to the output. + private def write_backtrace(io, backtrace) + backtrace.each do |entry| + # Dim entries that are outside the shard. + entry = entry.colorize.dim unless entry.starts_with?(/(src|spec)\//) + line(io) { io << entry } + end + end + end +end diff --git a/src/spectator/formatting/components/example_command.cr b/src/spectator/formatting/components/example_command.cr new file mode 100644 index 0000000..8b3c0b9 --- /dev/null +++ b/src/spectator/formatting/components/example_command.cr @@ -0,0 +1,26 @@ +require "../../example" +require "./comment" + +module Spectator::Formatting::Components + # Provides syntax for running a specific example from the command-line. + struct ExampleCommand + # Creates the component with the specified example. + def initialize(@example : Example) + end + + # Produces output for running the previously specified example. + def to_s(io) + io << "crystal spec " + + # Use location for argument if it's available, since it's simpler. + # Otherwise, use the example name filter argument. + if location = @example.location? + io << location + else + io << "-e " << @example + end + + io << ' ' << Comment.colorize(@example.to_s) + end + end +end diff --git a/src/spectator/formatting/components/fail_result_block.cr b/src/spectator/formatting/components/fail_result_block.cr new file mode 100644 index 0000000..74b2f60 --- /dev/null +++ b/src/spectator/formatting/components/fail_result_block.cr @@ -0,0 +1,49 @@ +require "colorize" +require "../../example" +require "../../expectation" +require "../../fail_result" +require "./result_block" + +module Spectator::Formatting::Components + # Displays information about a fail result. + struct FailResultBlock < ResultBlock + @longest_key : Int32 + + # Creates the component. + def initialize(example : Example, index : Int32, @expectation : Expectation, subindex = 0) + super(example, index, subindex) + @longest_key = expectation.values.max_of { |(key, _value)| key.to_s.size } + end + + # Content displayed on the second line of the block after the label. + private def subtitle + @expectation.failure_message + end + + # Prefix for the second line of the block. + private def subtitle_label + "Failure: ".colorize(:red) + end + + # Display expectation match data. + private def content(io) + indent do + @expectation.values.each do |(key, value)| + value_line(io, key, value) + end + end + + io.puts + end + + # Display a single line for a match data value. + private def value_line(io, key, value) + key = key.to_s + padding = " " * (@longest_key - key.size) + + line(io) do + io << padding << key.colorize(:red) << ": ".colorize(:red) << value + end + end + end +end diff --git a/src/spectator/formatting/components/failure_command_list.cr b/src/spectator/formatting/components/failure_command_list.cr new file mode 100644 index 0000000..7da5ed2 --- /dev/null +++ b/src/spectator/formatting/components/failure_command_list.cr @@ -0,0 +1,21 @@ +require "../../example" +require "./example_command" + +module Spectator::Formatting::Components + # Produces a list of commands to run failed examples. + struct FailureCommandList + # Creates the component. + # Requires a set of *failures* to display commands for. + def initialize(@failures : Enumerable(Example)) + end + + # Produces the list of commands to run failed examples. + def to_s(io) + io.puts "Failed examples:" + io.puts + @failures.each do |failure| + io.puts ExampleCommand.new(failure).colorize(:red) + end + end + end +end diff --git a/src/spectator/formatting/components/junit/root.cr b/src/spectator/formatting/components/junit/root.cr new file mode 100644 index 0000000..3c6f440 --- /dev/null +++ b/src/spectator/formatting/components/junit/root.cr @@ -0,0 +1,39 @@ +require "./test_suite" + +module Spectator::Formatting::Components::JUnit + # Root node of the JUnit XML document. + # This is the "testsuites" element and all of its children. + struct Root + # Creates the root element. + def initialize(@runtime : Time::Span, @suites : Array(TestSuite), *, + @total : Int32, @failures : Int32, @errors : Int32) + end + + # Constructs the element from a report. + def self.from_report(report) + hostname = System.hostname + counts = report.counts + suites = report.examples.group_by { |example| example.location?.try(&.path) || "anonymous" } + suites = suites.map do |file, examples| + TestSuite.from_examples(file, examples, hostname) + end + + new(report.runtime, suites, + total: counts.total, + failures: counts.fail, + errors: counts.error) + end + + # Produces the XML fragment. + def to_xml(xml) + xml.element("testsuites", + tests: @total, + failures: @failures, + errors: @errors, + time: @runtime.total_seconds, + name: "Spec") do + @suites.each(&.to_xml(xml)) + end + end + end +end diff --git a/src/spectator/formatting/components/junit/test_case.cr b/src/spectator/formatting/components/junit/test_case.cr new file mode 100644 index 0000000..796c97f --- /dev/null +++ b/src/spectator/formatting/components/junit/test_case.cr @@ -0,0 +1,102 @@ +require "xml" +require "../../../example" + +module Spectator::Formatting::Components::JUnit + # Test case node of the JUnit XML document. + # This is the "testsuite" element and all of its children. + struct TestCase + # Creates the test case element. + def initialize(@class_name : String, @example : Example) + end + + # Produces the XML fragment. + def to_xml(xml) + result = @example.result + xml.element("testcase", + name: @example, + assertions: result.expectations.size, + classname: @class_name, + status: result.accept(StatusVisitor), + time: result.elapsed.total_seconds) do + visitor = ElementVisitor.new(xml) + result.accept(visitor) + end + end + + # Picks the status string for a result. + private module StatusVisitor + extend self + + # Returns "PASS". + def pass(_result) + "PASS" + end + + # Returns "FAIL". + def fail(_result) + "FAIL" + end + + # :ditto: + def error(result) + fail(result) + end + + # Returns "TODO". + def pending(_result) + "TODO" + end + end + + # Result visitor that adds elements to the test case node depending on the result. + private struct ElementVisitor + # Creates the visitor. + def initialize(@xml : XML::Builder) + end + + # Does nothing. + def pass(_result) + # ... + end + + # Adds a failure element to the test case node. + def fail(result) + error = result.error + result.expectations.each do |expectation| + next unless expectation.failed? + + @xml.element("failure", message: expectation.failure_message, type: error.class) do + match_data(expectation.values) + end + end + end + + # Adds an error element to the test case node. + def error(result) + error = result.error + fail(result) # Include failed expectations. + @xml.element("error", message: error.message, type: error.class) do + if backtrace = error.backtrace + @xml.text(backtrace.join("\n")) + end + end + end + + # Adds a skipped element to the test case node. + def pending(result) + @xml.element("skipped", message: result.reason) + end + + # Writes match data for a failed expectation. + private def match_data(values) + values.each do |(key, value)| + @xml.text("\n") + @xml.text(key.to_s) + @xml.text(": ") + @xml.text(value) + end + @xml.text("\n") + end + end + end +end diff --git a/src/spectator/formatting/components/junit/test_suite.cr b/src/spectator/formatting/components/junit/test_suite.cr new file mode 100644 index 0000000..28b5adc --- /dev/null +++ b/src/spectator/formatting/components/junit/test_suite.cr @@ -0,0 +1,104 @@ +require "./test_case" + +module Spectator::Formatting::Components::JUnit + # Test suite node of the JUnit XML document. + # This is the "testsuite" element and all of its children. + struct TestSuite + # Amounts for each type of test result. + record Counts, total : Int32, failures : Int32, errors : Int32, skipped : Int32 + + # Creates the test suite element. + def initialize(@package : String, @name : String, @cases : Array(TestCase), + @time : Time::Span, @counts : Counts, @hostname : String) + end + + # Constructs the test suite element from a collection of tests. + # The *examples* should all come from the same *file*. + def self.from_examples(file, examples, hostname) + package, name = package_name_from_file(file) + counts = count_examples(examples) + time = examples.sum(&.result.elapsed) + cases = examples.map { |example| TestCase.new(name, example) } + new(package, name, cases, time, counts, hostname) + end + + # Constructs a package and suite name from a file path. + private def self.package_name_from_file(file) + path = Path.new(file.to_s) + name = path.stem + directory = path.dirname + package = directory.gsub(File::SEPARATOR, '.') + {package, name} + end + + # Counts the number of examples for each result type. + private def self.count_examples(examples) + visitor = CountVisitor.new + + # Iterate through each example and count the number of each type of result. + # Don't count examples that haven't run (indicated by `Node#finished?`). + # This typically happens in fail-fast mode. + examples.each do |example| + example.result.accept(visitor) if example.finished? + end + + visitor.counts + end + + # Produces the XML fragment. + def to_xml(xml) + xml.element("testsuite", + package: @package, + name: @name, + tests: @counts.total, + failures: @counts.failures, + errors: @counts.errors, + skipped: @counts.skipped, + time: @time.total_seconds, + hostname: @hostname) do + @cases.each(&.to_xml(xml)) + end + end + + # Totals up the number of each type of result. + # Defines methods for the different types of results. + # Call `#counts` to retrieve the `Counts` instance. + private class CountVisitor + @pass = 0 + @failures = 0 + @errors = 0 + @skipped = 0 + + # Increments the number of passing examples. + def pass(_result) + @pass += 1 + end + + # Increments the number of failing (non-error) examples. + def fail(_result) + @failures += 1 + end + + # Increments the number of error (and failed) examples. + def error(result) + fail(result) + @errors += 1 + end + + # Increments the number of pending (skipped) examples. + def pending(_result) + @skipped += 1 + end + + # Produces the total counts. + def counts + Counts.new( + total: @pass + @failures + @skipped, + failures: @failures, + errors: @errors, + skipped: @skipped + ) + end + end + end +end diff --git a/src/spectator/formatting/components/pending_result_block.cr b/src/spectator/formatting/components/pending_result_block.cr new file mode 100644 index 0000000..95c2481 --- /dev/null +++ b/src/spectator/formatting/components/pending_result_block.cr @@ -0,0 +1,29 @@ +require "colorize" +require "../../example" +require "../../pending_result" +require "./result_block" + +module Spectator::Formatting::Components + # Displays information about a pending result. + struct PendingResultBlock < ResultBlock + # Creates the component. + def initialize(example : Example, index : Int32, @result : PendingResult) + super(example, index) + end + + # Content displayed on the second line of the block after the label. + private def subtitle + @result.reason + end + + # Prefix for the second line of the block. + private def subtitle_label + # TODO: Could be pending or skipped. + "Pending: ".colorize(:yellow) + end + + # No content for this type of block. + private def content(io) + end + end +end diff --git a/src/spectator/formatting/components/profile.cr b/src/spectator/formatting/components/profile.cr new file mode 100644 index 0000000..2a56e0d --- /dev/null +++ b/src/spectator/formatting/components/profile.cr @@ -0,0 +1,41 @@ +require "../../profile" +require "./runtime" + +module Spectator::Formatting::Components + # Displays profiling information for slow examples. + struct Profile + # Creates the component with the specified *profile*. + def initialize(@profile : Spectator::Profile) + end + + # Produces the output containing the profiling information. + def to_s(io) + io << "Top " + io << @profile.size + io << " slowest examples (" + io << Runtime.new(@profile.time) + io << ", " + io << @profile.percentage.round(2) + io.puts "% of total time):" + + @profile.each do |example| + example_profile(io, example) + end + end + + # Writes a single example's timing to the output. + private def example_profile(io, example) + io << " " + io.puts example + io << " " + io << Runtime.new(example.result.elapsed).colorize.bold + + if location = example.location + io << ' ' + io.puts location + else + io.puts + end + end + end +end diff --git a/src/spectator/formatting/components/result_block.cr b/src/spectator/formatting/components/result_block.cr new file mode 100644 index 0000000..bf4b3d2 --- /dev/null +++ b/src/spectator/formatting/components/result_block.cr @@ -0,0 +1,95 @@ +require "../../example" +require "./block" +require "./comment" + +module Spectator::Formatting::Components + # Base class that displayed indexed results in block form. + # These typically take the form: + # ```text + # 1) Title + # Label: Subtitle + # + # Content + # # Source + # ``` + abstract struct ResultBlock < Block + # Creates the block with the specified *index* and for the given *example*. + def initialize(@example : Example, @index : Int32, @subindex : Int32 = 0) + super() + end + + # Content displayed on the first line of the block. + # Will be stringified. + # By default, uses the example name. + # Can be overridden to use a different value. + private def title + @example + end + + # Content displayed on the second line of the block after the label. + # Will be stringified. + private abstract def subtitle + + # Prefix for the second line of the block. + # Will be stringified. + # This is typically something like "Error:" or "Failure:" + private abstract def subtitle_label + + # Produces the main content of the block. + # *io* is the stream to write to. + # `#line` and `#indent` (from `Block`) should be used to maintain spacing. + private abstract def content(io) + + # Writes the component's output to the specified stream. + def to_s(io) + title_line(io) + # Ident over to align with the spacing used by the index. + indent(index_digit_count + 2) do + subtitle_line(io) + io.puts + content(io) + location_line(io) + end + end + + # Produces the title line. + private def title_line(io) + line(io) do + io << @index + io << '.' << @subindex if @subindex > 0 + io << ") " << title + end + end + + # Produces the subtitle line. + private def subtitle_line(io) + line(io) do + io << subtitle_label << subtitle + end + end + + # Produces the location line. + # This is where the result was determined. + private def location_line(io) + location = if (result = @example.result).responds_to?(:location?) + result.location? + end + location ||= @example.location? + return unless location + + line(io) { io << Comment.colorize(location) } + end + + # Computes the number of spaces the index takes + private def index_digit_count + count = digit_count(@index) + count += 1 + digit_count(@subindex) if @subindex > 0 + count + end + + # Computes the number of spaces an integer takes. + private def digit_count(integer) + (Math.log(integer.to_f + 1) / Math::LOG10).ceil.to_i + end + end +end diff --git a/src/spectator/formatting/components/runtime.cr b/src/spectator/formatting/components/runtime.cr new file mode 100644 index 0000000..9f28813 --- /dev/null +++ b/src/spectator/formatting/components/runtime.cr @@ -0,0 +1,66 @@ +module Spectator::Formatting::Components + # Presents a human readable time span. + struct Runtime + # Creates the component. + def initialize(@span : Time::Span) + end + + # Appends the elapsed time to the output. + # The text will be formatted as follows, depending on the magnitude: + # ```text + # ## microseconds + # ## milliseconds + # ## seconds + # #:## + # #:##:## + # # days #:##:## + # ``` + def to_s(io) + millis = @span.total_milliseconds + return format_micro(io, millis * 1000) if millis < 1 + + seconds = @span.total_seconds + return format_millis(io, millis) if seconds < 1 + return format_seconds(io, seconds) if seconds < 60 + + minutes, seconds = seconds.divmod(60) + return format_minutes(io, minutes, seconds) if minutes < 60 + + hours, minutes = minutes.divmod(60) + return format_hours(io, hours, minutes, seconds) if hours < 24 + + days, hours = hours.divmod(24) + format_days(io, days, hours, minutes, seconds) + end + + # Formats for microseconds. + private def format_micro(io, micros) + io << micros.round.to_i << " microseconds" + end + + # Formats for milliseconds. + private def format_millis(io, millis) + io << millis.round(2) << " milliseconds" + end + + # Formats for seconds. + private def format_seconds(io, seconds) + io << seconds.round(2) << " seconds" + end + + # Formats for minutes. + private def format_minutes(io, minutes, seconds) + io.printf("%i:%02i", minutes, seconds) + end + + # Formats for hours. + private def format_hours(io, hours, minutes, seconds) + io.printf("%i:%02i:%02i", hours, minutes, seconds) + end + + # Formats for days. + private def format_days(io, days, hours, minutes, seconds) + io.printf("%i days %i:%02i:%02i", days, hours, minutes, seconds) + end + end +end diff --git a/src/spectator/formatting/components/stats.cr b/src/spectator/formatting/components/stats.cr new file mode 100644 index 0000000..ab76c7c --- /dev/null +++ b/src/spectator/formatting/components/stats.cr @@ -0,0 +1,38 @@ +require "colorize" +require "../../report" +require "./runtime" +require "./totals" + +module Spectator::Formatting::Components + # Statistics information displayed at the end of a run. + struct Stats + # Creates the component with stats from *report*. + def initialize(@report : Report) + end + + # Displays the stats. + def to_s(io) + runtime(io) + totals(io) + if seed = @report.random_seed? + random(io, seed) + end + end + + # Displays the time it took to run the suite. + private def runtime(io) + io << "Finished in " + io.puts Runtime.new(@report.runtime) + end + + # Displays the counts for each type of result. + private def totals(io) + io.puts Totals.colorize(@report.counts) + end + + # Displays the random seed. + private def random(io, seed) + io.puts "Randomized with seed: #{seed}".colorize(:cyan) + end + end +end diff --git a/src/spectator/formatting/components/tap_profile.cr b/src/spectator/formatting/components/tap_profile.cr new file mode 100644 index 0000000..64c36b0 --- /dev/null +++ b/src/spectator/formatting/components/tap_profile.cr @@ -0,0 +1,42 @@ +require "../../profile" +require "./runtime" + +module Spectator::Formatting::Components + # Displays profiling information for slow examples in a TAP format. + # Produces output similar to `Profile`, but formatted for TAP. + struct TAPProfile + # Creates the component with the specified *profile*. + def initialize(@profile : Spectator::Profile) + end + + # Produces the output containing the profiling information. + def to_s(io) + io << "# Top " + io << @profile.size + io << " slowest examples (" + io << Runtime.new(@profile.time) + io << ", " + io << @profile.percentage.round(2) + io.puts "% of total time):" + + @profile.each do |example| + example_profile(io, example) + end + end + + # Writes a single example's timing to the output. + private def example_profile(io, example) + io << "# " + io.puts example + io << "# " + io << Runtime.new(example.result.elapsed) + + if location = example.location? + io << ' ' + io.puts location + else + io.puts + end + end + end +end diff --git a/src/spectator/formatting/components/totals.cr b/src/spectator/formatting/components/totals.cr new file mode 100644 index 0000000..941b6ee --- /dev/null +++ b/src/spectator/formatting/components/totals.cr @@ -0,0 +1,46 @@ +require "colorize" + +module Spectator::Formatting::Components + # Displays counts for each type of example result (pass, fail, error, pending). + struct Totals + # Creates the component with the specified counts. + def initialize(@examples : Int32, @failures : Int32, @errors : Int32, @pending : Int32) + end + + # Creates the component by pulling numbers from *counts*. + def initialize(counts) + @examples = counts.run + @failures = counts.fail + @errors = counts.error + @pending = counts.pending + end + + # Creates the component, but colors it whether there were pending or failed results. + # The component will be red if there were failures (or errors), + # yellow if there were pending/skipped tests, + # and green if everything passed. + def self.colorize(counts) + totals = new(counts) + if counts.fail > 0 + totals.colorize(:red) + elsif counts.pending > 0 + totals.colorize(:yellow) + else + totals.colorize(:green) + end + end + + # Writes the counts to the output. + def to_s(io) + io << @examples << " examples, " << @failures << " failures" + + if @errors > 0 + io << " (" << @errors << " errors)" + end + + if @pending > 0 + io << ", " << @pending << " pending" + end + end + end +end diff --git a/src/spectator/formatting/document_formatter.cr b/src/spectator/formatting/document_formatter.cr index c100e55..fe6c1ce 100644 --- a/src/spectator/formatting/document_formatter.cr +++ b/src/spectator/formatting/document_formatter.cr @@ -1,45 +1,76 @@ +require "../label" require "./formatter" -require "./suite_summary" +require "./summary" module Spectator::Formatting # Produces an indented document-style output. # Each nested group of examples increases the indent. # Example names are output in a color based on their result. class DocumentFormatter < Formatter - include SuiteSummary + include Summary + # Identation string. private INDENT = " " - @previous_hierarchy = [] of NestedExampleGroup + # String used for groups and examples that don't have a name. + private NO_NAME = "" + + # Output stream to write results to. + private getter io + + @previous_hierarchy = [] of Label # Creates the formatter. # By default, output is sent to STDOUT. def initialize(@io : IO = STDOUT) end - # Does nothing when an example is started. - def start_example(example) - hierarchy = group_hierarchy(example) + # Invoked just before an example runs. + # Prints the example group hierarchy if it changed. + def example_started(notification) + hierarchy = group_hierarchy(notification.example) tuple = hierarchy_diff(@previous_hierarchy, hierarchy) print_sub_hierarchy(*tuple) @previous_hierarchy = hierarchy end - # Produces a single character output based on a result. - def end_example(result) - @previous_hierarchy.size.times { @io.print INDENT } - @io.puts result.call(Color) { result.example.description } + # Invoked after an example completes successfully. + # Produces a successful example line. + def example_passed(notification) + name = (notification.example.name? || NO_NAME) + line(name.colorize(:green)) + end + + # Invoked after an example is skipped or marked as pending. + # Produces a pending example line. + def example_pending(notification) + name = (notification.example.name? || NO_NAME) + line(name.colorize(:yellow)) + end + + # Invoked after an example fails. + # Produces a failure example line. + def example_failed(notification) + name = (notification.example.name? || NO_NAME) + line(name.colorize(:red)) + end + + # Invoked after an example fails from an unexpected error. + # Produces a failure example line. + def example_error(notification) + example_failed(notification) end # Produces a list of groups making up the hierarchy for an example. private def group_hierarchy(example) - hierarchy = [] of NestedExampleGroup - group = example.group - while group.is_a?(NestedExampleGroup) - hierarchy << group - group = group.parent + Array(Label).new.tap do |hierarchy| + group = example.group? + while group && (parent = group.group?) + hierarchy << (group.name? || NO_NAME) + group = parent + end + hierarchy.reverse! end - hierarchy.reverse end # Generates a difference between two hierarchies. @@ -53,12 +84,16 @@ module Spectator::Formatting end # Displays an indented hierarchy starting partially into the whole hierarchy. - private def print_sub_hierarchy(index, sub_hierarchy) - sub_hierarchy.each do |group| - index.times { @io.print INDENT } - @io.puts group.description - index += 1 + private def print_sub_hierarchy(start_index, hierarchy) + hierarchy.each_with_index(start_index) do |name, index| + line(name, index) end end + + # Displays an indented line of text. + private def line(text, level = @previous_hierarchy.size) + level.times { @io << INDENT } + @io.puts text + end end end diff --git a/src/spectator/formatting/dots_formatter.cr b/src/spectator/formatting/dots_formatter.cr deleted file mode 100644 index 392b26e..0000000 --- a/src/spectator/formatting/dots_formatter.cr +++ /dev/null @@ -1,60 +0,0 @@ -require "./formatter" -require "./suite_summary" - -module Spectator::Formatting - # Produces a single character for each example. - # A dot is output for each successful example (hence the name). - # Other characters are output for non-successful results. - # At the end of the test suite, a summary of failures and results is displayed. - class DotsFormatter < Formatter - include SuiteSummary - - # Creates the formatter. - # By default, output is sent to STDOUT. - def initialize(@io : IO = STDOUT) - end - - # Does nothing when an example is started. - def start_example(example) - # ... - end - - # Produces a single character output based on a result. - def end_example(result) - @io.print result.call(Character) - end - - # Interface for `Result` to pick a character for output. - private module Character - extend self - - # Characters for each of the result types. - private CHARACTERS = { - success: '.', - failure: 'F', - error: 'E', - pending: '*', - } - - # Character output for a successful example. - def success - Color.success(CHARACTERS[:success]) - end - - # Character output for a failed example. - def failure - Color.failure(CHARACTERS[:failure]) - end - - # Character output for an errored example. - def error - Color.error(CHARACTERS[:error]) - end - - # Character output for a pending or skipped example. - def pending - Color.pending(CHARACTERS[:pending]) - end - end - end -end diff --git a/src/spectator/formatting/error_junit_test_case.cr b/src/spectator/formatting/error_junit_test_case.cr deleted file mode 100644 index f832355..0000000 --- a/src/spectator/formatting/error_junit_test_case.cr +++ /dev/null @@ -1,19 +0,0 @@ -require "./failure_junit_test_case" - -module Spectator::Formatting - # JUnit test case for a errored result. - private class ErrorJUnitTestCase < FailureJUnitTestCase - # Result for this test case. - private getter result - - # Creates the JUnit test case. - def initialize(@result : ErroredResult) - end - - # Adds the exception to the XML block. - private def content(xml) - xml.element("error", message: @result.error.message, type: @result.error.class) - super - end - end -end diff --git a/src/spectator/formatting/failure_block.cr b/src/spectator/formatting/failure_block.cr deleted file mode 100644 index b542775..0000000 --- a/src/spectator/formatting/failure_block.cr +++ /dev/null @@ -1,120 +0,0 @@ -module Spectator::Formatting - # Constructs a block of text containing information about a failed example. - # - # A failure block takes the form: - # - # ```text - # 1) Example name - # Failure: Reason or message - # - # Expected: value - # got: value - # - # # spec/source_spec.cr:42 - # ``` - private struct FailureBlock - # Creates the failure block. - # The *index* uniquely identifies the failure in the output. - # The *result* is the outcome of the failed example. - def initialize(@index : Int32, @result : FailedResult) - end - - # Creates the block of text describing the failure. - def to_s(io) - indent = Indent.new(io) - inner_indent = integer_length(@index) + 2 # +2 for ) and space after number. - - indent.increase do - title(indent) - indent.increase(inner_indent) do - content(indent) - source(indent) - end - end - end - - # Produces the title of the failure block. - # The line takes the form: - # ```text - # 1) Example name - # ``` - private def title(indent) - indent.line(NumberedItem.new(@index, @result.example)) - end - - # Produces the main content of the failure block. - # Any failed expectations are displayed, - # then an error stacktrace if an error occurred. - private def content(indent) - unsatisfied_expectations(indent) - error_stacktrace(indent) if @result.is_a?(ErroredResult) - end - - # Produces a list of unsatisfied expectations and their values. - private def unsatisfied_expectations(indent) - @result.expectations.each_unsatisfied do |expectation| - indent.line(Color.failure(LabeledText.new("Failure", expectation.failure_message))) - indent.line - indent.increase do - matcher_values(indent, expectation) - end - indent.line - end - end - - # Produces the values list for an expectation - private def matcher_values(indent, expectation) - MatchDataValues.new(expectation.values).each do |pair| - colored_pair = if expectation.satisfied? - Color.success(pair) - else - Color.failure(pair) - end - indent.line(colored_pair) - end - end - - # Produces the stack trace for an errored result. - private def error_stacktrace(indent) - error = @result.error - indent.line(Color.error(LabeledText.new("Error", error))) - indent.line - indent.increase do - loop do - display_error(indent, error) - if (next_error = error.cause) - error = next_error - else - break - end - end - end - indent.line - end - - # Display a single error and its stacktrace. - private def display_error(indent, error) : Nil - indent.line(Color.error(LabeledText.new(error.class.to_s, error))) - indent.increase do - error.backtrace.each do |frame| - indent.line(Color.error(frame)) - end - end - end - - # Produces the source line of the failure block. - private def source(indent) - indent.line(Comment.color(@result.example.source)) - end - - # Gets the number of characters a positive integer spans in base 10. - private def integer_length(index) - count = 1 - while index >= 10 - index /= 10 - count += 1 - end - count - end - end -end diff --git a/src/spectator/formatting/failure_command.cr b/src/spectator/formatting/failure_command.cr deleted file mode 100644 index 5d4187c..0000000 --- a/src/spectator/formatting/failure_command.cr +++ /dev/null @@ -1,19 +0,0 @@ -module Spectator::Formatting - # Produces a stringified command to run a failed test. - private struct FailureCommand - # Creates the failure command. - def initialize(@example : Example) - end - - # Appends the command to the output. - def to_s(io) - io << "crystal spec " - io << @example.source - end - - # Colorizes the command instance based on the result. - def self.color(result) - result.call(Color) { new(result.example) } - end - end -end diff --git a/src/spectator/formatting/failure_junit_test_case.cr b/src/spectator/formatting/failure_junit_test_case.cr deleted file mode 100644 index 99b0265..0000000 --- a/src/spectator/formatting/failure_junit_test_case.cr +++ /dev/null @@ -1,37 +0,0 @@ -require "./finished_junit_test_case" - -module Spectator::Formatting - # JUnit test case for a failed result. - private class FailureJUnitTestCase < FinishedJUnitTestCase - # Result for this test case. - private getter result - - # Creates the JUnit test case. - def initialize(@result : FailedResult) - end - - # Status string specific to the result type. - private def status : String - "FAIL" - end - - # Adds the failed expectations to the XML block. - private def content(xml) - super - @result.expectations.each_unsatisfied do |expectation| - xml.element("failure", message: expectation.failure_message) do - expectation_values(expectation.values, xml) - end - end - end - - # Adds the expectation values to the failure block. - private def expectation_values(labeled_values, xml) - labeled_values.each do |entry| - label = entry.first - value = entry.last - xml.text("#{label}: #{value}\n") - end - end - end -end diff --git a/src/spectator/formatting/finished_junit_test_case.cr b/src/spectator/formatting/finished_junit_test_case.cr deleted file mode 100644 index 0db79de..0000000 --- a/src/spectator/formatting/finished_junit_test_case.cr +++ /dev/null @@ -1,21 +0,0 @@ -require "./junit_test_case" - -module Spectator::Formatting - # Commonalities of all test cases that ran (success or failure). - private abstract class FinishedJUnitTestCase < JUnitTestCase - # Produces the test case XML element. - def to_xml(xml : ::XML::Builder) - xml.element("testcase", **full_attributes) do - content(xml) - end - end - - # Attributes that go in the "testcase" XML element. - private def full_attributes - attributes.merge( - time: result.elapsed.total_seconds, - assertions: result.expectations.size - ) - end - end -end diff --git a/src/spectator/formatting/formatter.cr b/src/spectator/formatting/formatter.cr index 9efd800..7c46e54 100644 --- a/src/spectator/formatting/formatter.cr +++ b/src/spectator/formatting/formatter.cr @@ -1,27 +1,128 @@ module Spectator::Formatting - # Interface for reporting test progress and results. + # Base class and interface used to notify systems of events. + # This is typically used for producing output from test results, + # but can also be used to send data to external systems. # - # The methods should be called in this order: - # 1. `#start_suite` - # 2. `#start_example` - # 3. `#end_example` - # 4. `#end_suite` + # All event methods are implemented as no-ops. + # To respond to an event, override its method. + # Every method receives a notification object containing information about the event. # - # Steps 2 and 3 are called for each example in the suite. + # Methods are called in this order: + # 1. `#start` + # 2. `#example_started` + # 3. `#example_finished` + # 4. `#example_passed` + # 5. `#example_pending` + # 6. `#example_failed` + # 7. `#stop` + # 8. `#start_dump` + # 9. `#dump_pending` + # 10. `#dump_failures` + # 11. `#dump_profile` + # 12. `#dump_summary` + # 13. `#close` + # + # Only one of the `#example_passed`, `#example_pending`, or `#example_failed` methods + # will be called after `#example_finished`, depending on the outcome of the test. + # + # The "dump" methods are called after all tests that will run have run. + # They are provided summarized information. abstract class Formatter - # Called when a test suite is starting to execute. - abstract def start_suite(suite : TestSuite) + # This method is the first method to be invoked + # and will be called only once. + # It is called before any examples run. + # The *notification* will be a `StartNotification` type of object. + def start(_notification) + end - # Called when a test suite finishes. - # The results from the entire suite are provided. - # The *profile* value is not nil when profiling results should be displayed. - abstract def end_suite(report : Report, profile : Profile?) + # Invoked just before an example runs. + # This method is called once for every example. + # The *notification* will be an `ExampleNotification` type of object. + def example_started(_notification) + end - # Called before a test starts. - abstract def start_example(example : Example) + # Invoked just after an example completes. + # This method is called once for every example. + # One of `#example_passed`, `#example_pending` or `#example_failed` + # will be called immediately after this method, depending on the example's result. + # The *notification* will be an `ExampleNotification` type of object. + def example_finished(_notification) + end - # Called when a test finishes. - # The result of the test is provided. - abstract def end_example(result : Result) + # Invoked after an example completes successfully. + # This is called right after `#example_finished`. + # The *notification* will be an `ExampleNotification` type of object. + def example_passed(_notification) + end + + # Invoked after an example is skipped or marked as pending. + # This is called right after `#example_finished`. + # The *notification* will be an `ExampleNotification` type of object. + def example_pending(_notification) + end + + # Invoked after an example fails. + # This is called right after `#example_finished`. + # The *notification* will be an `ExampleNotification` type of object. + # + # NOTE: Errors are normally considered failures, + # however `#example_error` is called instead if one occurs in an exmaple. + def example_failed(_notification) + end + + # Invoked after an example fails from an unexpected error. + # This is called right after `#example_finished`. + # The *notification* will be an `ExampleNotification` type of object. + def example_error(_notification) + end + + # Called whenever the example or framework produces a message. + # This is typically used for logging. + # The *notification* will be a `MessageNotification` type of object. + def message(_notification) + end + + # Invoked after all tests that will run have completed. + # When this method is called, it should be considered that the testing is done. + # Summary (dump) methods will be called after this. + def stop + end + + # Invoked after all examples finished. + # Indicates that summarized report data is about to be produced. + # This method is called after `#stop` and before `#dump_pending`. + def start_dump + end + + # Invoked after testing completes with a list of pending examples. + # This method will be called with an empty list if there were no pending (skipped) examples. + # Called after `#start_dump` and before `#dump_failures`. + # The *notification* will be an `ExampleSummaryNotification` type of object. + def dump_pending(_notification) + end + + # Invoked after testing completes with a list of failed examples. + # This method will be called with an empty list if there were no failures. + # Called after `#dump_pending` and before `#dump_summary`. + # The *notification* will be an `ExampleSummaryNotification` type of object. + def dump_failures(_notification) + end + + # Invoked after testing completes with profiling information. + # This method is only called if profiling is enabled. + # Called after `#dump_failures` and before `#dump_summary`. + def dump_profile(_notification) + end + + # Invoked after testing completes with summarized information from the test suite. + # Called after `#dump_profile` and before `#close`. + # The *notification* will be an `SummaryNotification` type of object. + def dump_summary(_notification) + end + + # Invoked at the end of the program. + # Allows the formatter to perform any cleanup and teardown. + def close + end end end diff --git a/src/spectator/formatting/html/body.ecr b/src/spectator/formatting/html/body.ecr new file mode 100644 index 0000000..9537921 --- /dev/null +++ b/src/spectator/formatting/html/body.ecr @@ -0,0 +1,86 @@ +
+

Test Results

+ <% escape(totals(report)) %> + <% escape(runtime(report.runtime)) %> +
+ +<%- if report.counts.fail > 0 -%> +

Failures (<%= report.counts.fail %>)

+
    + <%- report.failures.each do |example| -%> +
  1. + + <% escape(example) %> + +
    2. + <%- end -%> +
+<%- end -%> + +<%- if report.counts.pending > 0 -%> +

Pending (<%= report.counts.pending %>)

+
    + <%- report.pending.each do |example| -%> +
  1. + + <% escape(example) %> + +
    2. + <%- end -%> +
+<%- end -%> + +

Examples (<%= report.counts.total %>)

+
    + <%- report.examples.each do |example| -%> +
  • +

    <% escape(example) %>

    + <%= example.result %> + Took <% escape(runtime(example.result.elapsed)) %> + <% if location = example.location? %><% escape(location) %><% end %> + <% if result = example.result.as?(PendingResult) %>

    <% escape(result.reason) %>

    + + <%- elsif result = example.result.as?(ErrorResult) -%> +

    + <% escape(result.error.class) %> + <% escape(result.error.message) %> +

    + <%- if backtrace = result.error.backtrace? -%> +
    + <%- backtrace.each do |line| -%> + <% escape(line) %> + <%- end -%> +
    + <%- end -%> + + <%- elsif result = example.result.as?(FailResult) -%> +

    <% escape(result.error.message) %> + <%- end -%> + + <%- if example.result.expectations.empty? -%> + No expectations reported + <%- else -%> +

    Expectations

    +
      + <%- example.result.expectations.each do |expectation| -%> +
    1. "<% if location = expectation.location? %> title="<% escape(location) %>"<% end %>> + <% escape(expectation.description) %> + <%- if expectation.satisfied? -%> + pass + <%- else -%> + fail +

      <% escape(expectation.failure_message) %> +

      + <%- expectation.values.each do |key, value| -%> +
      <% escape(key) %>
      +
      <% escape(value) %>
      + <%- end -%> +
      + <%- end -%> +
      2. + <%- end -%> +
    + <%- end -%> +
    • + <%- end -%> +
diff --git a/src/spectator/formatting/html/foot.ecr b/src/spectator/formatting/html/foot.ecr new file mode 100644 index 0000000..308b1d0 --- /dev/null +++ b/src/spectator/formatting/html/foot.ecr @@ -0,0 +1,2 @@ + + diff --git a/src/spectator/formatting/html/head.ecr b/src/spectator/formatting/html/head.ecr new file mode 100644 index 0000000..f34fe42 --- /dev/null +++ b/src/spectator/formatting/html/head.ecr @@ -0,0 +1,213 @@ + + + + + + + + + Test Results + + + + + diff --git a/src/spectator/formatting/html_formatter.cr b/src/spectator/formatting/html_formatter.cr new file mode 100644 index 0000000..a4744c0 --- /dev/null +++ b/src/spectator/formatting/html_formatter.cr @@ -0,0 +1,69 @@ +require "ecr" +require "html" +require "./formatter" + +module Spectator::Formatting + # Produces an HTML document with results of the test suite. + class HTMLFormatter < Formatter + # Default HTML file name. + private OUTPUT_FILE = "output.html" + + # Output stream for the HTML file. + private getter! io : IO + + # Creates the formatter. + # The *output_path* can be a directory or path of an HTML file. + # If the former, then an "output.html" file will be generated in the specified directory. + def initialize(output_path = OUTPUT_FILE) + @output_path = if output_path.ends_with?(".html") + output_path + else + File.join(output_path, OUTPUT_FILE) + end + end + + # Prepares the formatter for writing. + def start(_notification) + @io = File.open(@output_path, "w") + ECR.embed(__DIR__ + "/html/head.ecr", io) + end + + # Invoked after testing completes with summarized information from the test suite. + # All results are gathered at the end, then the report is generated. + def dump_summary(notification) + report = notification.report # ameba:disable Lint/UselessAssign + ECR.embed(__DIR__ + "/html/body.ecr", io) + end + + # Invoked at the end of the program. + # Allows the formatter to perform any cleanup and teardown. + def close + ECR.embed(__DIR__ + "/html/foot.ecr", io) + io.flush + io.close + end + + private def escape(string) + HTML.escape(string.to_s, io) + end + + private def runtime(span) + Components::Runtime.new(span).to_s + end + + private def totals(report) + Components::Totals.new(report.counts) + end + + private def summary_result(report) + counts = report.counts + if counts.fail > 0 + "fail" + elsif counts.pending > 0 + "pending" + else + "pass" + end + end + end +end diff --git a/src/spectator/formatting/human_time.cr b/src/spectator/formatting/human_time.cr deleted file mode 100644 index c7ec687..0000000 --- a/src/spectator/formatting/human_time.cr +++ /dev/null @@ -1,41 +0,0 @@ -module Spectator::Formatting - # Provides a more human-friendly formatting for a time span. - # This produces a string with the minimum of - # microseconds, milliseconds, seconds, minutes, hours, or days. - private struct HumanTime - @string : String - - # Creates the wrapper - def initialize(span) - @string = simplify(span) - end - - # Produces the human-friendly string for a time span. - def to_s(io) - io << @string - end - - # Does the actual work of converting a time span to string. - private def simplify(span) - millis = span.total_milliseconds - return "#{(millis * 1000).round.to_i} microseconds" if millis < 1 - - seconds = span.total_seconds - return "#{millis.round(2)} milliseconds" if seconds < 1 - return "#{seconds.round(2)} seconds" if seconds < 60 - - int_seconds = seconds.to_i - minutes = int_seconds // 60 - int_seconds %= 60 - return sprintf("%i:%02i", minutes, int_seconds) if minutes < 60 - - hours = minutes // 60 - minutes %= 60 - return sprintf("%i:%02i:%02i", hours, minutes, int_seconds) if hours < 24 - - days = hours // 24 - hours %= 24 - sprintf("%i days %i:%02i:%02i", days, hours, minutes, int_seconds) - end - end -end diff --git a/src/spectator/formatting/indent.cr b/src/spectator/formatting/indent.cr deleted file mode 100644 index 103747b..0000000 --- a/src/spectator/formatting/indent.cr +++ /dev/null @@ -1,50 +0,0 @@ -module Spectator::Formatting - # Tracks indentation for text output. - # To use, create an instance and call `#increase` when a block should be indented. - # The `#increase` method yields, so additional `#increase` and `#line` methods can be called. - # Then call `#line` to produce a line of text at the current indent. - # ``` - # indent = Indent.new(io) - # indent.increase do - # indent.line("Text") - # indent.increase do - # indent.line("More text") - # end - # end - # ``` - private struct Indent - # Default number of spaces to indent by. - INDENT_SIZE = 2 - - # Creates the identation tracker. - # The *io* is the stream to output to. - # The *indent_size* is how much (number of spaces) to indent at each level. - # The *initial_indent* is what the ident should be set to. - def initialize(@io : IO, @indent_size = INDENT_SIZE, inital_indent @indent = 0) - end - - # Indents the text and yields. - def increase(&block) - increase(@indent_size, &block) - end - - # Indents the text by a specified amount and yields. - def increase(amount) : Nil - @indent += amount - yield - ensure - @indent -= amount - end - - # Produces an empty line. - def line - @io.puts - end - - # Produces a line of indented text. - def line(text) - @indent.times { @io << ' ' } - @io.puts text - end - end -end diff --git a/src/spectator/formatting/json_formatter.cr b/src/spectator/formatting/json_formatter.cr index da09f18..90bac6a 100644 --- a/src/spectator/formatting/json_formatter.cr +++ b/src/spectator/formatting/json_formatter.cr @@ -2,94 +2,65 @@ require "json" require "./formatter" module Spectator::Formatting - # Produces a JSON document containing the test results. - class JsonFormatter < Formatter + # Produces a JSON document with results of the test suite. + class JSONFormatter < Formatter # Creates the formatter. # By default, output is sent to STDOUT. - def initialize(io : IO = STDOUT) - @json = ::JSON::Builder.new(io) + def initialize(io = STDOUT) + @json = JSON::Builder.new(io) end - # Called when a test suite is starting to execute. - def start_suite(suite : TestSuite) + # Begins the JSON document output. + def start(_notification) @json.start_document @json.start_object + @json.field("version", Spectator::VERSION) + + # Start examples array. @json.string("examples") @json.start_array end - # Called when a test suite finishes. - # The results from the entire suite are provided. - # The *profile* value is not nil when profiling results should be displayed. - def end_suite(report : Report, profile : Profile?) - @json.end_array # examples - totals(report) - timing(report) - profile(profile) if profile - @json.field("result", report.failed? ? "fail" : "success") - @json.end_object + # Adds an object containing fields about the example. + def example_finished(notification) + notification.example.to_json(@json) end - # Called before a test starts. - def start_example(example : Example) + # Marks the end of the examples array. + def stop + @json.end_array # Close examples array. end - # Called when a test finishes. - # The result of the test is provided. - def end_example(result : Result) - result.to_json(@json) - end - - # Adds the totals section of the document. - private def totals(report) - @json.field("totals") do - @json.object do - @json.field("examples", report.example_count) - @json.field("success", report.successful_count) - @json.field("fail", report.failed_count) - @json.field("error", report.error_count) - @json.field("pending", report.pending_count) - @json.field("remaining", report.remaining_count) - end - end - end - - # Adds the timings section of the document. - private def timing(report) - @json.field("timing") do - @json.object do - @json.field("runtime", report.runtime.total_seconds) - @json.field("examples", report.example_runtime.total_seconds) - @json.field("overhead", report.overhead_time.total_seconds) - end - end - end - - # Adds the profile information to the document. - private def profile(profile) + # Adds the profiling information to the document. + def dump_profile(notification) @json.field("profile") do - @json.object do - @json.field("count", profile.size) - @json.field("time", profile.total_time.total_seconds) - @json.field("percentage", profile.percentage) - @json.field("results") do - @json.array do - profile.each do |result| - profile_entry(result) - end - end - end - end + notification.profile.to_json(@json) end end - # Adds a profile entry to the document. - private def profile_entry(result) - @json.object do - @json.field("example", result.example) - @json.field("time", result.elapsed.total_seconds) - @json.field("source", result.example.source) + # Adds the summary object to the document. + def dump_summary(notification) + report = notification.report + + @json.field("summary") do + @json.object do + @json.field("duration", report.runtime.total_seconds) + @json.field("example_count", report.counts.total) + @json.field("failure_count", report.counts.fail) + @json.field("error_count", report.counts.error) + @json.field("pending_count", report.counts.pending) + end end + + totals = Components::Totals.new(report.counts) + @json.field("summary_line", totals.to_s) + end + + # Ends the JSON document and flushes output. + def close + @json.end_object + @json.end_document + @json.flush end end end diff --git a/src/spectator/formatting/junit_formatter.cr b/src/spectator/formatting/junit_formatter.cr index 26a6a1a..a777371 100644 --- a/src/spectator/formatting/junit_formatter.cr +++ b/src/spectator/formatting/junit_formatter.cr @@ -1,63 +1,51 @@ require "xml" +require "./formatter" module Spectator::Formatting - # Formatter for producing a JUnit XML report. + # Produces a JUnit compatible XML file containing the test results. class JUnitFormatter < Formatter - # Name of the JUnit output file. - private JUNIT_XML_FILE = "output.xml" + # Default XML file name. + private OUTPUT_FILE = "output.xml" - # Name of the top-level test suites block. - private NAME = "Spec" + # XML builder for the entire document. + private getter! xml : XML::Builder + + # Output stream for the XML file. + private getter! io : IO # Creates the formatter. - # By default, output is sent to STDOUT. - def initialize(output_dir : String) - path = File.join(output_dir, JUNIT_XML_FILE) - @io = File.open(path, "w") - @xml = XML::Builder.new(@io) + # The *output_path* can be a directory or path of an XML file. + # If the former, then an "output.xml" file will be generated in the specified directory. + def initialize(output_path = OUTPUT_FILE) + @output_path = if output_path.ends_with?(".xml") + output_path + else + File.join(output_path, OUTPUT_FILE) + end end - # Called when a test suite is starting to execute. - def start_suite(suite : TestSuite) - @xml.start_document(encoding: "UTF-8") + # Prepares the formatter for writing. + def start(_notification) + @io = io = File.open(@output_path, "w") + @xml = xml = XML::Builder.new(io) + xml.start_document("1.0", "UTF-8") end - # Called when a test suite finishes. - # The results from the entire suite are provided. - # The *profile* value does nothing for this formatter. - def end_suite(report : Report, profile : Profile?) - test_suites_block(report) - @xml.end_document - @xml.flush - @io.close + # Invoked after testing completes with summarized information from the test suite. + # Unfortunately, the JUnit specification is not conducive to streaming data. + # All results are gathered at the end, then the report is generated. + def dump_summary(notification) + report = notification.report + root = Components::JUnit::Root.from_report(report) + root.to_xml(xml) end - # Called before a test starts. - def start_example(example : Example) - end - - # Called when a test finishes. - # The result of the test is provided. - def end_example(result : Result) - end - - # Creates the "testsuites" block in the XML. - private def test_suites_block(report) - @xml.element("testsuites", - tests: report.example_count, - failures: report.failed_count, - errors: report.error_count, - time: report.runtime.total_seconds, - name: NAME) do - add_test_suites(report) - end - end - - # Adds all of the individual test suite blocks. - private def add_test_suites(report) - report.group_by(&.example.source.path).each do |path, results| - JUnitTestSuite.new(path, results).to_xml(@xml) - end + # Invoked at the end of the program. + # Allows the formatter to perform any cleanup and teardown. + def close + xml.end_document + xml.flush + io.close end end end diff --git a/src/spectator/formatting/junit_test_case.cr b/src/spectator/formatting/junit_test_case.cr deleted file mode 100644 index c4b140e..0000000 --- a/src/spectator/formatting/junit_test_case.cr +++ /dev/null @@ -1,42 +0,0 @@ -module Spectator::Formatting - # Base type for all JUnit test case results. - private abstract class JUnitTestCase - # Produces the test case XML element. - def to_xml(xml : ::XML::Builder) - xml.element("testcase", **attributes) do - content(xml) - end - end - - # Attributes that go in the "testcase" XML element. - private def attributes - { - name: result.example, - status: status, - classname: classname, - } - end - - # Result to pull values from. - private abstract def result - - # Status string specific to the result type. - private abstract def status : String - - # Adds additional content to the "testcase" XML block. - # Override this to add more content. - private def content(xml) - # ... - end - - # Java-ified class name created from the spec. - private def classname - path = result.example.source.path - file = File.basename(path) - ext = File.extname(file) - name = file[0...-(ext.size)] - dir = path[0...-(file.size + 1)] - {dir.gsub('/', '.').underscore, name.camelcase}.join('.') - end - end -end diff --git a/src/spectator/formatting/junit_test_suite.cr b/src/spectator/formatting/junit_test_suite.cr deleted file mode 100644 index 9abca41..0000000 --- a/src/spectator/formatting/junit_test_suite.cr +++ /dev/null @@ -1,73 +0,0 @@ -module Spectator::Formatting - # Mapping of a single spec file into a JUnit test suite. - private struct JUnitTestSuite - # Creates the JUnit test suite. - # The *path* should be the file that all results are from. - # The *results* is a subset of all results that share the path. - def initialize(@path : String, results : Array(Result)) - @report = Report.new(results) - end - - # Generates the XML for the test suite (and all nested test cases). - def to_xml(xml : ::XML::Builder) - xml.element("testsuite", - tests: @report.example_count, - failures: @report.failed_count, - errors: @report.error_count, - skipped: @report.pending_count, - time: @report.runtime.total_seconds, - name: name, - package: package) do - add_test_cases(xml) - end - end - - # Adds the test case elements to the XML. - private def add_test_cases(xml) - @report.each do |result| - test_case = result.call(JUnitTestCaseSelector) { |r| r } - test_case.to_xml(xml) - end - end - - # Java-ified name of the test suite. - private def name - file = File.basename(@path) - ext = File.extname(file) - name = file[0...-(ext.size)] - name.camelcase - end - - # Java-ified package (path) of the test suite. - private def package - file = File.basename(@path) - dir = @path[0...-(file.size + 1)] - dir.gsub('/', '.').underscore - end - - # Selector for creating a JUnit test case based on a result. - private module JUnitTestCaseSelector - extend self - - # Creates a successful JUnit test case. - def success(result) - SuccessfulJUnitTestCase.new(result.as(SuccessfulResult)) - end - - # Creates a failure JUnit test case. - def failure(result) - FailureJUnitTestCase.new(result.as(FailedResult)) - end - - # Creates an error JUnit test case. - def error(result) - ErrorJUnitTestCase.new(result.as(ErroredResult)) - end - - # Creates a skipped JUnit test case. - def pending(result) - SkippedJUnitTestCase.new(result.as(PendingResult)) - end - end - end -end diff --git a/src/spectator/formatting/labeled_text.cr b/src/spectator/formatting/labeled_text.cr deleted file mode 100644 index b99152e..0000000 --- a/src/spectator/formatting/labeled_text.cr +++ /dev/null @@ -1,15 +0,0 @@ -module Spectator::Formatting - # Produces a stringified message with a prefix. - private struct LabeledText(T) - # Creates the labeled text. - def initialize(@label : String, @text : T) - end - - # Appends the message to the output. - def to_s(io) - io << @label - io << ": " - io << @text - end - end -end diff --git a/src/spectator/formatting/match_data_value_pair.cr b/src/spectator/formatting/match_data_value_pair.cr deleted file mode 100644 index dd7f255..0000000 --- a/src/spectator/formatting/match_data_value_pair.cr +++ /dev/null @@ -1,16 +0,0 @@ -module Spectator::Formatting - # A single labeled value from the `Spectator::Matchers::MatchData#value` method. - private struct MatchDataValuePair - # Creates the pair formatter. - def initialize(@key : Symbol, @value : String, @padding : Int32) - end - - # Appends the pair to the output. - def to_s(io) - @padding.times { io << ' ' } - io << @key - io << ": " - io << @value - end - end -end diff --git a/src/spectator/formatting/match_data_values.cr b/src/spectator/formatting/match_data_values.cr deleted file mode 100644 index bf194a9..0000000 --- a/src/spectator/formatting/match_data_values.cr +++ /dev/null @@ -1,24 +0,0 @@ -module Spectator::Formatting - # Produces a `MatchDataValuePair` for each key-value pair - # from `Spectator::Matchers::MatchData#values`. - private struct MatchDataValues - include Enumerable(Tuple(Symbol, String)) - - @max_key_length : Int32 - - # Creates the values mapper. - def initialize(@values : Array(Tuple(Symbol, String))) - @max_key_length = @values.map(&.first.to_s.size).max - end - - # Yields pairs that can be printed to output. - def each - @values.each do |labeled_value| - key = labeled_value.first - key_length = key.to_s.size - padding = @max_key_length - key_length - yield MatchDataValuePair.new(key, labeled_value.last, padding) - end - end - end -end diff --git a/src/spectator/formatting/notifications.cr b/src/spectator/formatting/notifications.cr new file mode 100644 index 0000000..e789514 --- /dev/null +++ b/src/spectator/formatting/notifications.cr @@ -0,0 +1,23 @@ +require "../example" +require "../profile" +require "../report" + +module Spectator::Formatting + # Structure indicating the test suite has started. + record StartNotification, example_count : Int32 + + # Structure indicating an event occurred with an example. + record ExampleNotification, example : Example + + # Structure containing a subset of examples from the test suite. + record ExampleSummaryNotification, examples : Enumerable(Example) + + # Structure containing profiling information. + record ProfileNotification, profile : Profile + + # Structure containing summarized information from the outcome of the test suite. + record SummaryNotification, report : Report + + # Structure containing a debug or log message from the test suite. + record MessageNotification, message : String +end diff --git a/src/spectator/formatting/numbered_item.cr b/src/spectator/formatting/numbered_item.cr deleted file mode 100644 index 15afd16..0000000 --- a/src/spectator/formatting/numbered_item.cr +++ /dev/null @@ -1,16 +0,0 @@ -module Spectator::Formatting - # Produces a stringified value with a numerical prefix. - private struct NumberedItem(T) - # Creates the numbered item. - def initialize(@number : Int32, @text : T) - end - - # Appends the numbered item to the output. - def to_s(io) - io << @number - io << ')' - io << ' ' - io << @text - end - end -end diff --git a/src/spectator/formatting/profile_block.cr b/src/spectator/formatting/profile_block.cr deleted file mode 100644 index d17cf86..0000000 --- a/src/spectator/formatting/profile_block.cr +++ /dev/null @@ -1,28 +0,0 @@ -module Spectator::Formatting - # Contents of a profile block. - private struct ProfileBlock - # Creates the block. - def initialize(@profile : Profile) - end - - # Appends the block to the output. - def to_s(io) - io.puts(ProfileSummary.new(@profile)) - - indent = Indent.new(io) - indent.increase do - @profile.each do |result| - entry(indent, result) - end - end - end - - # Adds a result entry to the output. - private def entry(indent, result) - indent.line(result.example) - indent.increase do - indent.line(SourceTiming.new(result.elapsed, result.example.source)) - end - end - end -end diff --git a/src/spectator/formatting/profile_summary.cr b/src/spectator/formatting/profile_summary.cr deleted file mode 100644 index 9d6a05c..0000000 --- a/src/spectator/formatting/profile_summary.cr +++ /dev/null @@ -1,29 +0,0 @@ -module Spectator::Formatting - # Top line of a profile block which gives a summary. - private struct ProfileSummary - # Creates the summary line. - def initialize(@profile : Profile) - end - - # Appends the summary to the output. - def to_s(io) - io << "Top " - io << @profile.size - io << " slowest examples (" - io << human_time - io << ", " - io.printf("%.2f", percentage) - io << "% of total time):" - end - - # Creates a human-friendly string for the total time. - private def human_time - HumanTime.new(@profile.total_time) - end - - # Percentage (0 to 100) of total time. - private def percentage - @profile.percentage * 100 - end - end -end diff --git a/src/spectator/formatting/progress_formatter.cr b/src/spectator/formatting/progress_formatter.cr new file mode 100644 index 0000000..20975f4 --- /dev/null +++ b/src/spectator/formatting/progress_formatter.cr @@ -0,0 +1,48 @@ +require "colorize" +require "./formatter" +require "./summary" + +module Spectator::Formatting + # Output formatter that produces a single character for each test as it completes. + # A '.' indicates a pass, 'F' a failure, 'E' an error, and '*' a skipped or pending test. + class ProgressFormatter < Formatter + include Summary + + @pass_char : Colorize::Object(Char) = '.'.colorize(:green) + @fail_char : Colorize::Object(Char) = 'F'.colorize(:red) + @error_char : Colorize::Object(Char) = 'E'.colorize(:red) + @skip_char : Colorize::Object(Char) = '*'.colorize(:yellow) + + # Output stream to write results to. + private getter io + + # Creates the formatter. + def initialize(@io : IO = STDOUT) + end + + # Produces a pass character. + def example_passed(_notification) + @pass_char.to_s(@io) + end + + # Produces a fail character. + def example_failed(_notification) + @fail_char.to_s(@io) + end + + # Produces an error character. + def example_error(_notification) + @error_char.to_s(@io) + end + + # Produces a skip character. + def example_pending(_notification) + @skip_char.to_s(@io) + end + + # Produces a new line after the tests complete. + def stop + @io.puts + end + end +end diff --git a/src/spectator/formatting/random_seed_text.cr b/src/spectator/formatting/random_seed_text.cr deleted file mode 100644 index 998fed1..0000000 --- a/src/spectator/formatting/random_seed_text.cr +++ /dev/null @@ -1,14 +0,0 @@ -module Spectator::Formatting - # Text displayed when using a random seed. - private struct RandomSeedText - # Creates the text object. - def initialize(@seed : UInt64) - end - - # Appends the command to the output. - def to_s(io) - io << "Randomized with seed " - io << @seed - end - end -end diff --git a/src/spectator/formatting/remaining_text.cr b/src/spectator/formatting/remaining_text.cr deleted file mode 100644 index b3639ec..0000000 --- a/src/spectator/formatting/remaining_text.cr +++ /dev/null @@ -1,15 +0,0 @@ -module Spectator::Formatting - # Text displayed when fail-fast is enabled and tests were skipped. - private struct RemainingText - # Creates the text object. - def initialize(@count : Int32) - end - - # Appends the command to the output. - def to_s(io) - io << "Text execution aborted (fail-fast) - " - io << @count - io << " examples were omitted." - end - end -end diff --git a/src/spectator/formatting/runtime.cr b/src/spectator/formatting/runtime.cr deleted file mode 100644 index 95541e3..0000000 --- a/src/spectator/formatting/runtime.cr +++ /dev/null @@ -1,24 +0,0 @@ -module Spectator::Formatting - # Produces a stringified time span for the runtime. - private struct Runtime - # Creates the runtime instance. - def initialize(@runtime : Time::Span) - end - - # Appends the runtime to the output. - # The text will be formatted as follows, - # depending on the length of time: - # ```text - # Finished in ## microseconds - # Finished in ## milliseconds - # Finished in ## seconds - # Finished in #:## - # Finished in #:##:## - # Finished in # days #:##:## - # ``` - def to_s(io) - io << "Finished in " - io << HumanTime.new(@runtime) - end - end -end diff --git a/src/spectator/formatting/silent_formatter.cr b/src/spectator/formatting/silent_formatter.cr deleted file mode 100644 index 2098480..0000000 --- a/src/spectator/formatting/silent_formatter.cr +++ /dev/null @@ -1,27 +0,0 @@ -module Spectator::Formatting - # Formatter that outputs nothing. - # Useful for testing and larger automated processes. - class SilentFormatter < Formatter - # Called when a test suite is starting to execute. - def start_suite(suite : TestSuite) - # ... crickets ... - end - - # Called when a test suite finishes. - # The results from the entire suite are provided. - def end_suite(report : Report, profile : Profile?) - # ... crickets ... - end - - # Called before a test starts. - def start_example(example : Example) - # ... crickets ... - end - - # Called when a test finishes. - # The result of the test is provided. - def end_example(result : Result) - # ... crickets ... - end - end -end diff --git a/src/spectator/formatting/skipped_junit_test_case.cr b/src/spectator/formatting/skipped_junit_test_case.cr deleted file mode 100644 index 6b90b4d..0000000 --- a/src/spectator/formatting/skipped_junit_test_case.cr +++ /dev/null @@ -1,24 +0,0 @@ -require "./junit_test_case" - -module Spectator::Formatting - # JUnit test case for a pending result. - private class SkippedJUnitTestCase < JUnitTestCase - # Result for this test case. - private getter result - - # Creates the JUnit test case. - def initialize(@result : PendingResult) - end - - # Status string specific to the result type. - private def status : String - "TODO" - end - - # Adds the skipped tag to the XML block. - private def content(xml) - super - xml.element("skipped") - end - end -end diff --git a/src/spectator/formatting/source_timing.cr b/src/spectator/formatting/source_timing.cr deleted file mode 100644 index 527fa17..0000000 --- a/src/spectator/formatting/source_timing.cr +++ /dev/null @@ -1,16 +0,0 @@ -module Spectator::Formatting - # Produces the timing line in a profile block. - # This contains the length of time, and the example's source. - private struct SourceTiming - # Creates the source timing line. - def initialize(@span : Time::Span, @source : Source) - end - - # Appends the source timing information to the output. - def to_s(io) - io << HumanTime.new(@span).colorize.bold - io << ' ' - io << @source - end - end -end diff --git a/src/spectator/formatting/stats_counter.cr b/src/spectator/formatting/stats_counter.cr deleted file mode 100644 index 714c63a..0000000 --- a/src/spectator/formatting/stats_counter.cr +++ /dev/null @@ -1,50 +0,0 @@ -module Spectator::Formatting - # Produces a stringified stats counter from result totals. - private struct StatsCounter - # Creates the instance with each of the counters. - private def initialize(@examples : Int32, @failures : Int32, @errors : Int32, @pending : Int32, @failed : Bool) - end - - # Creates the instance from the counters in a report. - def initialize(report) - initialize(report.example_count, report.failed_count, report.error_count, report.pending_count, report.failed?) - end - - # Produces a colorized formatting for the stats, - # depending on the number of each type of result. - def color - if @errors > 0 - Color.error(self) - elsif @failed || @failures > 0 - Color.failure(self) - elsif @pending > 0 - Color.pending(self) - else - Color.success(self) - end - end - - # Appends the counters to the output. - # The format will be: - # ```text - # # examples, # failures, # errors, # pending - # ``` - def to_s(io) - stats.each_with_index do |stat, value, index| - io << ", " if index > 0 - io << value - io << ' ' - io << stat - end - end - - private def stats - { - examples: @examples, - failures: @failures, - errors: @errors, - pending: @pending, - } - end - end -end diff --git a/src/spectator/formatting/successful_junit_test_case.cr b/src/spectator/formatting/successful_junit_test_case.cr deleted file mode 100644 index 108e966..0000000 --- a/src/spectator/formatting/successful_junit_test_case.cr +++ /dev/null @@ -1,16 +0,0 @@ -module Spectator::Formatting - # JUnit test case for a successful result. - private class SuccessfulJUnitTestCase < FinishedJUnitTestCase - # Result for this test case. - private getter result - - # Creates the JUnit test case. - def initialize(@result : SuccessfulResult) - end - - # Status string specific to the result type. - private def status : String - "PASS" - end - end -end diff --git a/src/spectator/formatting/suite_summary.cr b/src/spectator/formatting/suite_summary.cr deleted file mode 100644 index 183a78e..0000000 --- a/src/spectator/formatting/suite_summary.cr +++ /dev/null @@ -1,78 +0,0 @@ -module Spectator::Formatting - # Mix-in for producing a human-readable summary of a test suite. - module SuiteSummary - # Does nothing when starting a test suite. - def start_suite(suite) - # ... - end - - # Produces the summary of test suite from a report. - # A block describing each failure is displayed. - # At the end, the totals and runtime are printed. - # The *profile* value is not nil when profiling results should be displayed. - def end_suite(report, profile : Profile?) - if report.example_count > 0 - @io.puts if is_a?(DotsFormatter) - @io.puts - end - failures(report.failures) if report.failed_count > 0 - profile(profile) if profile - stats(report) - remaining(report) if report.remaining? - if report.failed? - if report.examples_ran > 0 - failure_commands(report.failures) - else - @io.puts Color.failure("Failing because no tests were run (fail-blank)") - end - end - end - - # Produces the failure section of the summary. - # This has a "Failures" title followed by a block for each failure. - private def failures(failures) - @io.puts "Failures:" - @io.puts - failures.each_with_index do |result, index| - @io.puts FailureBlock.new(index + 1, result) - end - end - - # Produces the profiling section of the summary. - private def profile(profile) - @io.puts ProfileBlock.new(profile) - end - - # Produces the statistical section of the summary. - # This contains how long the suite took to run - # and the counts for the results (total, failures, errors, and pending). - private def stats(report) - @io.puts Runtime.new(report.runtime) - @io.puts StatsCounter.new(report).color - if (seed = report.random_seed?) - @io.puts - @io.puts RandomSeedText.new(seed) - end - end - - # Produces the skipped tests text if fail-fast is enabled and tests were omitted. - private def remaining(report) - text = RemainingText.new(report.remaining_count) - @io.puts Color.failure(text) - end - - # Produces the failure commands section of the summary. - # This provides a set of commands the user can run - # to test just the examples that failed. - private def failure_commands(failures) - @io.puts - @io.puts "Failed examples:" - @io.puts - failures.each do |result| - @io << FailureCommand.color(result) - @io << ' ' - @io.puts Comment.color(result.example) - end - end - end -end diff --git a/src/spectator/formatting/summary.cr b/src/spectator/formatting/summary.cr new file mode 100644 index 0000000..50a2b1b --- /dev/null +++ b/src/spectator/formatting/summary.cr @@ -0,0 +1,86 @@ +require "../fail_result" +require "./components" + +module Spectator::Formatting + # Mix-in providing common output for summarized results. + # Implements the following methods: + # `Formatter#start_dump`, `Formatter#dump_pending`, `Formatter#dump_failures`, + # `Formatter#dump_summary`, and `Formatter#dump_profile`. + # Classes including this module must implement `#io`. + module Summary + # Stream to write results to. + private abstract def io + + def start_dump + io.puts + end + + # Invoked after testing completes with a list of pending examples. + # This method will be called with an empty list if there were no pending (skipped) examples. + # Called after `#start_dump` and before `#dump_failures`. + def dump_pending(notification) + return if (examples = notification.examples).empty? + + io.puts "Pending:" + io.puts + examples.each_with_index(1) do |example, index| + result = example.result.as(PendingResult) + io.puts Components::PendingResultBlock.new(example, index, result) + end + end + + # Invoked after testing completes with a list of failed examples. + # This method will be called with an empty list if there were no failures. + # Called after `#dump_pending` and before `#dump_summary`. + def dump_failures(notification) + return if (examples = notification.examples).empty? + + io.puts "Failures:" + io.puts + examples.each_with_index(1) do |example, index| + dump_failed_example(example, index) + end + end + + # Invoked after testing completes with profiling information. + # This method is only called if profiling is enabled. + # Called after `#dump_failures` and before `#dump_summary`. + def dump_profile(notification) + io.puts Components::Profile.new(notification.profile) + end + + # Invoked after testing completes with summarized information from the test suite. + # Called after `#dump_failures` and before `#dump_profile`. + def dump_summary(notification) + report = notification.report + io.puts Components::Stats.new(report) + + return if (failures = report.failures).empty? + + io.puts Components::FailureCommandList.new(failures) + end + + # Displays one or more blocks for a failed example. + # Each block is a failed expectation or error raised in the example. + private def dump_failed_example(example, index) + result = example.result.as?(ErrorResult) + failed_expectations = example.result.expectations.select(&.failed?) + block_count = failed_expectations.size + block_count += 1 if result + + # Don't use sub-index if there was only one problem. + if block_count == 1 + if result + io.puts Components::ErrorResultBlock.new(example, index, result) + else + io.puts Components::FailResultBlock.new(example, index, failed_expectations.first) + end + else + failed_expectations.each_with_index(1) do |expectation, subindex| + io.puts Components::FailResultBlock.new(example, index, expectation, subindex) + end + io.puts Components::ErrorResultBlock.new(example, index, result, block_count) if result + end + end + end +end diff --git a/src/spectator/formatting/tap_formatter.cr b/src/spectator/formatting/tap_formatter.cr index 816da4f..e5991a3 100644 --- a/src/spectator/formatting/tap_formatter.cr +++ b/src/spectator/formatting/tap_formatter.cr @@ -1,58 +1,71 @@ +require "./formatter" + module Spectator::Formatting - # Formatter for the "Test Anything Protocol". - # For details, see: https://testanything.org/ + # Produces TAP output from test results. + # See: https://testanything.org/ + # Version 12 of the specification is used. class TAPFormatter < Formatter + @counter = 0 + # Creates the formatter. - # By default, output is sent to STDOUT. def initialize(@io : IO = STDOUT) - @index = 1 end - # Called when a test suite is starting to execute. - def start_suite(suite : TestSuite) + # Invoked when the test suite begins. + def start(notification) @io << "1.." - @io.puts suite.size + @io.puts notification.example_count end - # Called when a test suite finishes. - # The results from the entire suite are provided. - # The *profile* value is not nil when profiling results should be displayed. - def end_suite(report : Report, profile : Profile?) - @io.puts "Bail out!" if report.remaining? - profile(profile) if profile + # Invoked just after an example completes. + def example_finished(_notification) + @counter += 1 end - # Called before a test starts. - def start_example(example : Example) + # Invoked after an example completes successfully. + def example_passed(notification) + @io << "ok " << @counter << " - " + @io.puts notification.example end - # Called when a test finishes. - # The result of the test is provided. - def end_example(result : Result) - @io.puts TAPTestLine.new(@index, result) - @index += 1 - end + # Invoked after an example is skipped or marked as pending. + def example_pending(notification) + # TODO: Skipped tests should report ok. + @io << "not ok " << @counter << " - " + @io << notification.example << " # TODO " - # Displays profiling information. - private def profile(profile) - @io.puts(Comment.new(ProfileSummary.new(profile))) - - indent = Indent.new(@io) - indent.increase do - profile.each do |result| - profile_entry(indent, result) - end + # This should never be false. + if (result = notification.example.result).responds_to?(:reason) + @io.puts result.reason end end - # Adds a profile result entry to the output. - private def profile_entry(indent, result) + # Invoked after an example fails. + def example_failed(notification) + @io << "not ok " << @counter << " - " + @io.puts notification.example + end + + # Invoked after an example fails from an unexpected error. + def example_error(notification) + example_failed(notification) + end + + # Called whenever the example or framework produces a message. + # This is typically used for logging. + def message(notification) @io << "# " - indent.line(result.example) - indent.increase do - @io << "# " - indent.line(SourceTiming.new(result.elapsed, result.example.source)) - end + @io.puts notification.message + end + + # Invoked after testing completes with profiling information. + def dump_profile(notification) + @io << Components::TAPProfile.new(notification.profile) + end + + # Invoked after testing completes with summarized information from the test suite. + def dump_summary(notification) + @io.puts "Bail out!" if notification.report.counts.remaining? end end end diff --git a/src/spectator/formatting/tap_test_line.cr b/src/spectator/formatting/tap_test_line.cr deleted file mode 100644 index 8c0e823..0000000 --- a/src/spectator/formatting/tap_test_line.cr +++ /dev/null @@ -1,33 +0,0 @@ -module Spectator::Formatting - # Produces a formatted TAP test line. - private struct TAPTestLine - # Creates the test line. - def initialize(@index : Int32, @result : Result) - end - - # Appends the line to the output. - def to_s(io) - io << status - io << ' ' - io << @index - io << " - " - io << example - io << " # skip" if pending? - end - - # The text "ok" or "not ok" depending on the result. - private def status - @result.is_a?(FailedResult) ? "not ok" : "ok" - end - - # The example that was tested. - private def example - @result.example - end - - # Indicates whether this test was skipped. - private def pending? - @result.is_a?(PendingResult) - end - end -end diff --git a/src/spectator/harness.cr b/src/spectator/harness.cr index 5268edc..3e0962b 100644 --- a/src/spectator/harness.cr +++ b/src/spectator/harness.cr @@ -1,78 +1,176 @@ -require "./mocks/registry" +require "./error_result" +require "./example_failed" +require "./example_pending" +require "./expectation" +require "./expectation_failed" +require "./mocks" +require "./multiple_expectations_failed" +require "./pass_result" +require "./result" module Spectator - # Helper class that acts as a gateway between example code and the test framework. - # Every example must be invoked by passing it to `#run`. - # This sets up the harness so that the example code can use it. - # The test framework does the following: + # Helper class that acts as a gateway between test code and the framework. + # + # Test code should be wrapped with a call to `.run`. + # This class will catch all errors raised by the test code. + # Errors caused by failed assertions (`AssertionFailed`) are translated to failed results (`FailResult`). + # Errors not caused by assertions are translated to error results (`ErrorResult`). + # + # Every runnable example should invoke the test code by calling `.run`. + # This sets up the harness so that the test code can use it. + # The framework does the following: # ``` - # result = Harness.run(example) + # result = Harness.run { run_example_code } # # Do something with the result. # ``` - # Then from the example code, the harness can be accessed via `#current` like so: + # + # Then from the test code, the harness can be accessed via `.current` like so: # ``` # harness = ::Spectator::Harness.current # # Do something with the harness. # ``` + # # Of course, the end-user shouldn't see this or work directly with the harness. - # Instead, methods the user calls can access it. + # Instead, methods the test calls can access it. # For instance, an expectation reporting a result. class Harness + Log = ::Spectator::Log.for(self) + # Retrieves the harness for the current running example. class_getter! current : self - # Wraps an example with a harness and runs the example. - # The `#current` harness will be set - # prior to running the example, and reset after. - # The *example* argument will be the example to run. - # The result returned from `Example#run` will be returned. - def self.run(example : Example) : Result - @@current = new(example) - example.run - ensure - @@current = nil + getter mocks = Mocks::Registry.new + + # Wraps an example with a harness and runs test code. + # A block provided to this method is considered to be the test code. + # The value of `.current` is set to the harness for the duration of the test. + # It will be reset after the test regardless of the outcome. + # The result of running the test code will be returned. + def self.run : Result + with_harness do |harness| + harness.run { yield } + end end - # Retrieves the current running example. - getter example : Example - - getter mocks : Mocks::Registry - - # Retrieves the group for the current running example. - def group - example.group + # Instanciates a new harness and yields it. + # The `.current` harness is set to the new harness for the duration of the block. + # `.current` is reset to the previous value (probably nil) afterwards, even if the block raises. + # The result of the block is returned. + private def self.with_harness + previous = @@current + begin + @@current = harness = new + yield harness + ensure + @@current = previous + end end - # Reports the outcome of an expectation. - # An exception will be raised when a failing result is given. - def report_expectation(expectation : Expectations::Expectation) : Nil - @example.description = expectation.description unless @example.test_wrapper.description? - @reporter.report(expectation) + @deferred = Deque(->).new + @expectations = [] of Expectation + @aggregate : Array(Expectation)? = nil + + # Runs test code and produces a result based on the outcome. + # The test code should be called from within the block given to this method. + def run : Result + elapsed, error = capture { yield } + elapsed2, error2 = capture { run_deferred } + translate(elapsed + elapsed2, error || error2) end - # Generates the reported expectations from the example. - # This should be run after the example has finished. - def expectations : Expectations::ExampleExpectations - @reporter.expectations + def report(expectation : Expectation) : Bool + Log.debug { "Reporting expectation #{expectation}" } + @expectations << expectation + + # TODO: Move this out of harness, maybe to `Example`. + Example.current.name = expectation.description unless Example.current.name? + + if expectation.failed? + raise ExpectationFailed.new(expectation, expectation.failure_message) unless (aggregate = @aggregate) + aggregate << expectation + false + else + true + end end - # Marks a block of code to run later. - def defer(&block : ->) : Nil + # Stores a block of code to be executed later. + # All deferred blocks run just before the `#run` method completes. + def defer(&block) : Nil @deferred << block end - # Runs all deferred blocks. - def run_deferred : Nil - @deferred.each(&.call) - @deferred.clear + def aggregate_failures(label = nil) + previous = @aggregate + @aggregate = aggregate = [] of Expectation + begin + yield.tap do + # If there's an nested aggregate (for some reason), allow the top-level one to handle things. + check_aggregate(aggregate, label) unless previous + end + ensure + @aggregate = previous + end end - # Creates a new harness. - # The example the harness is for should be passed in. - private def initialize(@example) - @reporter = Expectations::ExpectationReporter.new - @mocks = Mocks::Registry.new(@example.group.context) - @deferred = Deque(->).new + private def check_aggregate(aggregate, label) + failures = aggregate.select(&.failed?) + case failures.size + when 0 then return + when 1 + expectation = failures.first + raise ExpectationFailed.new(expectation, expectation.failure_message) + else + message = "Got #{failures.size} failures from failure aggregation block" + message += " \"#{label}\"" if label + raise MultipleExpectationsFailed.new(failures, message) + end + end + + # Yields to run the test code and returns information about the outcome. + # Returns a tuple with the elapsed time and an error if one occurred (otherwise nil). + private def capture : Tuple(Time::Span, Exception?) + error = nil + elapsed = Time.measure do + error = catch { yield } + end + error = nil if error.is_a?(SystemExit) && mocks.exit_handled? + {elapsed, error} + end + + # Yields to run a block of code and captures exceptions. + # If the block of code raises an error, the error is caught and returned. + # If the block doesn't raise an error, then nil is returned. + private def catch : Exception? + yield + rescue e + e + else + nil + end + + # Translates the outcome of running a test to a result. + # Takes the *elapsed* time and a possible *error* from the test. + # Returns a type of `Result`. + private def translate(elapsed, error) : Result + case error + when nil + PassResult.new(elapsed, @expectations) + when ExampleFailed + FailResult.new(elapsed, error, @expectations) + when ExamplePending + PendingResult.new(error.message || PendingResult::DEFAULT_REASON, error.location, elapsed, @expectations) + else + ErrorResult.new(elapsed, error, @expectations) + end + end + + # Runs all deferred blocks. + # This method executes code from tests and may raise an error. + # It should be wrapped in a call to `#capture`. + private def run_deferred + Log.debug { "Running deferred operations" } + @deferred.each(&.call) end end end diff --git a/src/spectator/hooks.cr b/src/spectator/hooks.cr new file mode 100644 index 0000000..f06b0e1 --- /dev/null +++ b/src/spectator/hooks.cr @@ -0,0 +1,83 @@ +module Spectator + # Mix-in for defining hook methods. + module Hooks + # Defines various methods for adding hooks of a specific type. + # + # The *declaration* defines the name and type of hook. + # It should be a type declaration in the form: `some_hook : ExampleHook`, + # where `some_hook` is the name of the hook, and `ExampleHook` is type type. + # + # A default order can be specified by *order*. + # The *order* argument must be *append* or *prepend*. + # This indicates the order hooks are added by default when called by client code. + # + # Multiple methods are generated. + # The primary methods will be named the same as the hook (from *declaration*). + # These take a pre-built hook instance, or arguments to pass to the hook type's initializer. + # The new hook is added a collection in the order specified by *order*. + # + # A private getter method is created so that the hooks can be accessed if needed. + # The getter method has `_hooks` appended to the hook name. + # For instance, if the *declaration* contains `important_thing`, then the getter is `important_thing_hooks`. + # + # Lastly, an optional block can be provided. + # If given, a protected method will be defined with the block's contents. + # This method typically operates on (calls) the hooks. + # The private getter method mentioned above can be used to access the hooks. + # Any block arguments will be used as argument in the method. + # The method name has the prefix `call_` followed by the hook name. + # + # ``` + # define_hook important_event : ImportantHook do |example| + # important_event_hooks.each &.call(example) + # end + # + # # ... + # + # important_event do |example| + # puts "An important event occurred for #{example}" + # end + # ``` + macro define_hook(declaration, order = :append, &block) + {% if order.id == :append.id + method = :push.id + elsif order.id == :prepend.id + method = :unshift.id + else + raise "Unknown hook order type - #{order}" + end %} + + # Retrieves all registered hooks for {{declaration.var}}. + protected getter {{declaration.var}}_hooks = Deque({{declaration.type}}).new + + # Registers a new "{{declaration.var}}" hook. + # The hook will be {{order.id}}ed to the list. + def {{declaration.var}}(hook : {{declaration.type}}) : Nil + @{{declaration.var}}_hooks.{{method}}(hook) + end + + # Registers a new "{{declaration.var}}" hook. + # The hook will be {{order.id}}ed to the list. + # A new hook will be created by passing args to `{{declaration.type}}.new`. + def {{declaration.var}}(*args, **kwargs) : Nil + hook = {{declaration.type}}.new(*args, **kwargs) + {{declaration.var}}(hook) + end + + # Registers a new "{{declaration.var}}" hook. + # The hook will be {{order.id}}ed to the list. + # A new hook will be created by passing args to `{{declaration.type}}.new`. + def {{declaration.var}}(*args, **kwargs, &block) : Nil + hook = {{declaration.type}}.new(*args, **kwargs, &block) + {{declaration.var}}(hook) + end + + {% if block %} + # Handles calling all "{{declaration.var}}" hooks. + protected def call_{{declaration.var}}({{block.args.splat}}) + {{block.body}} + end + {% end %} + end + end +end diff --git a/src/spectator/includes.cr b/src/spectator/includes.cr index 894a045..d649827 100644 --- a/src/spectator/includes.cr +++ b/src/spectator/includes.cr @@ -4,52 +4,54 @@ # Including all files with a wildcard would accidentally enable should-syntax. # Unfortunately, that leads to the existence of this file to include everything but that file. -# First the sub-modules. -require "./dsl" -require "./expectations" -require "./matchers" -require "./formatting" - -# Then all of the top-level types. -require "./spec_builder" -require "./example_component" -require "./example" -require "./runnable_example" -require "./pending_example" - -require "./example_conditions" -require "./example_hooks" -require "./example_group" -require "./nested_example_group" -require "./root_example_group" - -require "./mocks" - +require "./abstract_expression" +require "./anything" +require "./block" +require "./composite_node_filter" require "./config" -require "./config_builder" -require "./config_source" -require "./command_line_arguments_config_source" - -require "./example_filter" -require "./source_example_filter" -require "./line_example_filter" -require "./name_example_filter" -require "./null_example_filter" -require "./composite_example_filter" - -require "./example_failed" -require "./expectation_failed" -require "./test_suite" -require "./report" -require "./profile" -require "./runner" - -require "./result" -require "./finished_result" -require "./successful_result" -require "./pending_result" -require "./failed_result" -require "./errored_result" - -require "./source" +require "./context" +require "./context_delegate" +require "./context_method" +require "./dsl" +require "./error_result" +require "./example_context_delegate" +require "./example_context_method" +require "./example" +require "./node_filter" +require "./example_group" +require "./example_group_hook" +require "./example_hook" require "./example_iterator" +require "./example_procsy_hook" +require "./expectation" +require "./expectation_failed" +require "./expression" +require "./fail_result" +require "./formatting" +require "./harness" +require "./hooks" +require "./label" +require "./lazy" +require "./lazy_wrapper" +require "./line_node_filter" +require "./location" +require "./location_node_filter" +require "./matchers" +require "./metadata" +require "./mocks" +require "./name_node_filter" +require "./null_context" +require "./null_node_filter" +require "./pass_result" +require "./pending_result" +require "./profile" +require "./report" +require "./result" +require "./runner_events" +require "./runner" +require "./spec_builder" +require "./spec" +require "./tag_node_filter" +require "./test_context" +require "./value" +require "./wrapper" diff --git a/src/spectator/iterative_example_group_builder.cr b/src/spectator/iterative_example_group_builder.cr new file mode 100644 index 0000000..6bc866b --- /dev/null +++ b/src/spectator/iterative_example_group_builder.cr @@ -0,0 +1,48 @@ +require "./example_group" +require "./example_group_builder" +require "./example_group_iteration" +require "./location" +require "./metadata" + +module Spectator + # Progressively constructs an iterative example group. + # Hooks and builders for child nodes can be added over time to this builder. + # When done, call `#build` to produce an `ExampleGroup` with nested `ExampleGroupIteration` instances. + class IterativeExampleGroupBuilder(T) < ExampleGroupBuilder + # Creates the builder. + # Initially, the builder will have no children and no hooks. + # The *name*, *location*, and *metadata* will be applied to the `ExampleGroup` produced by `#build`. + # The *collection* is the set of items to create sub-nodes for. + # The *iterator* is an optional name given to a single item in the collection. + def initialize(@collection : Enumerable(T), name : String? = nil, @iterator : String? = nil, + location : Location? = nil, metadata : Metadata = Metadata.new) + super(name, location, metadata) + end + + # Constructs an iterative example group with previously defined attributes, children, and hooks. + # The *parent* is an already constructed example group to nest the new example group under. + # It can be nil if the new example group won't have a parent. + def build(parent = nil) + ExampleGroup.new(@name, @location, parent, @metadata).tap do |group| + # Hooks are applied once to the outer group, + # instead of multiple times for each inner group (iteration). + apply_hooks(group) + + @collection.each do |item| + ExampleGroupIteration.new(item, iteration_name(item), @location, group).tap do |iteration| + @children.each(&.build(iteration)) + end + end + end + end + + # Constructs the name of an example group iteration. + private def iteration_name(item) + if iterator = @iterator + "#{iterator}: #{item.inspect}" + else + item.inspect + end + end + end +end diff --git a/src/spectator/label.cr b/src/spectator/label.cr new file mode 100644 index 0000000..2a86b34 --- /dev/null +++ b/src/spectator/label.cr @@ -0,0 +1,7 @@ +module Spectator + # Identifier used in the spec. + # Signficant to the user. + # When a label is a symbol, then it is referencing a type or method. + # A label is nil when one can't be provided or captured. + alias Label = String | Symbol | Nil +end diff --git a/src/spectator/lazy.cr b/src/spectator/lazy.cr new file mode 100644 index 0000000..7a477b3 --- /dev/null +++ b/src/spectator/lazy.cr @@ -0,0 +1,34 @@ +module Spectator + # Lazily stores a value. + struct Lazy(T) + @value : Value(T)? + + # Retrieves the value, if it was previously fetched. + # On the first invocation of this method, it will yield. + # The block should return the value to store. + # Subsequent calls will return the same value and not yield. + def get(&block : -> T) + if (existing = @value) + existing.get + else + yield.tap do |value| + @value = Value.new(value) + end + end + end + + # Wrapper for a value. + # This is intended to be used as a union with nil. + # It allows storing (caching) a nillable value. + private struct Value(T) + # Creates the wrapper. + def initialize(@value : T) + end + + # Retrieves the value. + def get : T + @value + end + end + end +end diff --git a/src/spectator/lazy_wrapper.cr b/src/spectator/lazy_wrapper.cr new file mode 100644 index 0000000..9bc7125 --- /dev/null +++ b/src/spectator/lazy_wrapper.cr @@ -0,0 +1,36 @@ +require "./lazy" +require "./wrapper" + +module Spectator + # Lazily stores a value of any type. + # Combines `Lazy` and `Wrapper`. + # + # Contains no value until the first call to `#get` is made. + # Any type can be stored in this wrapper. + # However, the type must always be known when retrieving it via `#get`. + # The type is inferred from the block, and all blocks must return the same type. + # Because of this, it is recommended to only have `#get` called in one location. + # + # This type is expected to be used like so: + # ``` + # @wrapper : LazyWrapper + # + # # ... + # + # def lazy_load + # @wrapper.get { some_expensive_operation } + # end + # ``` + struct LazyWrapper + @lazy = Lazy(Wrapper).new + + # Retrieves the value, if it was previously fetched. + # On the first invocation of this method, it will yield. + # The block should return the value to store. + # Subsequent calls will return the same value and not yield. + def get(& : -> T) : T forall T + wrapper = @lazy.get { Wrapper.new(yield) } + wrapper.get { yield } + end + end +end diff --git a/src/spectator/line_example_filter.cr b/src/spectator/line_example_filter.cr deleted file mode 100644 index 0c65fba..0000000 --- a/src/spectator/line_example_filter.cr +++ /dev/null @@ -1,15 +0,0 @@ -module Spectator - # Filter that matches examples on a given line. - class LineExampleFilter < ExampleFilter - # Creates the example filter. - def initialize(@line : Int32) - end - - # Checks whether the example satisfies the filter. - def includes?(example) : Bool - start_line = example.source.line - end_line = example.source.end_line - (start_line..end_line).covers?(@line) - end - end -end diff --git a/src/spectator/line_node_filter.cr b/src/spectator/line_node_filter.cr new file mode 100644 index 0000000..52213e4 --- /dev/null +++ b/src/spectator/line_node_filter.cr @@ -0,0 +1,19 @@ +require "./node_filter" + +module Spectator + # Filter that matches nodes on a given line. + class LineNodeFilter < NodeFilter + # Creates the node filter. + def initialize(@line : Int32) + end + + # Checks whether the node satisfies the filter. + def includes?(node) : Bool + return false unless location = node.location? + + start_line = location.line + end_line = location.end_line + (start_line..end_line).covers?(@line) + end + end +end diff --git a/src/spectator/source.cr b/src/spectator/location.cr similarity index 80% rename from src/spectator/source.cr rename to src/spectator/location.cr index 6afa3a7..688fdb6 100644 --- a/src/spectator/source.cr +++ b/src/spectator/location.cr @@ -1,6 +1,6 @@ module Spectator - # Define the file and line number something originated from. - struct Source + # Defines the file and line number a piece of code originated from. + struct Location # Absolute file path. getter file : String @@ -10,14 +10,19 @@ module Spectator # Ending line number in the file. getter end_line : Int32 - # Creates the source. + # Creates the location. def initialize(@file, @line, end_line = nil) # if an end line is not provided, # make the end line the same as the start line @end_line = end_line || @line end - # Parses a source from a string. + # Parses a location from a string. + # The *string* should be in the form: + # ```text + # FILE:LINE + # ``` + # This matches the output of the `#to_s` method. def self.parse(string) # Make sure to handle multiple colons. # If this ran on Windows, there's a possibility of a colon in the path. @@ -49,20 +54,13 @@ module Spectator end end - # String representation of the source. + # String representation of the location. # This is formatted as: # ```text # FILE:LINE # ``` def to_s(io) - io << path - io << ':' - io << line - end - - # Creates the JSON representation of the source. - def to_json(json : ::JSON::Builder) - json.string(to_s) + io << path << ':' << line end end end diff --git a/src/spectator/location_node_filter.cr b/src/spectator/location_node_filter.cr new file mode 100644 index 0000000..3774db2 --- /dev/null +++ b/src/spectator/location_node_filter.cr @@ -0,0 +1,17 @@ +require "./location" +require "./node_filter" + +module Spectator + # Filter that matches nodes in a given file and line. + class LocationNodeFilter < NodeFilter + # Creates the filter. + # The *location* indicates which file and line the node must contain. + def initialize(@location : Location) + end + + # Checks whether the node satisfies the filter. + def includes?(node) : Bool + @location === node.location? + end + end +end diff --git a/src/spectator/matchers/all_matcher.cr b/src/spectator/matchers/all_matcher.cr index d99ade4..404b66d 100644 --- a/src/spectator/matchers/all_matcher.cr +++ b/src/spectator/matchers/all_matcher.cr @@ -1,4 +1,4 @@ -require "../test_value" +require "../value" require "./failed_match_data" require "./matcher" require "./successful_match_data" @@ -21,12 +21,12 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T - found = test_values(actual).each do |element| + def match(actual : Expression(T)) : MatchData forall T + found = values(actual).each do |element| match_data = matcher.match(element) break match_data unless match_data.matched? end - found || SuccessfulMatchData.new(description) + found || SuccessfulMatchData.new(match_data_description(actual)) end # Negated matching for this matcher is not supported. @@ -39,18 +39,18 @@ module Spectator::Matchers # What if the collection is empty? # # RSpec doesn't support this syntax either. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T {% raise "The `expect { }.to_not all()` syntax is not supported (ambiguous)." %} end # Maps all values in the test collection to their own test values. # Each value is given their own label, # which is the original label with an index appended. - private def test_values(actual) + private def values(actual) label_prefix = actual.label actual.value.map_with_index do |value, index| label = "#{label_prefix}[#{index}]" - TestValue.new(value, label) + Value.new(value, label) end end end diff --git a/src/spectator/matchers/array_matcher.cr b/src/spectator/matchers/array_matcher.cr index 2f0d264..6adf442 100644 --- a/src/spectator/matchers/array_matcher.cr +++ b/src/spectator/matchers/array_matcher.cr @@ -11,7 +11,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(Array(ExpectedType))) + def initialize(@expected : Value(Array(ExpectedType))) end # Short text about the matcher's purpose. @@ -22,7 +22,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:to_a) @@ -32,10 +32,10 @@ module Spectator::Matchers if missing.empty? && extra.empty? # Contents are identical. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else # Content differs. - FailedMatchData.new(description, "#{actual.label} does not contain exactly #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} does not contain exactly #{expected.label}", expected: expected_elements.inspect, actual: actual_elements.inspect, missing: missing.empty? ? "None" : missing.inspect, @@ -46,7 +46,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:to_a) @@ -56,13 +56,13 @@ module Spectator::Matchers if missing.empty? && extra.empty? # Contents are identical. - FailedMatchData.new(description, "#{actual.label} contains exactly #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} contains exactly #{expected.label}", expected: "Not #{expected_elements.inspect}", actual: actual_elements.inspect ) else # Content differs. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end @@ -110,9 +110,9 @@ module Spectator::Matchers end end.reject do |(_, count)| count <= 0 - end.map do |(element, count)| + end.flat_map do |(element, count)| Array.new(count, element) - end.flatten + end end private def unexpected(value, label) diff --git a/src/spectator/matchers/attributes_matcher.cr b/src/spectator/matchers/attributes_matcher.cr index ec8826f..30ddb38 100644 --- a/src/spectator/matchers/attributes_matcher.cr +++ b/src/spectator/matchers/attributes_matcher.cr @@ -1,4 +1,4 @@ -require "../test_value" +require "../value" require "./failed_match_data" require "./matcher" require "./successful_match_data" @@ -14,7 +14,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -25,23 +25,23 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} does not have attributes #{expected.label}", values(snapshot).to_a) + FailedMatchData.new(match_data_description(actual), "#{actual.label} does not have attributes #{expected.label}", values(snapshot).to_a) end end # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) - FailedMatchData.new(description, "#{actual.label} has attributes #{expected.label}", negated_values(snapshot).to_a) + FailedMatchData.new(match_data_description(actual), "#{actual.label} has attributes #{expected.label}", negated_values(snapshot).to_a) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end diff --git a/src/spectator/matchers/case_matcher.cr b/src/spectator/matchers/case_matcher.cr index fcb5c23..00618be 100644 --- a/src/spectator/matchers/case_matcher.cr +++ b/src/spectator/matchers/case_matcher.cr @@ -12,13 +12,13 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T expected.value === actual.value end # Overload that takes a regex so that the operands are flipped. # This mimics RSpec's behavior. - private def match?(actual : TestExpression(Regex)) : Bool forall T + private def match?(actual : Expression(Regex)) : Bool forall T actual.value === expected.value end diff --git a/src/spectator/matchers/change_exact_matcher.cr b/src/spectator/matchers/change_exact_matcher.cr index 6aa562d..2f0880e 100644 --- a/src/spectator/matchers/change_exact_matcher.cr +++ b/src/spectator/matchers/change_exact_matcher.cr @@ -15,7 +15,7 @@ module Spectator::Matchers private getter expected_after # Creates a new change matcher. - def initialize(@expression : TestBlock(ExpressionType), @expected_before : FromType, @expected_after : ToType) + def initialize(@expression : Block(ExpressionType), @expected_before : FromType, @expected_after : ToType) end # Short text about the matcher's purpose. @@ -26,25 +26,25 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if expected_before == before if before == after - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label}", before: before.inspect, after: after.inspect ) elsif expected_after == after - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label} to #{expected_after.inspect}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label} to #{expected_after.inspect}", before: before.inspect, after: after.inspect, expected: expected_after.inspect ) end else - FailedMatchData.new(description, "#{expression.label} was not initially #{expected_before.inspect}", + FailedMatchData.new(match_data_description(actual), "#{expression.label} was not initially #{expected_before.inspect}", expected: expected_before.inspect, actual: before.inspect, ) @@ -53,19 +53,19 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if expected_before == before if expected_after == after - FailedMatchData.new(description, "#{actual.label} changed #{expression.label} from #{expected_before.inspect} to #{expected_after.inspect}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} changed #{expression.label} from #{expected_before.inspect} to #{expected_after.inspect}", before: before.inspect, after: after.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end else - FailedMatchData.new(description, "#{expression.label} was not initially #{expected_before.inspect}", + FailedMatchData.new(match_data_description(actual), "#{expression.label} was not initially #{expected_before.inspect}", expected: expected_before.inspect, actual: before.inspect, ) diff --git a/src/spectator/matchers/change_from_matcher.cr b/src/spectator/matchers/change_from_matcher.cr index c0e08c1..e684e00 100644 --- a/src/spectator/matchers/change_from_matcher.cr +++ b/src/spectator/matchers/change_from_matcher.cr @@ -13,7 +13,7 @@ module Spectator::Matchers private getter expected # Creates a new change matcher. - def initialize(@expression : TestBlock(ExpressionType), @expected : FromType) + def initialize(@expression : Block(ExpressionType), @expected : FromType) end # Short text about the matcher's purpose. @@ -24,37 +24,37 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if expected != before - FailedMatchData.new(description, "#{expression.label} was not initially #{expected}", + FailedMatchData.new(match_data_description(actual), "#{expression.label} was not initially #{expected}", expected: expected.inspect, actual: before.inspect, ) elsif before == after - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label} from #{expected}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label} from #{expected}", before: before.inspect, after: after.inspect, expected: "Not #{expected.inspect}" ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if expected != before - FailedMatchData.new(description, "#{expression.label} was not initially #{expected}", + FailedMatchData.new(match_data_description(actual), "#{expression.label} was not initially #{expected}", expected: expected.inspect, actual: before.inspect ) elsif before == after - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} changed #{expression.label} from #{expected}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} changed #{expression.label} from #{expected}", before: before.inspect, after: after.inspect, expected: expected.inspect diff --git a/src/spectator/matchers/change_matcher.cr b/src/spectator/matchers/change_matcher.cr index a60891b..3be8111 100644 --- a/src/spectator/matchers/change_matcher.cr +++ b/src/spectator/matchers/change_matcher.cr @@ -11,7 +11,7 @@ module Spectator::Matchers private getter expression # Creates a new change matcher. - def initialize(@expression : TestBlock(ExpressionType)) + def initialize(@expression : Block(ExpressionType)) end # Short text about the matcher's purpose. @@ -22,26 +22,26 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if before == after - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label}", before: before.inspect, after: after.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if before == after - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} changed #{expression.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} changed #{expression.label}", before: before.inspect, after: after.inspect ) diff --git a/src/spectator/matchers/change_relative_matcher.cr b/src/spectator/matchers/change_relative_matcher.cr index 4ac6e38..8511dd4 100644 --- a/src/spectator/matchers/change_relative_matcher.cr +++ b/src/spectator/matchers/change_relative_matcher.cr @@ -9,7 +9,7 @@ module Spectator::Matchers private getter expression # Creates a new change matcher. - def initialize(@expression : TestBlock(ExpressionType), @relativity : String, + def initialize(@expression : Block(ExpressionType), @relativity : String, &evaluator : ExpressionType, ExpressionType -> Bool) @evaluator = evaluator end @@ -22,17 +22,17 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if before == after - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label}", before: before.inspect, after: after.inspect ) elsif @evaluator.call(before, after) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label} #{@relativity}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label} #{@relativity}", before: before.inspect, after: after.inspect ) @@ -41,7 +41,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T {% raise "The `expect { }.to_not change { }.by_...()` syntax is not supported (ambiguous)." %} end diff --git a/src/spectator/matchers/change_to_matcher.cr b/src/spectator/matchers/change_to_matcher.cr index fb43204..57deff5 100644 --- a/src/spectator/matchers/change_to_matcher.cr +++ b/src/spectator/matchers/change_to_matcher.cr @@ -13,7 +13,7 @@ module Spectator::Matchers private getter expected # Creates a new change matcher. - def initialize(@expression : TestBlock(ExpressionType), @expected : ToType) + def initialize(@expression : Block(ExpressionType), @expected : ToType) end # Short text about the matcher's purpose. @@ -24,18 +24,18 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T before, after = change(actual) if before == after - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label}", before: before.inspect, after: after.inspect, expected: expected.inspect ) elsif expected == after - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} did not change #{expression.label} to #{expected}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not change #{expression.label} to #{expected}", before: before.inspect, after: after.inspect, expected: expected.inspect @@ -52,7 +52,7 @@ module Spectator::Matchers # but it is the expected value? # # RSpec doesn't support this syntax either. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T {% raise "The `expect { }.to_not change { }.to()` syntax is not supported (ambiguous)." %} end diff --git a/src/spectator/matchers/collection_matcher.cr b/src/spectator/matchers/collection_matcher.cr index 52b4378..277c5c3 100644 --- a/src/spectator/matchers/collection_matcher.cr +++ b/src/spectator/matchers/collection_matcher.cr @@ -1,4 +1,4 @@ -require "../test_value" +require "../value" require "./range_matcher" require "./value_matcher" @@ -13,7 +13,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T expected.value.includes?(actual.value) end @@ -55,8 +55,8 @@ module Spectator::Matchers lower = center - diff upper = center + diff range = Range.new(lower, upper) - test_value = TestValue.new(range, "#{center} ± #{expected.label}") - RangeMatcher.new(test_value) + value = Value.new(range, "#{center} ± #{expected.label}") + RangeMatcher.new(value) end end end diff --git a/src/spectator/matchers/compiled_type_matcher.cr b/src/spectator/matchers/compiled_type_matcher.cr new file mode 100644 index 0000000..2b60e3b --- /dev/null +++ b/src/spectator/matchers/compiled_type_matcher.cr @@ -0,0 +1,59 @@ +require "./matcher" + +module Spectator::Matchers + # Matcher that tests a value is of a specified type at compile time. + # The values are compared with the `typeof` method. + # This can be used to inspect the inferred type of methods and variables. + struct CompiledTypeMatcher(Expected) < StandardMatcher + # Short text about the matcher's purpose. + # This explains what condition satisfies the matcher. + # The description is used when the one-liner syntax is used. + def description : String + "compiles as #{Expected}" + end + + # Checks whether the matcher is satisifed with the expression given to it. + private def match?(actual : Expression(T)) : Bool forall T + Expected == typeof(actual.value) + end + + # Message displayed when the matcher isn't satisifed. + # + # This is only called when `#match?` returns false. + # + # The message should typically only contain the test expression labels. + # Actual values should be returned by `#values`. + private def failure_message(actual) : String + "#{actual.label} does not compile as #{Expected}" + end + + # Message displayed when the matcher isn't satisifed and is negated. + # This is essentially what would satisfy the matcher if it wasn't negated. + # + # This is only called when `#does_not_match?` returns false. + # + # The message should typically only contain the test expression labels. + # Actual values should be returned by `#values`. + private def failure_message_when_negated(actual) : String + "#{actual.label} compiles as #{Expected}" + end + + # Additional information about the match failure. + # The return value is a NamedTuple with Strings for each value. + private def values(actual) + { + expected: Expected.to_s, + actual: typeof(actual.value).inspect, + } + end + + # Additional information about the match failure when negated. + # The return value is a NamedTuple with Strings for each value. + private def negated_values(actual) + { + expected: "Not #{Expected}", + actual: typeof(actual.value).inspect, + } + end + end +end diff --git a/src/spectator/matchers/contain_matcher.cr b/src/spectator/matchers/contain_matcher.cr index afc73fd..7ebe221 100644 --- a/src/spectator/matchers/contain_matcher.cr +++ b/src/spectator/matchers/contain_matcher.cr @@ -8,7 +8,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -19,7 +19,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:includes?) @@ -29,10 +29,10 @@ module Spectator::Matchers if missing.empty? # Contents are present. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else # Content is missing. - FailedMatchData.new(description, "#{actual.label} does not contain #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} does not contain #{expected.label}", expected: expected.value.inspect, actual: actual_value.inspect, missing: missing.inspect, @@ -42,7 +42,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:includes?) @@ -52,13 +52,13 @@ module Spectator::Matchers if satisfied # Contents are present. - FailedMatchData.new(description, "#{actual.label} contains #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} contains #{expected.label}", expected: "Not #{expected.value.inspect}", actual: actual_value.inspect ) else # Content is missing. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end diff --git a/src/spectator/matchers/empty_matcher.cr b/src/spectator/matchers/empty_matcher.cr index e032794..bdfa80d 100644 --- a/src/spectator/matchers/empty_matcher.cr +++ b/src/spectator/matchers/empty_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:empty?) diff --git a/src/spectator/matchers/end_with_matcher.cr b/src/spectator/matchers/end_with_matcher.cr index 319c559..039d182 100644 --- a/src/spectator/matchers/end_with_matcher.cr +++ b/src/spectator/matchers/end_with_matcher.cr @@ -11,7 +11,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -22,7 +22,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T value = actual.value if value.is_a?(String) || value.responds_to?(:ends_with?) match_ends_with(value, actual.label) @@ -33,7 +33,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T value = actual.value if value.is_a?(String) || value.responds_to?(:ends_with?) negated_match_ends_with(value, actual.label) @@ -46,9 +46,9 @@ module Spectator::Matchers # This method expects (and uses) the `#ends_with?` method on the value. private def match_ends_with(actual_value, actual_label) if actual_value.ends_with?(expected.value) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not end with #{expected.label} (using #ends_with?)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not end with #{expected.label} (using #ends_with?)", expected: expected.value.inspect, actual: actual_value.inspect ) @@ -62,9 +62,9 @@ module Spectator::Matchers last = list.last if expected.value === last - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not end with #{expected.label} (using expected === last)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not end with #{expected.label} (using expected === last)", expected: expected.value.inspect, actual: last.inspect, list: list.inspect @@ -76,12 +76,12 @@ module Spectator::Matchers # This method expects (and uses) the `#ends_with?` method on the value. private def negated_match_ends_with(actual_value, actual_label) if actual_value.ends_with?(expected.value) - FailedMatchData.new(description, "#{actual_label} ends with #{expected.label} (using #ends_with?)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} ends with #{expected.label} (using #ends_with?)", expected: "Not #{expected.value.inspect}", actual: actual_value.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) end end @@ -92,13 +92,13 @@ module Spectator::Matchers last = list.last if expected.value === last - FailedMatchData.new(description, "#{actual_label} ends with #{expected.label} (using expected === last)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} ends with #{expected.label} (using expected === last)", expected: "Not #{expected.value.inspect}", actual: last.inspect, list: list.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) end end end diff --git a/src/spectator/matchers/equality_matcher.cr b/src/spectator/matchers/equality_matcher.cr index bcdcd44..62f9624 100644 --- a/src/spectator/matchers/equality_matcher.cr +++ b/src/spectator/matchers/equality_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T expected.value == actual.value end diff --git a/src/spectator/matchers/exception_matcher.cr b/src/spectator/matchers/exception_matcher.cr index 82d67ab..adec663 100644 --- a/src/spectator/matchers/exception_matcher.cr +++ b/src/spectator/matchers/exception_matcher.cr @@ -1,4 +1,4 @@ -require "../test_value" +require "../value" require "./failed_match_data" require "./matcher" require "./successful_match_data" @@ -11,11 +11,11 @@ module Spectator::Matchers # Creates the matcher with no expectation of the message. def initialize - @expected = TestValue.new(nil, ExceptionType.to_s) + @expected = Value.new(nil, ExceptionType.to_s) end # Creates the matcher with an expected message. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -30,19 +30,19 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T exception = capture_exception { actual.value } if exception.nil? - FailedMatchData.new(description, "#{actual.label} did not raise", expected: ExceptionType.inspect) + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not raise", expected: ExceptionType.inspect) else if exception.is_a?(ExceptionType) if (value = expected.value).nil? - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else if value === exception.message - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} raised #{exception.class}, but the message is not #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} raised #{exception.class}, but the message is not #{expected.label}", "expected type": ExceptionType.inspect, "actual type": exception.class.inspect, "expected message": value.inspect, @@ -51,7 +51,7 @@ module Spectator::Matchers end end else - FailedMatchData.new(description, "#{actual.label} did not raise #{ExceptionType}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} did not raise #{ExceptionType}", expected: ExceptionType.inspect, actual: exception.class.inspect ) @@ -61,37 +61,37 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T exception = capture_exception { actual.value } if exception.nil? - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else if exception.is_a?(ExceptionType) if (value = expected.value).nil? - FailedMatchData.new(description, "#{actual.label} raised #{exception.class}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} raised #{exception.class}", expected: "Not #{ExceptionType}", actual: exception.class.inspect ) else if value === exception.message - FailedMatchData.new(description, "#{actual.label} raised #{exception.class} with message matching #{expected.label}", + FailedMatchData.new(match_data_description(actual), "#{actual.label} raised #{exception.class} with message matching #{expected.label}", "expected type": ExceptionType.inspect, "actual type": exception.class.inspect, "expected message": value.inspect, "actual message": exception.message.to_s ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end end def with_message(message : T) forall T - value = TestValue.new(message) + value = Value.new(message) ExceptionMatcher(ExceptionType, T).new(value) end @@ -114,13 +114,13 @@ module Spectator::Matchers # Creates a new exception matcher with a message check. def self.create(value, label : String) - expected = TestValue.new(value, label) + expected = Value.new(value, label) ExceptionMatcher(Exception, typeof(value)).new(expected) end # Creates a new exception matcher with a type and message check. def self.create(exception_type : T.class, value, label : String) forall T - expected = TestValue.new(value, label) + expected = Value.new(value, label) ExceptionMatcher(T, typeof(value)).new(expected) end end diff --git a/src/spectator/matchers/greater_than_equal_matcher.cr b/src/spectator/matchers/greater_than_equal_matcher.cr index 08eb88c..cfaf4f1 100644 --- a/src/spectator/matchers/greater_than_equal_matcher.cr +++ b/src/spectator/matchers/greater_than_equal_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value >= expected.value end diff --git a/src/spectator/matchers/greater_than_matcher.cr b/src/spectator/matchers/greater_than_matcher.cr index 5dfc90c..4aaf6c1 100644 --- a/src/spectator/matchers/greater_than_matcher.cr +++ b/src/spectator/matchers/greater_than_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value > expected.value end diff --git a/src/spectator/matchers/have_key_matcher.cr b/src/spectator/matchers/have_key_matcher.cr index 5c07590..57224d3 100644 --- a/src/spectator/matchers/have_key_matcher.cr +++ b/src/spectator/matchers/have_key_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:has_key?) diff --git a/src/spectator/matchers/have_matcher.cr b/src/spectator/matchers/have_matcher.cr index 6e535ca..9c16c9f 100644 --- a/src/spectator/matchers/have_matcher.cr +++ b/src/spectator/matchers/have_matcher.cr @@ -9,7 +9,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -20,7 +20,7 @@ module Spectator::Matchers end # Entrypoint for the matcher, forwards to the correct method for string or enumerable. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T if (value = actual.value).is_a?(String) match_string(value, actual.label) else @@ -39,10 +39,10 @@ module Spectator::Matchers if missing.empty? # Contents are present. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else # Content is missing. - FailedMatchData.new(description, "#{actual_label} does not have #{expected.label}", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not have #{expected.label}", expected: expected.value.inspect, actual: actual_value.inspect, missing: missing.inspect, @@ -58,9 +58,9 @@ module Spectator::Matchers end if missing.empty? - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not have #{expected.label}", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not have #{expected.label}", expected: expected.value.inspect, actual: actual_value.inspect, missing: missing.inspect, @@ -70,7 +70,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T if (value = actual.value).is_a?(String) negated_match_string(value, actual.label) else @@ -89,13 +89,13 @@ module Spectator::Matchers if satisfied # Contents are present. - FailedMatchData.new(description, "#{actual_label} has #{expected.label}", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} has #{expected.label}", expected: "Not #{expected.value.inspect}", actual: actual_value.inspect ) else # Content is missing. - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) end end @@ -107,9 +107,9 @@ module Spectator::Matchers end if satisfied - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not have #{expected.label}", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not have #{expected.label}", expected: expected.value.inspect, actual: actual_value.inspect, missing: missing.inspect, diff --git a/src/spectator/matchers/have_predicate_matcher.cr b/src/spectator/matchers/have_predicate_matcher.cr index 5cb3f94..df4d90e 100644 --- a/src/spectator/matchers/have_predicate_matcher.cr +++ b/src/spectator/matchers/have_predicate_matcher.cr @@ -15,7 +15,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) SuccessfulMatchData.new(description) @@ -26,7 +26,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) FailedMatchData.new(description, "#{actual.label} has #{expected.label}", values(snapshot).to_a) diff --git a/src/spectator/matchers/have_value_matcher.cr b/src/spectator/matchers/have_value_matcher.cr index 54d5f40..aa059d8 100644 --- a/src/spectator/matchers/have_value_matcher.cr +++ b/src/spectator/matchers/have_value_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:has_value?) diff --git a/src/spectator/matchers/inequality_matcher.cr b/src/spectator/matchers/inequality_matcher.cr index f721ab4..145f5a1 100644 --- a/src/spectator/matchers/inequality_matcher.cr +++ b/src/spectator/matchers/inequality_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T expected.value != actual.value end diff --git a/src/spectator/matchers/instance_matcher.cr b/src/spectator/matchers/instance_matcher.cr index c77531e..bd8f51f 100644 --- a/src/spectator/matchers/instance_matcher.cr +++ b/src/spectator/matchers/instance_matcher.cr @@ -11,7 +11,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value.class == Expected end diff --git a/src/spectator/matchers/less_than_equal_matcher.cr b/src/spectator/matchers/less_than_equal_matcher.cr index bc56dab..620afb3 100644 --- a/src/spectator/matchers/less_than_equal_matcher.cr +++ b/src/spectator/matchers/less_than_equal_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value <= expected.value end diff --git a/src/spectator/matchers/less_than_matcher.cr b/src/spectator/matchers/less_than_matcher.cr index 4e14cd4..05256b1 100644 --- a/src/spectator/matchers/less_than_matcher.cr +++ b/src/spectator/matchers/less_than_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value < expected.value end diff --git a/src/spectator/matchers/matcher.cr b/src/spectator/matchers/matcher.cr index f5d0d4c..e54e55e 100644 --- a/src/spectator/matchers/matcher.cr +++ b/src/spectator/matchers/matcher.cr @@ -16,10 +16,22 @@ module Spectator::Matchers abstract def description : String # Actually performs the test against the expression (value or block). - abstract def match(actual : TestExpression(T)) : MatchData forall T + abstract def match(actual : Expression(T)) : MatchData forall T # Performs the test against the expression (value or block), but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - abstract def negated_match(actual : TestExpression(T)) : MatchData forall T + abstract def negated_match(actual : Expression(T)) : MatchData forall T + + private def match_data_description(actual : Expression(T)) : String forall T + match_data_description(actual.label) + end + + private def match_data_description(actual_label : String | Symbol) : String + "#{actual_label} #{description}" + end + + private def match_data_description(actual_label : Nil) : String + description + end end end diff --git a/src/spectator/matchers/nil_matcher.cr b/src/spectator/matchers/nil_matcher.cr index 5334037..187f809 100644 --- a/src/spectator/matchers/nil_matcher.cr +++ b/src/spectator/matchers/nil_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value.nil? end diff --git a/src/spectator/matchers/predicate_matcher.cr b/src/spectator/matchers/predicate_matcher.cr index bda4f9b..ebf1d87 100644 --- a/src/spectator/matchers/predicate_matcher.cr +++ b/src/spectator/matchers/predicate_matcher.cr @@ -10,7 +10,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with a expected values. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -21,23 +21,23 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} is not #{expected.label}", values(snapshot).to_a) + FailedMatchData.new(match_data_description(actual), "#{actual.label} is not #{expected.label}", values(snapshot).to_a) end end # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if match?(snapshot) - FailedMatchData.new(description, "#{actual.label} is #{expected.label}", values(snapshot).to_a) + FailedMatchData.new(match_data_description(actual), "#{actual.label} is #{expected.label}", values(snapshot).to_a) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end diff --git a/src/spectator/matchers/range_matcher.cr b/src/spectator/matchers/range_matcher.cr index fe54b7e..b4850f5 100644 --- a/src/spectator/matchers/range_matcher.cr +++ b/src/spectator/matchers/range_matcher.cr @@ -15,7 +15,7 @@ module Spectator::Matchers def inclusive label = expected.label new_range = Range.new(range.begin, range.end, exclusive: false) - expected = TestValue.new(new_range, label) + expected = Value.new(new_range, label) RangeMatcher.new(expected) end @@ -23,12 +23,12 @@ module Spectator::Matchers def exclusive label = expected.label new_range = Range.new(range.begin, range.end, exclusive: true) - expected = TestValue.new(new_range, label) + expected = Value.new(new_range, label) RangeMatcher.new(expected) end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T expected.value.includes?(actual.value) end diff --git a/src/spectator/matchers/receive_matcher.cr b/src/spectator/matchers/receive_matcher.cr index 9a96f82..7f3d233 100644 --- a/src/spectator/matchers/receive_matcher.cr +++ b/src/spectator/matchers/receive_matcher.cr @@ -5,7 +5,7 @@ module Spectator::Matchers struct ReceiveMatcher < StandardMatcher alias Range = ::Range(Int32, Int32) | ::Range(Nil, Int32) | ::Range(Int32, Nil) - def initialize(@expected : TestExpression(Symbol), @args : Mocks::Arguments? = nil, @range : Range? = nil) + def initialize(@expected : Expression(Symbol), @args : Mocks::Arguments? = nil, @range : Range? = nil) end def description : String @@ -13,7 +13,7 @@ module Spectator::Matchers "received message #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "At least once"} with #{@args || "any arguments"}" end - def match?(actual : TestExpression(T)) : Bool forall T + def match?(actual : Expression(T)) : Bool forall T calls = Harness.current.mocks.calls_for(actual.value, @expected.value) calls.select! { |call| @args === call.args } if @args if (range = @range) @@ -23,7 +23,7 @@ module Spectator::Matchers end end - def failure_message(actual : TestExpression(T)) : String forall T + def failure_message(actual : Expression(T)) : String forall T range = @range "#{actual.label} did not receive #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "at least once"} with #{@args || "any arguments"}" end @@ -33,7 +33,7 @@ module Spectator::Matchers "#{actual.label} received #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "at least once"} with #{@args || "any arguments"}" end - def values(actual : TestExpression(T)) forall T + def values(actual : Expression(T)) forall T calls = Harness.current.mocks.calls_for(actual.value, @expected.value) calls.select! { |call| @args === call.args } if @args range = @range @@ -43,7 +43,7 @@ module Spectator::Matchers } end - def negated_values(actual : TestExpression(T)) forall T + def negated_values(actual : Expression(T)) forall T calls = Harness.current.mocks.calls_for(actual.value, @expected.value) calls.select! { |call| @args === call.args } if @args range = @range @@ -115,7 +115,7 @@ module Spectator::Matchers end private struct Count - def initialize(@expected : TestExpression(Symbol), @args : Mocks::Arguments?, @range : Range) + def initialize(@expected : Expression(Symbol), @args : Mocks::Arguments?, @range : Range) end def times diff --git a/src/spectator/matchers/receive_type_matcher.cr b/src/spectator/matchers/receive_type_matcher.cr index c716f05..362707d 100644 --- a/src/spectator/matchers/receive_type_matcher.cr +++ b/src/spectator/matchers/receive_type_matcher.cr @@ -5,7 +5,7 @@ module Spectator::Matchers struct ReceiveTypeMatcher < StandardMatcher alias Range = ::Range(Int32, Int32) | ::Range(Nil, Int32) | ::Range(Int32, Nil) - def initialize(@expected : TestExpression(Symbol), @args : Mocks::Arguments? = nil, @range : Range? = nil) + def initialize(@expected : Expression(Symbol), @args : Mocks::Arguments? = nil, @range : Range? = nil) end def description : String @@ -13,7 +13,7 @@ module Spectator::Matchers "received message #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "At least once"} with #{@args || "any arguments"}" end - def match?(actual : TestExpression(T)) : Bool forall T + def match?(actual : Expression(T)) : Bool forall T calls = Harness.current.mocks.calls_for_type(actual.value, @expected.value) calls.select! { |call| @args === call.args } if @args if (range = @range) @@ -23,17 +23,17 @@ module Spectator::Matchers end end - def failure_message(actual : TestExpression(T)) : String forall T + def failure_message(actual : Expression(T)) : String forall T range = @range "#{actual.label} did not receive #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "at least once"} with #{@args || "any arguments"}" end - def failure_message_when_negated(actual : TestExpression(T)) : String forall T + def failure_message_when_negated(actual : Expression(T)) : String forall T range = @range "#{actual.label} received #{@expected.label} #{range ? "#{humanize_range(range)} time(s)" : "at least once"} with #{@args || "any arguments"}" end - def values(actual : TestExpression(T)) forall T + def values(actual : Expression(T)) forall T calls = Harness.current.mocks.calls_for_type(T, @expected.value) calls.select! { |call| @args === call.args } if @args range = @range @@ -43,7 +43,7 @@ module Spectator::Matchers } end - def negated_values(actual : TestExpression(T)) forall T + def negated_values(actual : Expression(T)) forall T calls = Harness.current.mocks.calls_for_type(T, @expected.value) calls.select! { |call| @args === call.args } if @args range = @range @@ -115,7 +115,7 @@ module Spectator::Matchers end private struct Count - def initialize(@expected : TestExpression(Symbol), @args : Mocks::Arguments?, @range : Range) + def initialize(@expected : Expression(Symbol), @args : Mocks::Arguments?, @range : Range) end def times diff --git a/src/spectator/matchers/reference_matcher.cr b/src/spectator/matchers/reference_matcher.cr index 74ce6e7..8385eb6 100644 --- a/src/spectator/matchers/reference_matcher.cr +++ b/src/spectator/matchers/reference_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T value = expected.value if value && value.responds_to?(:same?) value.same?(actual.value) diff --git a/src/spectator/matchers/respond_matcher.cr b/src/spectator/matchers/respond_matcher.cr index 87f95a9..9712da5 100644 --- a/src/spectator/matchers/respond_matcher.cr +++ b/src/spectator/matchers/respond_matcher.cr @@ -14,23 +14,24 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) if snapshot.values.all? - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} does not respond to #{label}", values(snapshot).to_a) + FailedMatchData.new(match_data_description(actual), "#{actual.label} does not respond to #{label}", values(snapshot).to_a) end end # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T snapshot = snapshot_values(actual.value) - if snapshot.values.any? - FailedMatchData.new(description, "#{actual.label} responds to #{label}", values(snapshot).to_a) + # Intentionally check truthiness of each value. + if snapshot.values.any? # ameba:disable Performance/AnyInsteadOfEmpty + FailedMatchData.new(match_data_description(actual), "#{actual.label} responds to #{label}", values(snapshot).to_a) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end diff --git a/src/spectator/matchers/size_matcher.cr b/src/spectator/matchers/size_matcher.cr index f707159..ea51c89 100644 --- a/src/spectator/matchers/size_matcher.cr +++ b/src/spectator/matchers/size_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:size?) diff --git a/src/spectator/matchers/size_of_matcher.cr b/src/spectator/matchers/size_of_matcher.cr index 8784cea..ad06f99 100644 --- a/src/spectator/matchers/size_of_matcher.cr +++ b/src/spectator/matchers/size_of_matcher.cr @@ -12,7 +12,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:size?) diff --git a/src/spectator/matchers/standard_matcher.cr b/src/spectator/matchers/standard_matcher.cr index 52cdc67..43992ec 100644 --- a/src/spectator/matchers/standard_matcher.cr +++ b/src/spectator/matchers/standard_matcher.cr @@ -1,4 +1,4 @@ -require "../test_value" +require "../expression" require "./failed_match_data" require "./matcher" require "./successful_match_data" @@ -23,11 +23,11 @@ module Spectator::Matchers # If it returns true, then a `SuccessfulMatchData` instance is returned. # Otherwise, a `FailedMatchData` instance is returned. # Additionally, `#failure_message` and `#values` are called for a failed match. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T if match?(actual) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, failure_message(actual), values(actual).to_a) + FailedMatchData.new(match_data_description(actual), failure_message(actual), values(actual).to_a) end end @@ -38,12 +38,12 @@ module Spectator::Matchers # If it returns true, then a `SuccessfulMatchData` instance is returned. # Otherwise, a `FailedMatchData` instance is returned. # Additionally, `#failure_message_when_negated` and `#negated_values` are called for a failed match. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T # TODO: Invert description. if does_not_match?(actual) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, failure_message_when_negated(actual), negated_values(actual).to_a) + FailedMatchData.new(match_data_description(actual), failure_message_when_negated(actual), negated_values(actual).to_a) end end @@ -53,7 +53,7 @@ module Spectator::Matchers # # The message should typically only contain the test expression labels. # Actual values should be returned by `#values`. - private abstract def failure_message(actual : TestExpression(T)) : String forall T + private abstract def failure_message(actual : Expression(T)) : String forall T # Message displayed when the matcher isn't satisifed and is negated. # This is essentially what would satisfy the matcher if it wasn't negated. @@ -66,12 +66,12 @@ module Spectator::Matchers # # The message should typically only contain the test expression labels. # Actual values should be returned by `#values`. - private def failure_message_when_negated(actual : TestExpression(T)) : String forall T + private def failure_message_when_negated(actual : Expression(T)) : String forall T raise "Negation with #{self.class} is not supported." end # Checks whether the matcher is satisifed with the expression given to it. - private abstract def match?(actual : TestExpression(T)) : Bool forall T + private abstract def match?(actual : Expression(T)) : Bool forall T # If the expectation is negated, then this method is called instead of `#match?`. # @@ -79,7 +79,7 @@ module Spectator::Matchers # If the matcher requires custom handling of negated matches, # then this method should be overriden. # Remember to override `#failure_message_when_negated` as well. - private def does_not_match?(actual : TestExpression(T)) : Bool forall T + private def does_not_match?(actual : Expression(T)) : Bool forall T !match?(actual) end @@ -101,7 +101,7 @@ module Spectator::Matchers # # The values should typically only contain the test expression values, not the labels. # Labeled should be returned by `#failure_message`. - private def values(actual : TestExpression(T)) forall T + private def values(actual : Expression(T)) forall T {actual: actual.value.inspect} end @@ -123,7 +123,7 @@ module Spectator::Matchers # # The values should typically only contain the test expression values, not the labels. # Labeled should be returned by `#failure_message_when_negated`. - private def negated_values(actual : TestExpression(T)) forall T + private def negated_values(actual : Expression(T)) forall T values(actual) end end diff --git a/src/spectator/matchers/start_with_matcher.cr b/src/spectator/matchers/start_with_matcher.cr index b459bb4..4bfc6c8 100644 --- a/src/spectator/matchers/start_with_matcher.cr +++ b/src/spectator/matchers/start_with_matcher.cr @@ -10,7 +10,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Short text about the matcher's purpose. @@ -21,7 +21,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T value = actual.value if value.is_a?(String) || value.responds_to?(:starts_with?) match_starts_with(value, actual.label) @@ -32,7 +32,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T value = actual.value if value.is_a?(String) || value.responds_to?(:starts_with?) negated_match_starts_with(value, actual.label) @@ -45,9 +45,9 @@ module Spectator::Matchers # This method expects (and uses) the `#starts_with?` method on the value. private def match_starts_with(actual_value, actual_label) if actual_value.starts_with?(expected.value) - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not start with #{expected.label} (using #starts_with?)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not start with #{expected.label} (using #starts_with?)", expected: expected.value.inspect, actual: actual_value.inspect ) @@ -61,9 +61,9 @@ module Spectator::Matchers first = list.first if expected.value === first - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) else - FailedMatchData.new(description, "#{actual_label} does not start with #{expected.label} (using expected === first)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} does not start with #{expected.label} (using expected === first)", expected: expected.value.inspect, actual: first.inspect, list: list.inspect @@ -75,12 +75,12 @@ module Spectator::Matchers # This method expects (and uses) the `#starts_with?` method on the value. private def negated_match_starts_with(actual_value, actual_label) if actual_value.starts_with?(expected.value) - FailedMatchData.new(description, "#{actual_label} starts with #{expected.label} (using #starts_with?)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} starts with #{expected.label} (using #starts_with?)", expected: "Not #{expected.value.inspect}", actual: actual_value.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) end end @@ -91,13 +91,13 @@ module Spectator::Matchers first = list.first if expected.value === first - FailedMatchData.new(description, "#{actual_label} starts with #{expected.label} (using expected === first)", + FailedMatchData.new(match_data_description(actual_label), "#{actual_label} starts with #{expected.label} (using expected === first)", expected: "Not #{expected.value.inspect}", actual: first.inspect, list: list.inspect ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual_label)) end end end diff --git a/src/spectator/matchers/truthy_matcher.cr b/src/spectator/matchers/truthy_matcher.cr index eeecfb9..0e8d424 100644 --- a/src/spectator/matchers/truthy_matcher.cr +++ b/src/spectator/matchers/truthy_matcher.cr @@ -28,7 +28,7 @@ module Spectator::Matchers # expect(0).to be < 1 # ``` def <(value) - expected = TestValue.new(value) + expected = Value.new(value) LessThanMatcher.new(expected) end @@ -38,7 +38,7 @@ module Spectator::Matchers # expect(0).to be <= 1 # ``` def <=(value) - expected = TestValue.new(value) + expected = Value.new(value) LessThanEqualMatcher.new(expected) end @@ -48,7 +48,7 @@ module Spectator::Matchers # expect(2).to be > 1 # ``` def >(value) - expected = TestValue.new(value) + expected = Value.new(value) GreaterThanMatcher.new(expected) end @@ -58,7 +58,7 @@ module Spectator::Matchers # expect(2).to be >= 1 # ``` def >=(value) - expected = TestValue.new(value) + expected = Value.new(value) GreaterThanEqualMatcher.new(expected) end @@ -68,7 +68,7 @@ module Spectator::Matchers # expect(0).to be == 0 # ``` def ==(value) - expected = TestValue.new(value) + expected = Value.new(value) EqualityMatcher.new(expected) end @@ -78,7 +78,7 @@ module Spectator::Matchers # expect(0).to be != 1 # ``` def !=(value) - expected = TestValue.new(value) + expected = Value.new(value) InequalityMatcher.new(expected) end @@ -103,7 +103,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T @truthy == !!actual.value end diff --git a/src/spectator/matchers/type_matcher.cr b/src/spectator/matchers/type_matcher.cr index 325265a..10d11f4 100644 --- a/src/spectator/matchers/type_matcher.cr +++ b/src/spectator/matchers/type_matcher.cr @@ -17,7 +17,7 @@ module Spectator::Matchers end # Checks whether the matcher is satisifed with the expression given to it. - private def match?(actual : TestExpression(T)) : Bool forall T + private def match?(actual : Expression(T)) : Bool forall T actual.value.is_a?(Expected) end diff --git a/src/spectator/matchers/unordered_array_matcher.cr b/src/spectator/matchers/unordered_array_matcher.cr index e408809..510465f 100644 --- a/src/spectator/matchers/unordered_array_matcher.cr +++ b/src/spectator/matchers/unordered_array_matcher.cr @@ -8,7 +8,7 @@ module Spectator::Matchers private getter expected # Creates the matcher with an expected value. - def initialize(@expected : TestValue(Array(ExpectedType))) + def initialize(@expected : Value(Array(ExpectedType))) end # Short text about the matcher's purpose. @@ -19,7 +19,7 @@ module Spectator::Matchers end # Actually performs the test against the expression. - def match(actual : TestExpression(T)) : MatchData forall T + def match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:to_a) @@ -28,9 +28,9 @@ module Spectator::Matchers missing, extra = array_diff(expected_elements, actual_elements) if missing.empty? && extra.empty? - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) else - FailedMatchData.new(description, "#{actual.label} does not contain #{expected.label} (unordered)", + FailedMatchData.new(match_data_description(actual), "#{actual.label} does not contain #{expected.label} (unordered)", expected: expected_elements.inspect, actual: actual_elements.inspect, missing: missing.inspect, @@ -41,7 +41,7 @@ module Spectator::Matchers # Performs the test against the expression, but inverted. # A successful match with `#match` should normally fail for this method, and vice-versa. - def negated_match(actual : TestExpression(T)) : MatchData forall T + def negated_match(actual : Expression(T)) : MatchData forall T actual_value = actual.value return unexpected(actual_value, actual.label) unless actual_value.responds_to?(:to_a) @@ -50,12 +50,12 @@ module Spectator::Matchers missing, extra = array_diff(expected_elements, actual_elements) if missing.empty? && extra.empty? - FailedMatchData.new(description, "#{actual.label} contains #{expected.label} (unordered)", + FailedMatchData.new(match_data_description(actual), "#{actual.label} contains #{expected.label} (unordered)", expected: "Not #{expected_elements.inspect}", actual: actual_elements.inspect, ) else - SuccessfulMatchData.new(description) + SuccessfulMatchData.new(match_data_description(actual)) end end diff --git a/src/spectator/matchers/value_matcher.cr b/src/spectator/matchers/value_matcher.cr index 5049320..c7dffe9 100644 --- a/src/spectator/matchers/value_matcher.cr +++ b/src/spectator/matchers/value_matcher.cr @@ -22,7 +22,7 @@ module Spectator::Matchers # Creates the value matcher. # The expected value is stored for later use. - def initialize(@expected : TestValue(ExpectedType)) + def initialize(@expected : Value(ExpectedType)) end # Additional information about the match failure. @@ -40,7 +40,7 @@ module Spectator::Matchers # actual: "bar", # } # ``` - private def values(actual : TestExpression(T)) forall T + private def values(actual : Expression(T)) forall T super.merge(expected: expected.value.inspect) end @@ -60,7 +60,7 @@ module Spectator::Matchers # actual: "bar", # } # ``` - private def negated_values(actual : TestExpression(T)) forall T + private def negated_values(actual : Expression(T)) forall T super.merge(expected: "Not #{expected.value.inspect}") end end diff --git a/src/spectator/metadata.cr b/src/spectator/metadata.cr new file mode 100644 index 0000000..9baaab6 --- /dev/null +++ b/src/spectator/metadata.cr @@ -0,0 +1,11 @@ +module Spectator + # User-defined keywords used for filtering and behavior modification. + alias Tags = Set(Symbol) + + # User-defined keywords used for filtering and behavior modification. + # The value of a tag is optional, but may contain useful information. + # If the value is nil, the tag exists, but has no data. + # However, when tags are given on examples and example groups, + # if the value is falsey (false or nil), then the tag should be removed from the overall collection. + alias Metadata = Hash(Symbol, String?) +end diff --git a/src/spectator/mocks/double.cr b/src/spectator/mocks/double.cr index 9684aed..6768c49 100644 --- a/src/spectator/mocks/double.cr +++ b/src/spectator/mocks/double.cr @@ -124,9 +124,7 @@ module Spectator::Mocks end def to_s(io) - io << "Double(" - io << @spectator_double_name - io << ')' + io << "Double(" << @spectator_double_name << ')' end end end diff --git a/src/spectator/mocks/exception_method_stub.cr b/src/spectator/mocks/exception_method_stub.cr index 96342bc..2f223fd 100644 --- a/src/spectator/mocks/exception_method_stub.cr +++ b/src/spectator/mocks/exception_method_stub.cr @@ -3,8 +3,8 @@ require "./generic_method_stub" module Spectator::Mocks class ExceptionMethodStub(ExceptionType) < GenericMethodStub(Nil) - def initialize(name, source, @exception : ExceptionType, args = nil) - super(name, source, args) + def initialize(name, location, @exception : ExceptionType, args = nil) + super(name, location, args) end def call(_args : GenericArguments(T, NT), &_original : -> RT) forall T, NT, RT diff --git a/src/spectator/mocks/expect_any_instance.cr b/src/spectator/mocks/expect_any_instance.cr index 9d9769e..607c425 100644 --- a/src/spectator/mocks/expect_any_instance.cr +++ b/src/spectator/mocks/expect_any_instance.cr @@ -2,15 +2,15 @@ require "./registry" module Spectator::Mocks struct ExpectAnyInstance(T) - def initialize(@source : Source) + def initialize(@location : Location) end def to(stub : MethodStub) : Nil - actual = TestValue.new(T) + actual = Value.new(T) Harness.current.mocks.expect(T, stub) - value = TestValue.new(stub.name, stub.to_s) + value = Value.new(stub.name, stub.to_s) matcher = Matchers::ReceiveTypeMatcher.new(value, stub.arguments?) - partial = Expectations::ExpectationPartial.new(actual, @source) + partial = Expectations::ExpectationPartial.new(actual, @location) partial.to_eventually(matcher) end diff --git a/src/spectator/mocks/generic_arguments.cr b/src/spectator/mocks/generic_arguments.cr index 18d4193..156ef27 100644 --- a/src/spectator/mocks/generic_arguments.cr +++ b/src/spectator/mocks/generic_arguments.cr @@ -32,8 +32,7 @@ module Spectator::Mocks end io << ", " unless @args.empty? || @opts.empty? @opts.each_with_index do |key, value, i| - io << key - io << ": " + io << key << ": " value.inspect(io) io << ", " if i < @opts.size - 1 end diff --git a/src/spectator/mocks/generic_method_stub.cr b/src/spectator/mocks/generic_method_stub.cr index dc8eba6..15ef9b4 100644 --- a/src/spectator/mocks/generic_method_stub.cr +++ b/src/spectator/mocks/generic_method_stub.cr @@ -7,8 +7,8 @@ module Spectator::Mocks abstract class GenericMethodStub(ReturnType) < MethodStub getter! arguments : Arguments - def initialize(name, source, @args : Arguments? = nil) - super(name, source) + def initialize(name, location, @args : Arguments? = nil) + super(name, location) end def callable?(call : MethodCall) : Bool @@ -18,14 +18,9 @@ module Spectator::Mocks def to_s(io) super(io) if @args - io << '(' - io << @args - io << ')' + io << '(' << @args << ')' end - io << " : " - io << ReturnType - io << " at " - io << @source + io << " : " << ReturnType << " at " << @location end end end diff --git a/src/spectator/mocks/method_call.cr b/src/spectator/mocks/method_call.cr index f89138f..79b3a8b 100644 --- a/src/spectator/mocks/method_call.cr +++ b/src/spectator/mocks/method_call.cr @@ -7,8 +7,7 @@ module Spectator::Mocks end def to_s(io) - io << '#' - io << @name + io << '#' << @name end end end diff --git a/src/spectator/mocks/method_stub.cr b/src/spectator/mocks/method_stub.cr index 950fad0..9ad78a3 100644 --- a/src/spectator/mocks/method_stub.cr +++ b/src/spectator/mocks/method_stub.cr @@ -1,13 +1,13 @@ -require "../source" +require "../location" require "./method_call" module Spectator::Mocks abstract class MethodStub getter name : Symbol - getter source : Source + getter location : Location - def initialize(@name, @source) + def initialize(@name, @location) end def callable?(call : MethodCall) : Bool @@ -28,8 +28,7 @@ module Spectator::Mocks end def to_s(io) - io << '#' - io << @name + io << '#' << @name end end end diff --git a/src/spectator/mocks/multi_value_method_stub.cr b/src/spectator/mocks/multi_value_method_stub.cr index c623a4c..f025271 100644 --- a/src/spectator/mocks/multi_value_method_stub.cr +++ b/src/spectator/mocks/multi_value_method_stub.cr @@ -5,8 +5,8 @@ module Spectator::Mocks class MultiValueMethodStub(ReturnType) < GenericMethodStub(ReturnType) @index = 0 - def initialize(name, source, @values : ReturnType, args = nil) - super(name, source, args) + def initialize(name, location, @values : ReturnType, args = nil) + super(name, location, args) raise ArgumentError.new("Values must have at least one item") if @values.size < 1 end diff --git a/src/spectator/mocks/nil_method_stub.cr b/src/spectator/mocks/nil_method_stub.cr index d33e5be..1e87db6 100644 --- a/src/spectator/mocks/nil_method_stub.cr +++ b/src/spectator/mocks/nil_method_stub.cr @@ -13,40 +13,40 @@ module Spectator::Mocks end def and_return(value) - ValueMethodStub.new(@name, @source, value, @args) + ValueMethodStub.new(@name, @location, value, @args) end def and_return(*values) - MultiValueMethodStub.new(@name, @source, values.to_a, @args) + MultiValueMethodStub.new(@name, @location, values.to_a, @args) end def and_raise(exception_type : Exception.class) - ExceptionMethodStub.new(@name, @source, exception_type.new, @args) + ExceptionMethodStub.new(@name, @location, exception_type.new, @args) end def and_raise(exception : Exception) - ExceptionMethodStub.new(@name, @source, exception, @args) + ExceptionMethodStub.new(@name, @location, exception, @args) end def and_raise(message : String) - ExceptionMethodStub.new(@name, @source, Exception.new(message), @args) + ExceptionMethodStub.new(@name, @location, Exception.new(message), @args) end def and_raise(exception_type : Exception.class, *args) forall T - ExceptionMethodStub.new(@name, @source, exception_type.new(*args), @args) + ExceptionMethodStub.new(@name, @location, exception_type.new(*args), @args) end def with(*args : *T, **opts : **NT) forall T, NT args = GenericArguments.new(args, opts) - NilMethodStub.new(@name, @source, args) + NilMethodStub.new(@name, @location, args) end def with(args : Arguments) - NilMethodStub.new(@name, @source, @args) + NilMethodStub.new(@name, @location, @args) end def and_call_original - OriginalMethodStub.new(@name, @source, @args) + OriginalMethodStub.new(@name, @location, @args) end end end diff --git a/src/spectator/mocks/proc_method_stub.cr b/src/spectator/mocks/proc_method_stub.cr index 1d55db3..de3a68d 100644 --- a/src/spectator/mocks/proc_method_stub.cr +++ b/src/spectator/mocks/proc_method_stub.cr @@ -3,12 +3,12 @@ require "./generic_method_stub" module Spectator::Mocks class ProcMethodStub(ReturnType) < GenericMethodStub(ReturnType) - def initialize(name, source, @proc : -> ReturnType, args = nil) - super(name, source, args) + def initialize(name, location, @proc : -> ReturnType, args = nil) + super(name, location, args) end - def self.create(name, source, args = nil, &block : -> T) forall T - ProcMethodStub.new(name, source, block, args) + def self.create(name, location, args = nil, &block : -> T) forall T + ProcMethodStub.new(name, location, block, args) end def call(_args : GenericArguments(T, NT), &_original : -> RT) forall T, NT, RT diff --git a/src/spectator/mocks/reflection.cr b/src/spectator/mocks/reflection.cr index cf014e1..9a03fd9 100644 --- a/src/spectator/mocks/reflection.cr +++ b/src/spectator/mocks/reflection.cr @@ -4,7 +4,7 @@ module Spectator::Mocks module Reflection private macro _spectator_reflect {% for meth in @type.methods %} - %source = ::Spectator::Source.new({{meth.filename}}, {{meth.line_number}}) + %location = ::Spectator::Location.new({{meth.filename}}, {{meth.line_number}}) %args = ::Spectator::Mocks::GenericArguments.create( {% for arg, i in meth.args %} {% matcher = if arg.restriction @@ -19,7 +19,7 @@ module Spectator::Mocks {{matcher}}{% if i < meth.args.size %},{% end %} {% end %} ) - ::Spectator::Mocks::TypeRegistry.add({{@type.id.stringify}}, {{meth.name.symbolize}}, %source, %args) + ::Spectator::Mocks::TypeRegistry.add({{@type.id.stringify}}, {{meth.name.symbolize}}, %location, %args) {% end %} end end diff --git a/src/spectator/mocks/registry.cr b/src/spectator/mocks/registry.cr index 0c334b9..c7b9a81 100644 --- a/src/spectator/mocks/registry.cr +++ b/src/spectator/mocks/registry.cr @@ -11,23 +11,6 @@ module Spectator::Mocks @all_instances = {} of String => Entry @entries = {} of Key => Entry - def initialize(context : TestContext) - current_context = context - while current_context - current_context.stubs.each do |k, v| - stubs = if @all_instances.has_key?(k) - @all_instances[k].stubs - else - entry = Entry.new - @all_instances[k] = entry - entry.stubs - end - stubs.concat(v) - end - current_context = current_context.parent? - end - end - def reset : Nil @entries.clear end @@ -112,11 +95,11 @@ module Spectator::Mocks end end - private def unique_key(reference : Reference) + private def unique_key(reference : ::Reference) {reference.class.name, reference.object_id} end - private def unique_key(value : Value) + private def unique_key(value : ::Value) {value.class.name, value.hash} end end diff --git a/src/spectator/mocks/type_registry.cr b/src/spectator/mocks/type_registry.cr index 4f960f3..631fec9 100644 --- a/src/spectator/mocks/type_registry.cr +++ b/src/spectator/mocks/type_registry.cr @@ -6,14 +6,14 @@ module Spectator::Mocks @@entries = {} of Key => Deque(MethodStub) - def add(type_name : String, method_name : Symbol, source : Source, args : Arguments) : Nil + def add(type_name : String, method_name : Symbol, location : Location, args : Arguments) : Nil key = {type_name, method_name} list = if @@entries.has_key?(key) @@entries[key] else @@entries[key] = Deque(MethodStub).new end - list << NilMethodStub.new(method_name, source, args) + list << NilMethodStub.new(method_name, location, args) end def exists?(type_name : String, call : MethodCall) : Bool diff --git a/src/spectator/mocks/value_method_stub.cr b/src/spectator/mocks/value_method_stub.cr index 43bfcaa..d3e7f5e 100644 --- a/src/spectator/mocks/value_method_stub.cr +++ b/src/spectator/mocks/value_method_stub.cr @@ -3,8 +3,8 @@ require "./generic_method_stub" module Spectator::Mocks class ValueMethodStub(ReturnType) < GenericMethodStub(ReturnType) - def initialize(name, source, @value : ReturnType, args = nil) - super(name, source, args) + def initialize(name, location, @value : ReturnType, args = nil) + super(name, location, args) end def call(_args : GenericArguments(T, NT), &_original : -> RT) forall T, NT, RT diff --git a/src/spectator/mocks/verifying_double.cr b/src/spectator/mocks/verifying_double.cr index 6c8e157..f7be6e3 100644 --- a/src/spectator/mocks/verifying_double.cr +++ b/src/spectator/mocks/verifying_double.cr @@ -100,9 +100,7 @@ module Spectator::Mocks end def to_s(io) - io << "Double(" - io << T - io << ')' + io << "Double(" << T << ')' end end end diff --git a/src/spectator/multiple_expectations_failed.cr b/src/spectator/multiple_expectations_failed.cr new file mode 100644 index 0000000..9a911fa --- /dev/null +++ b/src/spectator/multiple_expectations_failed.cr @@ -0,0 +1,16 @@ +require "./example_failed" +require "./expectation" + +module Spectator + # Exception that indicates more than one expectation from a test failed. + # When raised within a test, the test should abort. + class MultipleExpectationsFailed < ExampleFailed + # Expectations that failed. + getter expectations : Array(Expectation) + + # Creates the exception. + def initialize(@expectations : Array(Expectation), message : String? = nil, cause : Exception? = nil) + super(nil, message, cause) + end + end +end diff --git a/src/spectator/name_example_filter.cr b/src/spectator/name_example_filter.cr deleted file mode 100644 index a90560c..0000000 --- a/src/spectator/name_example_filter.cr +++ /dev/null @@ -1,13 +0,0 @@ -module Spectator - # Filter that matches examples based on their name. - class NameExampleFilter < ExampleFilter - # Creates the example filter. - def initialize(@name : String) - end - - # Checks whether the example satisfies the filter. - def includes?(example) : Bool - @name == example.to_s - end - end -end diff --git a/src/spectator/name_node_filter.cr b/src/spectator/name_node_filter.cr new file mode 100644 index 0000000..6d4e64a --- /dev/null +++ b/src/spectator/name_node_filter.cr @@ -0,0 +1,15 @@ +require "./node_filter" + +module Spectator + # Filter that matches nodes based on their name. + class NameNodeFilter < NodeFilter + # Creates the node filter. + def initialize(@name : String) + end + + # Checks whether the node satisfies the filter. + def includes?(node) : Bool + @name == node.to_s + end + end +end diff --git a/src/spectator/nested_example_group.cr b/src/spectator/nested_example_group.cr deleted file mode 100644 index c21345e..0000000 --- a/src/spectator/nested_example_group.cr +++ /dev/null @@ -1,54 +0,0 @@ -require "./example_group" - -module Spectator - # A collection of examples and other example groups. - # This group can be nested under other groups. - class NestedExampleGroup < ExampleGroup - # Description from the user of the group's contents. - # This is a symbol when referencing a type. - getter description : Symbol | String - - getter source : Source - - # Group that this is nested in. - getter parent : ExampleGroup - - # Creates a new example group. - # The *description* argument is a description from the user. - # The *parent* should contain this group. - # After creating this group, the parent's children should be updated. - # The parent's children must contain this group, - # otherwise there may be unexpected behavior. - # The *hooks* are stored to be triggered later. - def initialize(@description, @source, @parent, context) - super(context) - end - - # Indicates wheter the group references a type. - def symbolic? : Bool - @description.is_a?(Symbol) - end - - # Creates a string representation of the group. - # The string consists of `#description` appended to the parent. - # This results in a string like: - # ```text - # Foo#bar does something - # ``` - # for the following structure: - # ``` - # describe Foo do - # describe "#bar" do - # it "does something" do - # # ... - # end - # end - # end - # ``` - def to_s(io) - parent.to_s(io) - io << ' ' unless (symbolic? || parent.is_a?(RootExampleGroup)) && parent.symbolic? - io << description - end - end -end diff --git a/src/spectator/node.cr b/src/spectator/node.cr new file mode 100644 index 0000000..c5a64b6 --- /dev/null +++ b/src/spectator/node.cr @@ -0,0 +1,84 @@ +require "./label" +require "./location" +require "./metadata" + +module Spectator + # A single item in a test spec. + # This is commonly an `Example` or `ExampleGroup`, + # but can be anything that should be iterated over when running the spec. + abstract class Node + # Default text used if none was given by the user for skipping a node. + DEFAULT_PENDING_REASON = "No reason given" + + # Location of the node in source code. + getter! location : Location + + # User-provided name or description of the node. + # This does not include the group name or descriptions. + # Use `#to_s` to get the full name. + # + # This value will be nil if no name was provided. + # In this case, and the node is a runnable example, + # the name should be set to the description + # of the first matcher that runs in the test case. + # + # If this value is a `Symbol`, the user specified a type for the name. + getter! name : Label + + # Updates the name of the node. + protected def name=(@name : String) + end + + # User-defined tags and values used for filtering and behavior modification. + getter metadata : Metadata + + # Creates the node. + # The *name* describes the purpose of the node. + # It can be a `Symbol` to describe a type. + # The *location* tracks where the node exists in source code. + # A set of *metadata* can be used for filtering and modifying example behavior. + def initialize(@name : Label = nil, @location : Location? = nil, @metadata : Metadata = Metadata.new) + end + + # Indicates whether the node has completed. + abstract def finished? : Bool + + # Checks if the node has been marked as pending. + # Pending items should be skipped during execution. + def pending? + metadata.has_key?(:pending) || metadata.has_key?(:skip) + end + + # Gets the reason the node has been marked as pending. + def pending_reason + metadata[:pending]? || metadata[:skip]? || metadata[:reason]? || DEFAULT_PENDING_REASON + end + + # Retrieves just the tag names applied to the node. + def tags + Tags.new(metadata.keys) + end + + # Non-nil name used to show the node name. + def display_name + @name || "" + end + + # Constructs the full name or description of the node. + # This prepends names of groups this node is part of. + def to_s(io) + display_name.to_s(io) + end + + # Exposes information about the node useful for debugging. + def inspect(io) + # Full node name. + io << '"' << self << '"' + + # Add location if it's available. + if (location = self.location) + io << " @ " << location + end + end + end +end diff --git a/src/spectator/node_builder.cr b/src/spectator/node_builder.cr new file mode 100644 index 0000000..939374b --- /dev/null +++ b/src/spectator/node_builder.cr @@ -0,0 +1,6 @@ +module Spectator + abstract class NodeBuilder + # Produces a node for a spec. + abstract def build(parent = nil) + end +end diff --git a/src/spectator/node_filter.cr b/src/spectator/node_filter.cr new file mode 100644 index 0000000..6530d4a --- /dev/null +++ b/src/spectator/node_filter.cr @@ -0,0 +1,14 @@ +module Spectator + # Base class for all node filters. + # Checks whether a node should be included in the test run. + # Sub-classes must implement the `#includes?` method. + abstract class NodeFilter + # Checks if a node is in the filter, and should be included in the test run. + abstract def includes?(node) : Bool + + # :ditto: + def ===(node) + includes?(node) + end + end +end diff --git a/src/spectator/node_iterator.cr b/src/spectator/node_iterator.cr new file mode 100644 index 0000000..0dbc2bb --- /dev/null +++ b/src/spectator/node_iterator.cr @@ -0,0 +1,44 @@ +require "./node" + +module Spectator + # Iterates through all nodes in a group and its nested groups. + # Nodes are iterated in pre-order. + class NodeIterator + include Iterator(Node) + + # A stack is used to track where in the tree this iterator is. + @stack = Deque(Node).new(1) + + # Creates a new iterator. + # The *group* is the example group to iterate through. + def initialize(@group : Node) + @stack.push(@group) + end + + # Retrieves the next `Node`. + # If there are none left, then `Iterator::Stop` is returned. + def next + # Nothing left to iterate. + return stop if @stack.empty? + + # Retrieve the next node. + node = @stack.pop + + # If the node is a group, add its direct children to the queue + # in reverse order so that the tree is traversed in pre-order. + if node.is_a?(Indexable(Node)) + node.reverse_each { |child| @stack.push(child) } + end + + # Return the current node. + node + end + + # Restart the iterator at the beginning. + def rewind + @stack.clear + @stack.push(@group) + self + end + end +end diff --git a/src/spectator/null_context.cr b/src/spectator/null_context.cr new file mode 100644 index 0000000..3597203 --- /dev/null +++ b/src/spectator/null_context.cr @@ -0,0 +1,6 @@ +module Spectator + # Empty context used to construct examples that don't have contexts. + # This is useful for dynamically creating examples outside of a spec. + class NullContext < Context + end +end diff --git a/src/spectator/null_example_filter.cr b/src/spectator/null_example_filter.cr deleted file mode 100644 index 5a3e036..0000000 --- a/src/spectator/null_example_filter.cr +++ /dev/null @@ -1,9 +0,0 @@ -module Spectator - # Filter that matches all examples. - class NullExampleFilter < ExampleFilter - # Checks whether the example satisfies the filter. - def includes?(example) : Bool - true - end - end -end diff --git a/src/spectator/null_node_filter.cr b/src/spectator/null_node_filter.cr new file mode 100644 index 0000000..cee8736 --- /dev/null +++ b/src/spectator/null_node_filter.cr @@ -0,0 +1,16 @@ +require "./node_filter" + +module Spectator + # Filter that matches all nodes. + class NullNodeFilter < NodeFilter + # Creates the filter. + # The *match* flag indicates whether all examples should match or not. + def initialize(@match : Bool = true) + end + + # Checks whether the node satisfies the filter. + def includes?(_node) : Bool + @match + end + end +end diff --git a/src/spectator/pass_result.cr b/src/spectator/pass_result.cr new file mode 100644 index 0000000..2b62383 --- /dev/null +++ b/src/spectator/pass_result.cr @@ -0,0 +1,37 @@ +require "./result" + +module Spectator + # Outcome that indicates running an example was successful. + class PassResult < Result + # Calls the `pass` method on *visitor*. + def accept(visitor) + visitor.pass(self) + end + + # Calls the `pass` method on *visitor*. + def accept(visitor) + visitor.pass(yield self) + end + + # Indicates whether the example passed. + def pass? : Bool + true + end + + # Indicates whether the example failed. + def fail? : Bool + false + end + + # One-word description of the result. + def to_s(io) + io << "pass" + end + + # Creates a JSON object from the result information. + def to_json(json : JSON::Builder) + super + json.field("status", "passed") + end + end +end diff --git a/src/spectator/pending_example.cr b/src/spectator/pending_example.cr deleted file mode 100644 index 743b778..0000000 --- a/src/spectator/pending_example.cr +++ /dev/null @@ -1,12 +0,0 @@ -require "./example" - -module Spectator - # Common class for all examples marked as pending. - # This class will not run example code. - class PendingExample < Example - # Returns a pending result. - private def run_impl : Result - PendingResult.new(self) - end - end -end diff --git a/src/spectator/pending_example_builder.cr b/src/spectator/pending_example_builder.cr new file mode 100644 index 0000000..a1f0292 --- /dev/null +++ b/src/spectator/pending_example_builder.cr @@ -0,0 +1,24 @@ +require "./example" +require "./location" +require "./metadata" +require "./node_builder" + +module Spectator + # Constructs pending examples. + # Call `#build` to produce an `Example`. + class PendingExampleBuilder < NodeBuilder + # Creates the builder. + # The *name*, *location*, and *metadata* will be applied to the `Example` produced by `#build`. + # A default *reason* can be given in case the user didn't provide one. + def initialize(@name : String? = nil, @location : Location? = nil, + @metadata : Metadata = Metadata.new, @reason : String? = nil) + end + + # Constructs an example with previously defined attributes. + # The *parent* is an already constructed example group to nest the new example under. + # It can be nil if the new example won't have a parent. + def build(parent = nil) + Example.pending(@name, @location, parent, @metadata, @reason) + end + end +end diff --git a/src/spectator/pending_result.cr b/src/spectator/pending_result.cr index 65512e9..03700d9 100644 --- a/src/spectator/pending_result.cr +++ b/src/spectator/pending_result.cr @@ -5,21 +5,53 @@ module Spectator # A pending result means the example is not ready to run yet. # This can happen when the functionality to be tested is not implemented yet. class PendingResult < Result - # Calls the `pending` method on *interface*. - def call(interface) - interface.pending + DEFAULT_REASON = "No reason given" + + # Reason the example was skipped or marked pending. + getter reason : String + + # Location the pending result was triggered from. + getter! location : Location + + # Creates the result. + # *elapsed* is the length of time it took to run the example. + # A *reason* for the skip/pending result can be specified. + # If the pending result was triggered inside of an example, then *location* can be set. + def initialize(@reason = DEFAULT_REASON, @location = nil, + elapsed = Time::Span::ZERO, expectations = [] of Expectation) + super(elapsed, expectations) end - # Calls the `pending` method on *interface* - # and passes the yielded value. - def call(interface) - value = yield self - interface.pending(value) + # Calls the `pending` method on the *visitor*. + def accept(visitor) + visitor.pending(self) end - # One-word descriptor of the result. + # Calls the `pending` method on the *visitor*. + def accept(visitor) + visitor.pending(yield self) + end + + # Indicates whether the example passed. + def pass? : Bool + false + end + + # Indicates whether the example failed. + def fail? : Bool + false + end + + # One-word description of the result. def to_s(io) io << "pending" end + + # Creates a JSON object from the result information. + def to_json(json : JSON::Builder) + super + json.field("status", "pending") + json.field("pending_message", @reason) + end end end diff --git a/src/spectator/profile.cr b/src/spectator/profile.cr index ff6f426..382b85a 100644 --- a/src/spectator/profile.cr +++ b/src/spectator/profile.cr @@ -1,14 +1,26 @@ +require "json" +require "./example" + module Spectator # Information about the runtimes of examples. class Profile - include Indexable(Result) + include Indexable(Example) # Total length of time it took to run all examples in the test suite. getter total_time : Time::Span # Creates the profiling information. # The *slowest* results must already be sorted, longest time first. - private def initialize(@slowest : Array(FinishedResult), @total_time) + def initialize(@slowest : Array(Example), @total_time) + end + + # Produces the profile from a report. + def self.generate(examples, size = 10) + finished = examples.select(&.finished?).to_a + total_time = finished.sum(&.result.elapsed) + sorted_examples = finished.sort_by(&.result.elapsed) + slowest = sorted_examples.last(size).reverse + new(slowest, total_time) end # Number of results in the profile. @@ -23,20 +35,27 @@ module Spectator # Length of time it took to run the results in the profile. def time - @slowest.sum(&.elapsed) + @slowest.sum(&.result.elapsed) end - # Percentage (from 0 to 1) of time the results in this profile took compared to all examples. + # Percentage (from 0 to 100) of time the results in this profile took compared to all examples. def percentage - time / @total_time + time / @total_time * 100 end - # Produces the profile from a report. - def self.generate(report, size = 10) - results = report.compact_map(&.as?(FinishedResult)) - sorted_results = results.sort_by(&.elapsed) - slowest = sorted_results.last(size).reverse - self.new(slowest, report.example_runtime) + # Produces a JSON fragment containing the profiling information. + def to_json(json : JSON::Builder) + json.object do + json.field("examples") do + json.array do + @slowest.each(&.to_json(json)) + end + end + + json.field("slowest", @slowest.max_of(&.result.elapsed).total_seconds) + json.field("total", time.total_seconds) + json.field("percentage", percentage) + end end end end diff --git a/src/spectator/report.cr b/src/spectator/report.cr index 9ce1ebd..93a88ea 100644 --- a/src/spectator/report.cr +++ b/src/spectator/report.cr @@ -3,113 +3,130 @@ require "./result" module Spectator # Outcome of all tests in a suite. class Report - include Enumerable(Result) + # Records the number of examples that had each type of result. + record Counts, pass = 0, fail = 0, error = 0, pending = 0, remaining = 0 do + # Number of examples that actually ran. + def run + pass + fail + pending + end + + # Total number of examples in the suite that were selected to run. + def total + run + remaining + end + + # Indicates whether there were skipped tests + # because of a failure causing the test suite to abort. + def remaining? + remaining > 0 + end + end + + # Retrieves all examples that were planned to run as part of the suite. + getter examples : Array(Example) # Total length of time it took to execute the test suite. # This includes examples, hooks, and framework processes. getter runtime : Time::Span - # Number of passing examples. - getter successful_count = 0 + # Number of examples of each result type. + getter counts : Counts - # Number of failing examples (includes errors). - getter failed_count = 0 - - # Number of examples that had errors. - getter error_count = 0 - - # Number of pending examples. - getter pending_count = 0 - - # Number of remaining tests. - # This will be greater than zero only in fail-fast mode. - getter remaining_count - - # Random seed used to determine test ordering. + # Seed used for random number generation. getter! random_seed : UInt64? # Creates the report. - # The *results* are from running the examples in the test suite. + # The *examples* are all examples in the test suite that were selected to run. # The *runtime* is the total time it took to execute the suite. - # The *remaining_count* is the number of tests skipped due to fail-fast. - # The *fail_blank* flag indicates whether it is a failure if there were no tests run. + # The *counts* is the number of examples for each type of result. # The *random_seed* is the seed used for random number generation. - def initialize(@results : Array(Result), @runtime, @remaining_count = 0, @fail_blank = false, @random_seed = nil) - @results.each do |result| - case result - when SuccessfulResult - @successful_count += 1 - when ErroredResult - @error_count += 1 - @failed_count += 1 - when FailedResult - @failed_count += 1 - when PendingResult - @pending_count += 1 - when Result - # This case isn't possible, but gets the compiler to stop complaining. - nil + def initialize(@examples : Array(Example), @runtime, @counts : Counts, @random_seed = nil) + end + + # Generates the report from a set of examples. + def self.generate(examples : Enumerable(Example), runtime, random_seed = nil) + counts = count_examples(examples) + new(examples.to_a, runtime, counts, random_seed) + end + + # Counts the number of examples for each result type. + private def self.count_examples(examples) + visitor = CountVisitor.new + + # Number of tests not run. + remaining = 0 + + # Iterate through each example and count the number of each type of result. + # If an example hasn't run (indicated by `Node#finished?`), then count is as "remaining." + # This typically happens in fail-fast mode. + examples.each do |example| + if example.finished? + example.result.accept(visitor) + else + remaining += 1 end end + + visitor.counts(remaining) end - # Creates the report. - # This constructor is intended for reports of subsets of results. - # The *results* are from running the examples in the test suite. - # The runtime is calculated from the *results*. - def initialize(results : Array(Result)) - runtime = results.each.compact_map(&.as?(FinishedResult)).sum(&.elapsed) - initialize(results, runtime) - end - - # Yields each result in turn. - def each - @results.each do |result| - yield result - end - end - - # Number of examples. - def example_count - @results.size - end - - # Number of examples run (not skipped or pending). - def examples_ran - @successful_count + @failed_count - end - - # Indicates whether the test suite failed. - def failed? - failed_count > 0 || (@fail_blank && examples_ran == 0) - end - - # Indicates whether there were skipped tests - # because of a failure causing the test to abort. - def remaining? - remaining_count > 0 - end - - # Returns a set of results for all failed examples. + # Returns a collection of all failed examples. def failures - @results.each.compact_map(&.as?(FailedResult)) + @examples.select(&.result.fail?) end - # Returns a set of results for all errored examples. - def errors - @results.each.compact_map(&.as?(ErroredResult)) + # Returns a collection of all pending (skipped) examples. + def pending + @examples.select(&.result.pending?) end # Length of time it took to run just example code. # This does not include hooks, # but it does include pre- and post-conditions. def example_runtime - @results.each.compact_map(&.as?(FinishedResult)).sum(&.elapsed) + @examples.sum(&.result.elapsed) end # Length of time spent in framework processes and hooks. def overhead_time @runtime - example_runtime end + + # Totals up the number of each type of result. + # Defines methods for the different types of results. + # Call `#counts` to retrieve the `Counts` instance. + private class CountVisitor + @pass = 0 + @fail = 0 + @error = 0 + @pending = 0 + + # Increments the number of passing examples. + def pass(_result) + @pass += 1 + end + + # Increments the number of failing (non-error) examples. + def fail(_result) + @fail += 1 + end + + # Increments the number of error (and failed) examples. + def error(result) + fail(result) + @error += 1 + end + + # Increments the number of pending (skipped) examples. + def pending(_result) + @pending += 1 + end + + # Produces the total counts. + # The *remaining* number of examples should be provided. + def counts(remaining) + Counts.new(@pass, @fail, @error, @pending, remaining) + end + end end end diff --git a/src/spectator/result.cr b/src/spectator/result.cr index c2f0b45..2da334d 100644 --- a/src/spectator/result.cr +++ b/src/spectator/result.cr @@ -1,40 +1,44 @@ +require "json" +require "./expectation" + module Spectator # Base class that represents the outcome of running an example. # Sub-classes contain additional information specific to the type of result. abstract class Result - # Example that was run that this result is for. - getter example : Example + # Length of time it took to run the example. + getter elapsed : Time::Span - # Constructs the base of the result. - # The *example* should refer to the example that was run - # and that this result is for. - def initialize(@example) + # The assertions checked in the example. + getter expectations : Enumerable(Expectation) + + # Creates the result. + # *elapsed* is the length of time it took to run the example. + def initialize(@elapsed, @expectations = [] of Expectation) end # Calls the corresponding method for the type of result. - # This is used to avoid placing if or case-statements everywhere based on type. - # Each sub-class implements this method by calling the correct method on *interface*. - abstract def call(interface) + # This is the visitor design pattern. + abstract def accept(visitor) - # Calls the corresponding method for the type of result. - # This is used to avoid placing if or case-statements everywhere based on type. - # Each sub-class implements this method by calling the correct method on *interface*. - # This variation takes a block, which is passed the result. - # The value returned from the block will be returned by this method. - abstract def call(interface, &block : Result -> _) + # Indicates whether the example passed. + abstract def pass? : Bool + + # Indicates whether the example failed. + abstract def fail? : Bool + + # Indicates whether the example was skipped. + def pending? : Bool + !pass? && !fail? + end # Creates a JSON object from the result information. - def to_json(json : ::JSON::Builder) - json.object do - add_json_fields(json) + def to_json(json : JSON::Builder) + json.field("run_time", @elapsed.total_seconds) + json.field("expectations") do + json.array do + @expectations.each(&.to_json(json)) + end end end - - # Adds the common fields for a result to a JSON builder. - private def add_json_fields(json : ::JSON::Builder) - json.field("name", example) - json.field("location", example.source) - json.field("result", to_s) - end end end diff --git a/src/spectator/root_example_group.cr b/src/spectator/root_example_group.cr deleted file mode 100644 index 16380e0..0000000 --- a/src/spectator/root_example_group.cr +++ /dev/null @@ -1,28 +0,0 @@ -require "./example_group" - -module Spectator - # Top-most group of examples and sub-groups. - # The root has no parent. - class RootExampleGroup < ExampleGroup - # Dummy value - this should never be used. - def description : Symbol | String - :root - end - - def source : Source - Source.new(__FILE__, __LINE__) - end - - # Indicates that the group is symbolic. - def symbolic? : Bool - true - end - - # Does nothing. - # This prevents the root group - # from showing up in output. - def to_s(io) - # ... - end - end -end diff --git a/src/spectator/run_flags.cr b/src/spectator/run_flags.cr new file mode 100644 index 0000000..3ce2d9e --- /dev/null +++ b/src/spectator/run_flags.cr @@ -0,0 +1,21 @@ +module Spectator + # Toggles indicating how the test spec should execute. + @[Flags] + enum RunFlags + # Indicates whether the test should abort on first failure. + FailFast + + # Indicates whether the test should fail if there are no examples. + FailBlank + + # Indicates whether the test should be done as a dry-run. + # Examples won't run, but the output will show that they did. + DryRun + + # Indicates whether examples run in a random order. + Randomize + + # Indicates whether timing information should be generated. + Profile + end +end diff --git a/src/spectator/runnable_example.cr b/src/spectator/runnable_example.cr deleted file mode 100644 index 43f4018..0000000 --- a/src/spectator/runnable_example.cr +++ /dev/null @@ -1,83 +0,0 @@ -require "./example" - -module Spectator - # Includes all the logic for running example hooks, - # the example code, and capturing a result. - class RunnableExample < Example - # Runs the example, hooks, and captures the result - # and translates to a usable result. - def run_impl : Result - result = capture_result - expectations = Harness.current.expectations - translate_result(result, expectations) - end - - # Runs all hooks and the example code. - # A captured result is returned. - private def capture_result - context = group.context - ResultCapture.new.tap do |result| - context.run_before_hooks(self) - run_example(result) - @finished = true - context.run_after_hooks(self) - result.error = nil if result.error.is_a?(SystemExit) && Harness.current.mocks.exit_handled? - run_deferred(result) unless result.error - end - end - - # Runs the test code and captures the result. - private def run_example(result) - context = group.context - wrapper = test_wrapper.around_hook(context) - - # Capture how long it takes to run the test code. - result.elapsed = Time.measure do - begin - context.run_pre_conditions(self) - wrapper.call - context.run_post_conditions(self) - rescue ex # Catch all errors and handle them later. - result.error = ex - end - end - end - - # Runs the deferred blocks of code and captures the result. - private def run_deferred(result) - result.elapsed += Time.measure do - begin - Harness.current.run_deferred - rescue ex # Catch all errors and handle them later. - result.error = ex - end - end - end - - # Creates a result instance from captured result information. - private def translate_result(result, expectations) - case (error = result.error) - when Nil - # If no errors occurred, then the example ran successfully. - SuccessfulResult.new(self, result.elapsed, expectations) - when ExampleFailed - # If a test fails or required expectation is not met, then an `ExampleFailed` exception was raised. - FailedResult.new(self, result.elapsed, expectations, error) - else - # Any other exception that is raised is unexpected and is an errored result. - ErroredResult.new(self, result.elapsed, expectations, error) - end - end - - # Utility class for storing parts of the result while the example is running. - private class ResultCapture - # Length of time that it took to run the test code. - # This does not include hooks. - property elapsed = Time::Span.zero - - # The error that occurred while running the test code. - # If no error occurred, this will be nil. - property error : Exception? - end - end -end diff --git a/src/spectator/runner.cr b/src/spectator/runner.cr index d5c50f9..49b6065 100644 --- a/src/spectator/runner.cr +++ b/src/spectator/runner.cr @@ -1,81 +1,98 @@ -require "./harness" +require "./example" +require "./formatting/formatter" +require "./profile" +require "./report" +require "./run_flags" +require "./runner_events" module Spectator - # Main driver for executing tests and feeding results to formatters. - class Runner - # Creates the test suite runner. - # Specify the test *suite* to run and any additonal configuration. - def initialize(@suite : TestSuite, @config : Config) + # Logic for executing examples and collecting results. + struct Runner + include RunnerEvents + + # Formatter to send events to. + private getter formatter : Formatting::Formatter + + # Creates the runner. + # The collection of *examples* should be pre-filtered and shuffled. + # This runner will run each example in the order provided. + # The *formatter* will be called for various events. + def initialize(@examples : Array(Example), @formatter : Formatting::Formatter, + @run_flags = RunFlags::None, @random_seed : UInt64? = nil) end - # Runs the test suite. - # This will run the selected examples - # and invoke the formatter to output results. - # True will be returned if the test suite ran successfully, + # Runs the spec. + # This will run the provided examples + # and invoke the reporters to communicate results. + # True will be returned if the spec ran successfully, # or false if there was at least one failure. def run : Bool - # Indicate the suite is starting. - @config.each_formatter(&.start_suite(@suite)) + start + elapsed = Time.measure { run_examples } + stop - # Run all examples and capture the results. - results = Array(Result).new(@suite.size) - elapsed = Time.measure do - collect_results(results) - end + report = Report.generate(@examples, elapsed, @random_seed) + profile = Profile.generate(@examples) if @run_flags.profile? && report.counts.run > 0 + summarize(report, profile) - # Generate a report and pass it along to the formatter. - remaining = @suite.size - results.size - seed = (@config.random_seed? if @config.randomize?) - report = Report.new(results, elapsed, remaining, @config.fail_blank?, seed) - @config.each_formatter(&.end_suite(report, profile(report))) - - !report.failed? + report.counts.fail.zero? + ensure + close end - # Runs all examples and adds results to a list. - private def collect_results(results) - example_order.each do |example| - result = run_example(example).as(Result) - results << result - if @config.fail_fast? && result.is_a?(FailedResult) - example.group.context.run_after_all_hooks(example.group, ignore_unfinished: true) - break - end - end - end + # Attempts to run all examples. + # Returns a list of examples that ran. + private def run_examples + @examples.each do |example| + result = run_example(example) - # Retrieves an enumerable for the examples to run. - # The order of examples is randomized - # if specified by the configuration. - private def example_order - @suite.to_a.tap do |examples| - examples.shuffle!(@config.random) if @config.randomize? + # Bail out if the example failed + # and configured to stop after the first failure. + break fail_fast if fail_fast? && result.fail? end end # Runs a single example and returns the result. # The formatter is given the example and result information. private def run_example(example) - @config.each_formatter(&.start_example(example)) - result = if @config.dry_run? && example.is_a?(RunnableExample) - dry_run_result(example) + example_started(example) + result = if dry_run? + # TODO: Pending examples return a pending result instead of pass in RSpec dry-run. + dry_run_result else - Harness.run(example) + example.run end - @config.each_formatter(&.end_example(result)) + example_finished(example) result end - # Creates a fake result for an example. - private def dry_run_result(example) - expectations = [] of Expectations::Expectation - example_expectations = Expectations::ExampleExpectations.new(expectations) - SuccessfulResult.new(example, Time::Span.zero, example_expectations) + # Creates a fake result. + private def dry_run_result + expectations = [] of Expectation + PassResult.new(Time::Span.zero, expectations) end # Generates and returns a profile if one should be displayed. private def profile(report) Profile.generate(report) if @config.profile? end + + # Indicates whether examples should be simulated, but not run. + private def dry_run? + @run_flags.dry_run? + end + + # Indicates whether test execution should stop after the first failure. + private def fail_fast? + @run_flags.fail_fast? + end + + private def fail_fast : Nil + end + + # Number of examples configured to run. + private def example_count + @examples.size + end end end diff --git a/src/spectator/runner_events.cr b/src/spectator/runner_events.cr new file mode 100644 index 0000000..8f4e97e --- /dev/null +++ b/src/spectator/runner_events.cr @@ -0,0 +1,93 @@ +require "./formatting/formatter" +require "./formatting/notifications" + +module Spectator + # Mix-in for announcing events from a `Runner`. + # All events invoke their corresponding method on the formatter. + module RunnerEvents + # Triggers the 'start' event. + # See `Formatting::Formatter#start` + private def start + notification = Formatting::StartNotification.new(example_count) + formatter.start(notification) + end + + # Triggers the 'example started' event. + # Must be passed the *example* about to run. + # See `Formatting::Formatter#example_started` + private def example_started(example) + notification = Formatting::ExampleNotification.new(example) + formatter.example_started(notification) + end + + # Triggers the 'example started' event. + # Also triggers the example result event corresponding to the example's outcome. + # Must be passed the completed *example*. + # See `Formatting::Formatter#example_finished` + private def example_finished(example) + notification = Formatting::ExampleNotification.new(example) + visitor = ResultVisitor.new(formatter, notification) + formatter.example_finished(notification) + example.result.accept(visitor) + end + + # Triggers the 'stop' event. + # See `Formatting::Formatter#stop` + private def stop + formatter.stop + end + + # Triggers the 'dump' events. + private def summarize(report, profile) + formatter.start_dump + + notification = Formatting::ExampleSummaryNotification.new(report.pending) + formatter.dump_pending(notification) + + notification = Formatting::ExampleSummaryNotification.new(report.failures) + formatter.dump_failures(notification) + + if profile + notification = Formatting::ProfileNotification.new(profile) + formatter.dump_profile(notification) + end + + notification = Formatting::SummaryNotification.new(report) + formatter.dump_summary(notification) + end + + # Triggers the 'close' event. + # See `Formatting::Formatter#close` + private def close + formatter.close + end + + # Provides methods for the various result types. + private struct ResultVisitor + # Creates the visitor. + # Requires the *formatter* to notify and the *notification* to send it. + def initialize(@formatter : Formatting::Formatter, @notification : Formatting::ExampleNotification) + end + + # Invokes the example passed method. + def pass(_result) + @formatter.example_passed(@notification) + end + + # Invokes the example failed method. + def fail(_result) + @formatter.example_failed(@notification) + end + + # Invokes the example error method. + def error(_result) + @formatter.example_error(@notification) + end + + # Invokes the example pending method. + def pending(_result) + @formatter.example_pending(@notification) + end + end + end +end diff --git a/src/spectator/should.cr b/src/spectator/should.cr index b8cda4a..71e5574 100644 --- a/src/spectator/should.cr +++ b/src/spectator/should.cr @@ -9,6 +9,12 @@ class Object # end # ``` # + # An optional message can be used in case the expectation fails. + # It can be a string or proc returning a string. + # ``` + # subject.should_not be_nil, "Shouldn't be nil" + # ``` + # # NOTE: By default, the should-syntax is disabled. # The expect-syntax is preferred, # since it doesn't [monkey-patch](https://en.wikipedia.org/wiki/Monkey_patch) all objects. @@ -16,51 +22,69 @@ class Object # ``` # require "spectator/should" # ``` - def should(matcher) - # First argument of the `Expectation` initializer is the expression label. - # However, since this isn't a macro and we can't "look behind" this method call - # to see what it was invoked on, the argument is an empty string. - # Additionally, the source file and line can't be obtained. - actual = ::Spectator::TestValue.new(self) - source = ::Spectator::Source.new(__FILE__, __LINE__) - ::Spectator::Expectations::ExpectationPartial.new(actual, source).to(matcher) + def should(matcher, message = nil) + actual = ::Spectator::Value.new(self) + match_data = matcher.match(actual) + expectation = ::Spectator::Expectation.new(match_data, message: message) + ::Spectator::Harness.current.report(expectation) end # Works the same as `#should` except the condition is inverted. # When `#should` succeeds, this method will fail, and vice-versa. - def should_not(matcher) - actual = ::Spectator::TestValue.new(self) - source = ::Spectator::Source.new(__FILE__, __LINE__) - ::Spectator::Expectations::ExpectationPartial.new(actual, source).to_not(matcher) + def should_not(matcher, message = nil) + actual = ::Spectator::Value.new(self) + match_data = matcher.negated_match(actual) + expectation = ::Spectator::Expectation.new(match_data, message: message) + ::Spectator::Harness.current.report(expectation) end # Works the same as `#should` except that the condition check is postphoned. # The expectation is checked after the example finishes and all hooks have run. - def should_eventually(matcher) - ::Spectator::Harness.current.defer { should(matcher) } + def should_eventually(matcher, message = nil) + ::Spectator::Harness.current.defer { should(matcher, message) } end # Works the same as `#should_not` except that the condition check is postphoned. # The expectation is checked after the example finishes and all hooks have run. - def should_never(matcher) - ::Spectator::Harness.current.defer { should_not(matcher) } + def should_never(matcher, message = nil) + ::Spectator::Harness.current.defer { should_not(matcher, message) } end end struct Proc(*T, R) # Extension method to create an expectation for a block of code (proc). # Depending on the matcher, the proc may be executed multiple times. - def should(matcher) - actual = ::Spectator::TestBlock.new(self) - source = ::Spectator::Source.new(__FILE__, __LINE__) - ::Spectator::Expectations::ExpectationPartial.new(actual, source).to(matcher) + def should(matcher, message = nil) + actual = ::Spectator::Block.new(self) + match_data = matcher.match(actual) + expectation = ::Spectator::Expectation.new(match_data, message: message) + ::Spectator::Harness.current.report(expectation) end # Works the same as `#should` except the condition is inverted. # When `#should` succeeds, this method will fail, and vice-versa. - def should_not(matcher) - actual = ::Spectator::TestBlock.new(self) - source = ::Spectator::Source.new(__FILE__, __LINE__) - ::Spectator::Expectations::BlockExpectationPartial.new(actual, source).to_not(matcher) + def should_not(matcher, message = nil) + actual = ::Spectator::Block.new(self) + match_data = matcher.negated_match(actual) + expectation = ::Spectator::Expectation.new(match_data, message: message) + ::Spectator::Harness.current.report(expectation) + end +end + +module Spectator::DSL::Expectations + macro should(*args) + expect(subject).to({{args.splat}}) + end + + macro should_not(*args) + expect(subject).to_not({{args.splat}}) + end + + macro should_eventually(*args) + expect(subject).to_eventually({{args.splat}}) + end + + macro should_never(*args) + expect(subject).to_never({{args.splat}}) end end diff --git a/src/spectator/source_example_filter.cr b/src/spectator/source_example_filter.cr deleted file mode 100644 index d3ad4dc..0000000 --- a/src/spectator/source_example_filter.cr +++ /dev/null @@ -1,14 +0,0 @@ -module Spectator - # Filter that matches examples in a given file and line. - class SourceExampleFilter < ExampleFilter - # Creates the filter. - # The *source* indicates which file and line the example must be on. - def initialize(@source : Source) - end - - # Checks whether the example satisfies the filter. - def includes?(example) : Bool - @source === example.source - end - end -end diff --git a/src/spectator/spec.cr b/src/spectator/spec.cr new file mode 100644 index 0000000..a086171 --- /dev/null +++ b/src/spectator/spec.cr @@ -0,0 +1,30 @@ +require "./config" +require "./example_group" +require "./runner" + +module Spectator + # Contains examples to be tested and configuration for running them. + class Spec + # Creates the spec. + # The *root* is the top-most example group. + # All examples in this group and groups nested under are candidates for execution. + # The *config* provides settings controlling how tests will be executed. + def initialize(@root : ExampleGroup, @config : Config) + end + + # Runs all selected examples and returns the results. + # True will be returned if the spec ran successfully, + # or false if there was at least one failure. + def run : Bool + random_seed = (@config.random_seed if @config.run_flags.randomize?) + runner = Runner.new(examples, @config.formatter, @config.run_flags, random_seed) + runner.run + end + + # Selects and shuffles the examples that should run. + private def examples + iterator = @config.iterator(@root) + @config.shuffle!(iterator.to_a) + end + end +end diff --git a/src/spectator/spec_builder.cr b/src/spectator/spec_builder.cr index 39834be..c4d1485 100644 --- a/src/spectator/spec_builder.cr +++ b/src/spectator/spec_builder.cr @@ -1,110 +1,212 @@ -require "./spec_builder/*" +require "./config" +require "./example" +require "./example_builder" +require "./example_context_method" +require "./example_group" +require "./example_group_builder" +require "./hooks" +require "./iterative_example_group_builder" +require "./pending_example_builder" +require "./spec" +require "./metadata" module Spectator - # Global builder used to create the runtime instance of the spec. - # The DSL methods call into this module to generate parts of the spec. - # Once the DSL is done, the `#build` method can be invoked - # to create the entire spec as a runtime instance. - module SpecBuilder - extend self + # Progressively builds a test spec. + # + # A stack is used to track the current example group. + # Adding an example or group will nest it under the group at the top of the stack. + class SpecBuilder + Log = ::Spectator::Log.for(self) - @@stack = ExampleGroupStack.new + delegate before_all, after_all, before_each, after_each, around_each, to: current - # Begins a new nested group in the spec. - # A corresponding `#end_group` call must be made - # when the group being started is finished. - # See `NestedExampleGroupBuilder#initialize` for the arguments - # as arguments to this method are passed directly to it. - def start_group(*args) : Nil - group = NestedExampleGroupBuilder.new(*args) - @@stack.push(group) + # Stack tracking the current group. + # The bottom of the stack (first element) is the root group. + # The root group should never be removed. + # The top of the stack (last element) is the current group. + # New examples should be added to the current group. + @stack : Deque(ExampleGroupBuilder) + + # Creates a new spec builder. + # A root group is pushed onto the group stack. + def initialize(@config : Config) + root = ExampleGroupBuilder.new + @stack = Deque(ExampleGroupBuilder).new + @stack.push(root) end - # Begins a new sample group in the spec - - # that is, a group defined by the `StructureDSL#sample` macro in the DSL. - # A corresponding `#end_group` call must be made - # when the group being started is finished. - # See `SampleExampleGroupBuilder#initialize` for the arguments - # as arguments to this method are passed directly to it. - def start_sample_group(*args, &block : TestValues -> Array(T)) : Nil forall T - group = SampleExampleGroupBuilder(T).new(*args, block) - @@stack.push(group) + # Constructs the test spec. + # Returns the spec instance. + # + # Raises an error if there were not symmetrical calls to `#start_group` and `#end_group`. + # This would indicate a logical error somewhere in Spectator or an extension of it. + def build : Spec + raise "Mismatched start and end groups" unless root? + + group = root.build + apply_config_hooks(group) + Spec.new(group, @config) end - # Marks the end of a group in the spec. - # This must be called for every `#start_group` and `#start_sample_group` call. - # It is also important to line up the start and end calls. - # Otherwise examples might get placed into wrong groups. + # Defines a new example group and pushes it onto the group stack. + # Examples and groups defined after calling this method will be nested under the new group. + # The group will be finished and popped off the stack when `#end_example` is called. + # + # The *name* is the name or brief description of the group. + # This should be a symbol when describing a type - the type name is represented as a symbol. + # Otherwise, a string should be used. + # + # The *location* optionally defined where the group originates in source code. + # + # A set of *metadata* can be used for filtering and modifying example behavior. + # For instance, adding a "pending" tag will mark tests as pending and skip execution. + def start_group(name, location = nil, metadata = Metadata.new) : Nil + Log.trace { "Start group: #{name.inspect} @ #{location}; metadata: #{metadata}" } + builder = ExampleGroupBuilder.new(name, location, metadata) + + # `before_all` and `after_all` hooks from config are slightly different. + # They are applied to every top-level group (groups just under root). + apply_top_level_config_hooks(builder) if root? + + # Add group to the stack. + current << builder + @stack.push(builder) + end + + # Defines a new iterative example group and pushes it onto the group stack. + # Examples and groups defined after calling this method will be nested under the new group. + # The group will be finished and popped off the stack when `#end_example` is called. + # + # The *collection* is the set of items to iterate over. + # Child nodes in this group will be executed once for every item in the collection. + # The *name* should be a string representation of *collection*. + # The *iterator* is an optional name given to a single item in *collection*. + # + # The *location* optionally defined where the group originates in source code. + # + # A set of *metadata* can be used for filtering and modifying example behavior. + # For instance, adding a "pending" tag will mark tests as pending and skip execution. + def start_iterative_group(collection, name, iterator = nil, location = nil, metadata = Metadata.new) : Nil + Log.trace { "Start iterative group: #{name} (#{typeof(collection)}) @ #{location}; metadata: #{metadata}" } + builder = IterativeExampleGroupBuilder.new(collection, name, iterator, location, metadata) + + # `before_all` and `after_all` hooks from config are slightly different. + # They are applied to every top-level group (groups just under root). + apply_top_level_config_hooks(builder) if root? + + # Add group to the stack. + current << builder + @stack.push(builder) + end + + # Completes a previously defined example group and pops it off the group stack. + # Be sure to call `#start_group` and `#end_group` symmetically. def end_group : Nil - @@stack.pop + Log.trace { "End group: #{current}" } + raise "Can't pop root group" if root? + + @stack.pop end - # Adds an example type to the current group. - # The class name of the example should be passed as an argument. - # The example will be instantiated later. - def add_example(description : String?, source : Source, - example_type : ::SpectatorTest.class, &runner : ::SpectatorTest ->) : Nil - builder = ->(values : TestValues) { example_type.new(values).as(::SpectatorTest) } - factory = RunnableExampleBuilder.new(description, source, builder, runner) - @@stack.current.add_child(factory) + # Defines a new example. + # The example is added to the group currently on the top of the stack. + # + # The *name* is the name or brief description of the example. + # This should be a string or nil. + # When nil, the example's name will be populated by the first expectation run inside of the test code. + # + # The *location* optionally defined where the example originates in source code. + # + # The *context_builder* is a proc that creates a context the test code should run in. + # See `Context` for more information. + # + # A set of *metadata* can be used for filtering and modifying example behavior. + # For instance, adding a "pending" tag will mark the test as pending and skip execution. + # + # A block must be provided. + # It will be yielded two arguments - the example created by this method, and the *context* argument. + # The return value of the block is ignored. + # It is expected that the test code runs when the block is called. + def add_example(name, location, context_builder, metadata = Metadata.new, &block : Example -> _) : Nil + Log.trace { "Add example: #{name} @ #{location}; metadata: #{metadata}" } + current << ExampleBuilder.new(context_builder, block, name, location, metadata) end - # Adds an example type to the current group. - # The class name of the example should be passed as an argument. - # The example will be instantiated later. - def add_pending_example(description : String?, source : Source, - example_type : ::SpectatorTest.class, &runner : ::SpectatorTest ->) : Nil - builder = ->(values : TestValues) { example_type.new(values).as(::SpectatorTest) } - factory = PendingExampleBuilder.new(description, source, builder, runner) - @@stack.current.add_child(factory) + # Defines a new pending example. + # The example is added to the group currently on the top of the stack. + # + # The *name* is the name or brief description of the example. + # This should be a string or nil. + # When nil, the example's name will be an anonymous example reference. + # + # The *location* optionally defined where the example originates in source code. + # + # A set of *metadata* can be used for filtering and modifying example behavior. + # For instance, adding a "pending" tag will mark the test as pending and skip execution. + # A default *reason* can be given in case the user didn't provide one. + def add_pending_example(name, location, metadata = Metadata.new, reason = nil) : Nil + Log.trace { "Add pending example: #{name} @ #{location}; metadata: #{metadata}" } + current << PendingExampleBuilder.new(name, location, metadata, reason) end - # Adds a block of code to run before all examples in the current group. - def add_before_all_hook(&block : ->) : Nil - @@stack.current.add_before_all_hook(block) + # Registers a new "before_suite" hook. + # The hook will be appended to the list. + # A new hook will be created by passing args to `ExampleGroupHook.new`. + def before_suite(*args, **kwargs) : Nil + root.before_all(*args, **kwargs) end - # Adds a block of code to run before each example in the current group. - def add_before_each_hook(&block : TestMetaMethod) : Nil - @@stack.current.add_before_each_hook(block) + # Registers a new "before_suite" hook. + # The hook will be appended to the list. + # A new hook will be created by passing args to `ExampleGroupHook.new`. + def before_suite(*args, **kwargs, &block) : Nil + root.before_all(*args, **kwargs, &block) end - # Adds a block of code to run after all examples in the current group. - def add_after_all_hook(&block : ->) : Nil - @@stack.current.add_after_all_hook(block) + # Registers a new "after_suite" hook. + # The hook will be prepended to the list. + # A new hook will be created by passing args to `ExampleGroupHook.new`. + def after_suite(*args, **kwargs) : Nil + root.before_all(*args, **kwargs) end - # Adds a block of code to run after each example in the current group. - def add_after_each_hook(&block : TestMetaMethod) : Nil - @@stack.current.add_after_each_hook(block) + # Registers a new "after_suite" hook. + # The hook will be prepended to the list. + # A new hook will be created by passing args to `ExampleGroupHook.new`. + def after_suite(*args, **kwargs, &block) : Nil + root.after_all(*args, **kwargs, &block) end - # Adds a block of code to run before and after each example in the current group. - # The block of code will be given another hook as an argument. - # It is expected that the block will call the hook. - def add_around_each_hook(&block : ::SpectatorTest, Proc(Nil) ->) : Nil - @@stack.current.add_around_each_hook(block) + # Checks if the current group is the root group. + private def root? + @stack.size == 1 end - # Adds a pre-condition to run at the start of every example in the current group. - def add_pre_condition(&block : TestMetaMethod) : Nil - @@stack.current.add_pre_condition(block) + # Retrieves the root group. + private def root + @stack.first end - # Adds a post-condition to run at the end of every example in the current group. - def add_post_condition(&block : TestMetaMethod) : Nil - @@stack.current.add_post_condition(block) + # Retrieves the current group, which is at the top of the stack. + # This is the group that new examples should be added to. + private def current + @stack.last end - def add_default_stub(*args) : Nil - @@stack.current.add_default_stub(*args) + # Copy all hooks from config to root group. + private def apply_config_hooks(group) + @config.before_suite_hooks.each { |hook| group.before_all(hook) } + @config.after_suite_hooks.reverse_each { |hook| group.after_all(hook) } + @config.before_each_hooks.each { |hook| group.before_each(hook) } + @config.after_each_hooks.reverse_each { |hook| group.after_each(hook) } + @config.around_each_hooks.each { |hook| group.around_each(hook) } end - # Builds the entire spec and returns it as a test suite. - # This should be called only once after the entire spec has been defined. - protected def build(filter : ExampleFilter) : TestSuite - group = @@stack.root.build - TestSuite.new(group, filter) + # Copy `before_all` and `after_all` hooks to a group. + private def apply_top_level_config_hooks(group) + # Hooks are dupped so that they retain their original state (call once). + @config.before_all_hooks.each { |hook| group.before_all(hook.dup) } + @config.after_all_hooks.reverse_each { |hook| group.after_all(hook.dup) } end end end diff --git a/src/spectator/spec_builder/example_builder.cr b/src/spectator/spec_builder/example_builder.cr deleted file mode 100644 index 378c024..0000000 --- a/src/spectator/spec_builder/example_builder.cr +++ /dev/null @@ -1,19 +0,0 @@ -require "../../spectator_test" -require "../test_values" -require "../test_wrapper" - -module Spectator::SpecBuilder - abstract class ExampleBuilder - alias FactoryMethod = TestValues -> ::SpectatorTest - - def initialize(@description : String?, @source : Source, @builder : FactoryMethod, @runner : TestMethod) - end - - abstract def build(group) : ExampleComponent - - private def build_test_wrapper(group) - test = @builder.call(group.context.values) - TestWrapper.new(@description, @source, test, @runner) - end - end -end diff --git a/src/spectator/spec_builder/example_group_builder.cr b/src/spectator/spec_builder/example_group_builder.cr deleted file mode 100644 index 950f99d..0000000 --- a/src/spectator/spec_builder/example_group_builder.cr +++ /dev/null @@ -1,74 +0,0 @@ -require "../test_context" -require "./example_builder" - -module Spectator::SpecBuilder - abstract class ExampleGroupBuilder - alias Child = NestedExampleGroupBuilder | ExampleBuilder - - private getter children = Deque(Child).new - - @before_each_hooks = Deque(TestMetaMethod).new - @after_each_hooks = Deque(TestMetaMethod).new - @before_all_hooks = Deque(->).new - @after_all_hooks = Deque(->).new - @around_each_hooks = Deque(::SpectatorTest, Proc(Nil) ->).new - @pre_conditions = Deque(TestMetaMethod).new - @post_conditions = Deque(TestMetaMethod).new - @default_stubs = {} of String => Deque(Mocks::MethodStub) - - def add_child(child : Child) - @children << child - end - - def add_before_each_hook(hook : TestMetaMethod) - @before_each_hooks << hook - end - - def add_after_each_hook(hook : TestMetaMethod) - @after_each_hooks << hook - end - - def add_before_all_hook(hook : ->) - @before_all_hooks << hook - end - - def add_after_all_hook(hook : ->) - @after_all_hooks << hook - end - - def add_around_each_hook(hook : ::SpectatorTest, Proc(Nil) ->) - @around_each_hooks << hook - end - - def add_pre_condition(hook : TestMetaMethod) - @pre_conditions << hook - end - - def add_post_condition(hook : TestMetaMethod) - @post_conditions << hook - end - - def add_default_stub(type : T.class, stub : Mocks::MethodStub) forall T - key = type.name - @default_stubs[key] = Deque(Mocks::MethodStub).new unless @default_stubs.has_key?(key) - @default_stubs[key].unshift(stub) - end - - private def build_hooks - ExampleHooks.new( - @before_all_hooks.to_a, - @before_each_hooks.to_a, - @after_all_hooks.to_a, - @after_each_hooks.to_a, - @around_each_hooks.to_a - ) - end - - private def build_conditions - ExampleConditions.new( - @pre_conditions.to_a, - @post_conditions.to_a - ) - end - end -end diff --git a/src/spectator/spec_builder/example_group_stack.cr b/src/spectator/spec_builder/example_group_stack.cr deleted file mode 100644 index 50385bb..0000000 --- a/src/spectator/spec_builder/example_group_stack.cr +++ /dev/null @@ -1,28 +0,0 @@ -require "./root_example_group_builder" -require "./nested_example_group_builder" - -module Spectator::SpecBuilder - struct ExampleGroupStack - getter root - - def initialize - @root = RootExampleGroupBuilder.new - @stack = Deque(ExampleGroupBuilder).new(1, @root) - end - - def current - @stack.last - end - - def push(group : NestedExampleGroupBuilder) - current.add_child(group) - @stack.push(group) - end - - def pop - raise "Attempted to pop root example group from stack" if current == root - - @stack.pop - end - end -end diff --git a/src/spectator/spec_builder/nested_example_group_builder.cr b/src/spectator/spec_builder/nested_example_group_builder.cr deleted file mode 100644 index 6d7bb77..0000000 --- a/src/spectator/spec_builder/nested_example_group_builder.cr +++ /dev/null @@ -1,18 +0,0 @@ -require "../test_context" -require "./example_group_builder" - -module Spectator::SpecBuilder - class NestedExampleGroupBuilder < ExampleGroupBuilder - def initialize(@description : String | Symbol, @source : Source) - end - - def build(parent_group) - context = TestContext.new(parent_group.context, build_hooks, build_conditions, parent_group.context.values, @default_stubs) - NestedExampleGroup.new(@description, @source, parent_group, context).tap do |group| - group.children = children.map do |child| - child.build(group).as(ExampleComponent) - end - end - end - end -end diff --git a/src/spectator/spec_builder/pending_example_builder.cr b/src/spectator/spec_builder/pending_example_builder.cr deleted file mode 100644 index 731b9ab..0000000 --- a/src/spectator/spec_builder/pending_example_builder.cr +++ /dev/null @@ -1,10 +0,0 @@ -require "./example_builder" - -module Spectator::SpecBuilder - class PendingExampleBuilder < ExampleBuilder - def build(group) : ExampleComponent - wrapper = build_test_wrapper(group) - PendingExample.new(group, wrapper).as(ExampleComponent) - end - end -end diff --git a/src/spectator/spec_builder/root_example_group_builder.cr b/src/spectator/spec_builder/root_example_group_builder.cr deleted file mode 100644 index 3dbbcf7..0000000 --- a/src/spectator/spec_builder/root_example_group_builder.cr +++ /dev/null @@ -1,15 +0,0 @@ -require "../test_values" -require "./example_group_builder" - -module Spectator::SpecBuilder - class RootExampleGroupBuilder < ExampleGroupBuilder - def build - context = TestContext.new(nil, build_hooks, build_conditions, TestValues.empty, {} of String => Deque(Mocks::MethodStub)) - RootExampleGroup.new(context).tap do |group| - group.children = children.map do |child| - child.build(group).as(ExampleComponent) - end - end - end - end -end diff --git a/src/spectator/spec_builder/runnable_example_builder.cr b/src/spectator/spec_builder/runnable_example_builder.cr deleted file mode 100644 index 8c22a15..0000000 --- a/src/spectator/spec_builder/runnable_example_builder.cr +++ /dev/null @@ -1,10 +0,0 @@ -require "./example_builder" - -module Spectator::SpecBuilder - class RunnableExampleBuilder < ExampleBuilder - def build(group) : ExampleComponent - wrapper = build_test_wrapper(group) - RunnableExample.new(group, wrapper).as(ExampleComponent) - end - end -end diff --git a/src/spectator/spec_builder/sample_example_group_builder.cr b/src/spectator/spec_builder/sample_example_group_builder.cr deleted file mode 100644 index 955ac7a..0000000 --- a/src/spectator/spec_builder/sample_example_group_builder.cr +++ /dev/null @@ -1,30 +0,0 @@ -require "./nested_example_group_builder" - -module Spectator::SpecBuilder - class SampleExampleGroupBuilder(T) < NestedExampleGroupBuilder - def initialize(description : String | Symbol, source : Source, @id : Symbol, @label : String, @collection_builder : TestValues -> Array(T)) - super(description, source) - end - - def build(parent_group) - values = parent_group.context.values - collection = @collection_builder.call(values) - context = TestContext.new(parent_group.context, build_hooks, build_conditions, values, @default_stubs) - NestedExampleGroup.new(@description, @source, parent_group, context).tap do |group| - group.children = collection.map do |element| - build_sub_group(group, element).as(ExampleComponent) - end - end - end - - private def build_sub_group(parent_group, element) - values = parent_group.context.values.add(@id, @description.to_s, element) - context = TestContext.new(parent_group.context, ExampleHooks.empty, ExampleConditions.empty, values, {} of String => Deque(Mocks::MethodStub)) - NestedExampleGroup.new("#{@label} = #{element.inspect}", @source, parent_group, context).tap do |group| - group.children = children.map do |child| - child.build(group).as(ExampleComponent) - end - end - end - end -end diff --git a/src/spectator/successful_result.cr b/src/spectator/successful_result.cr deleted file mode 100644 index bd3f0ed..0000000 --- a/src/spectator/successful_result.cr +++ /dev/null @@ -1,23 +0,0 @@ -require "./finished_result" - -module Spectator - # Outcome that indicates running an example was successful. - class SuccessfulResult < FinishedResult - # Calls the `success` method on *interface*. - def call(interface) - interface.success - end - - # Calls the `success` method on *interface* - # and passes the yielded value. - def call(interface) - value = yield self - interface.success(value) - end - - # One-word descriptor of the result. - def to_s(io) - io << "success" - end - end -end diff --git a/src/spectator/tag_node_filter.cr b/src/spectator/tag_node_filter.cr new file mode 100644 index 0000000..d360712 --- /dev/null +++ b/src/spectator/tag_node_filter.cr @@ -0,0 +1,16 @@ +require "./node_filter" + +module Spectator + # Filter that matches nodes with a given tag. + class TagNodeFilter < NodeFilter + # Creates the filter. + # The *tag* indicates which tag the node must have in its metadata. + def initialize(@tag : String, @value : String? = nil) + end + + # Checks whether the node satisfies the filter. + def includes?(node) : Bool + node.metadata.any? { |key, value| key.to_s == @tag && (!@value || value == @value) } + end + end +end diff --git a/src/spectator/test_block.cr b/src/spectator/test_block.cr deleted file mode 100644 index a6af675..0000000 --- a/src/spectator/test_block.cr +++ /dev/null @@ -1,48 +0,0 @@ -require "./test_expression" - -module Spectator - # Captures an block from a test and its label. - struct TestBlock(ReturnType) < TestExpression(ReturnType) - # Calls the block and retrieves the value. - def value : ReturnType - @proc.call - end - - # Creates the block expression with a custom label. - # Typically the label is the code in the block/proc. - def initialize(@proc : -> ReturnType, label : String) - super(label) - end - - def self.create(proc : -> T, label : String) forall T - {% if T.id == "ReturnType".id %} - wrapper = ->{ proc.call; nil } - TestBlock(Nil).new(wrapper, label) - {% else %} - TestBlock(T).new(proc, label) - {% end %} - end - - # Creates the block expression with a generic label. - # This is used for the "should" syntax and when the label doesn't matter. - def initialize(@proc : -> ReturnType) - super("") - end - - def self.create(proc : -> T) forall T - {% if T.id == "ReturnType".id %} - wrapper = ->{ proc.call; nil } - TestBlock(Nil).new(wrapper) - {% else %} - TestBlock(T).new(proc) - {% end %} - end - - # Reports complete information about the expression. - def inspect(io) - io << label - io << " -> " - io << value - end - end -end diff --git a/src/spectator/test_context.cr b/src/spectator/test_context.cr index bf2c612..a68c5b9 100644 --- a/src/spectator/test_context.cr +++ b/src/spectator/test_context.cr @@ -1,82 +1,40 @@ -require "./example_hooks" -require "./test_values" +require "./context" +require "./dsl" +require "./lazy_wrapper" +require "./metadata" -module Spectator - class TestContext - getter! parent +# Class used as the base for all specs using the DSL. +# It adds methods and macros necessary to use the DSL from the spec. +# This type is intentionally outside the `Spectator` module. +# The reason for this is to prevent name collision when using the DSL to define a spec. +class SpectatorTestContext < SpectatorContext + include ::Spectator::DSL::Concise + include ::Spectator::DSL::Examples + include ::Spectator::DSL::Expectations + include ::Spectator::DSL::Groups + include ::Spectator::DSL::Hooks + include ::Spectator::DSL::Matchers + include ::Spectator::DSL::Memoize + include ::Spectator::DSL::Mocks - getter values + @subject = ::Spectator::LazyWrapper.new - getter stubs : Hash(String, Deque(Mocks::MethodStub)) + # Initial implicit subject for tests. + # This method should be overridden by example groups when an object is described. + private def _spectator_implicit_subject + nil + end - def initialize(@parent : TestContext?, - @hooks : ExampleHooks, - @conditions : ExampleConditions, - @values : TestValues, - @stubs : Hash(String, Deque(Mocks::MethodStub))) - @before_all_hooks_run = false - @after_all_hooks_run = false - end + # Initial subject for tests. + # Returns the implicit subject. + # This method should be overridden when an explicit subject is defined by the DSL. + private def subject + @subject.get { _spectator_implicit_subject } + end - def run_before_hooks(example : Example) - run_before_all_hooks - run_before_each_hooks(example) - end - - protected def run_before_all_hooks - return if @before_all_hooks_run - - @parent.try &.run_before_all_hooks - @hooks.run_before_all - ensure - @before_all_hooks_run = true - end - - protected def run_before_each_hooks(example : Example) - @parent.try &.run_before_each_hooks(example) - @hooks.run_before_each(example.test_wrapper, example) - end - - def run_after_hooks(example : Example) - run_after_each_hooks(example) - run_after_all_hooks(example.group) - end - - protected def run_after_all_hooks(group : ExampleGroup, *, ignore_unfinished = false) - return if @after_all_hooks_run - return unless ignore_unfinished || group.finished? - - @hooks.run_after_all - @parent.try do |parent_context| - parent_group = group.as(NestedExampleGroup).parent - parent_context.run_after_all_hooks(parent_group, ignore_unfinished: ignore_unfinished) - end - ensure - @after_all_hooks_run = true - end - - protected def run_after_each_hooks(example : Example) - @hooks.run_after_each(example.test_wrapper, example) - @parent.try &.run_after_each_hooks(example) - end - - def wrap_around_each_hooks(test, &block : ->) - wrapper = @hooks.wrap_around_each(test, block) - if (parent = @parent) - parent.wrap_around_each_hooks(test, &wrapper) - else - wrapper - end - end - - def run_pre_conditions(example) - @parent.try &.run_pre_conditions(example) - @conditions.run_pre_conditions(example.test_wrapper, example) - end - - def run_post_conditions(example) - @conditions.run_post_conditions(example.test_wrapper, example) - @parent.try &.run_post_conditions(example) - end + # Initial metadata for tests. + # This method should be overridden by example groups and examples. + private def self.metadata + ::Spectator::Metadata.new end end diff --git a/src/spectator/test_expression.cr b/src/spectator/test_expression.cr deleted file mode 100644 index d5e3cdd..0000000 --- a/src/spectator/test_expression.cr +++ /dev/null @@ -1,25 +0,0 @@ -module Spectator - # Base type for capturing an expression from a test. - abstract struct TestExpression(T) - # User-friendly string displayed for the actual expression being tested. - # For instance, in the expectation: - # ``` - # expect(foo).to eq(bar) - # ``` - # This property will be "foo". - # It will be the literal string "foo", - # and not the actual value of the foo. - getter label : String - - # Creates the common base of the expression. - def initialize(@label) - end - - abstract def value : T - - # String representation of the expression. - def to_s(io) - io << label - end - end -end diff --git a/src/spectator/test_suite.cr b/src/spectator/test_suite.cr deleted file mode 100644 index 2106193..0000000 --- a/src/spectator/test_suite.cr +++ /dev/null @@ -1,25 +0,0 @@ -module Spectator - # Encapsulates the tests to run and additional properties about them. - # Use `#each` to enumerate over all tests in the suite. - class TestSuite - include Enumerable(Example) - - # Creates the test suite. - # The example *group* provided will be run. - # The *filter* identifies which examples to run from the *group*. - def initialize(@group : ExampleGroup, @filter : ExampleFilter) - end - - # Yields each example in the test suite. - def each : Nil - iterator.each do |example| - yield example if @filter.includes?(example) - end - end - - # Creates an iterator for the example group. - private def iterator - ExampleIterator.new(@group) - end - end -end diff --git a/src/spectator/test_value.cr b/src/spectator/test_value.cr deleted file mode 100644 index b621562..0000000 --- a/src/spectator/test_value.cr +++ /dev/null @@ -1,29 +0,0 @@ -require "./test_expression" - -module Spectator - # Captures a value from a test and its label. - struct TestValue(T) < TestExpression(T) - # Actual value. - getter value : T - - # Creates the expression value with a custom label. - def initialize(@value : T, label : String) - super(label) - end - - # Creates the expression with a stringified value. - # This is used for the "should" syntax and when the label doesn't matter. - def initialize(@value : T) - super(@value.to_s) - end - - # Reports complete information about the expression. - def inspect(io) - io << label - io << '=' - io << @value - end - end - - alias LabeledValue = TestValue(String) -end diff --git a/src/spectator/test_values.cr b/src/spectator/test_values.cr deleted file mode 100644 index 8503ca8..0000000 --- a/src/spectator/test_values.cr +++ /dev/null @@ -1,64 +0,0 @@ -require "./typed_value_wrapper" -require "./value_wrapper" - -module Spectator - # Collection of test values supplied to examples. - # Each value is labeled by a symbol that the example knows. - # The values also come with a name that can be given to humans. - struct TestValues - # Creates an empty set of sample values. - def self.empty - new({} of Symbol => Entry) - end - - # Creates a collection of sample values. - protected def initialize(@values = {} of Symbol => Entry) - end - - # Adds a new value by duplicating the current set and adding to it. - # The new sample values with the additional value is returned. - # The original set of sample values is not modified. - def add(id : Symbol, name : String, value) : TestValues - wrapper = TypedValueWrapper.new(value) - TestValues.new(@values.merge({ - id => Entry.new(name, wrapper), - })) - end - - # Retrieves the wrapper for a value. - # The symbol for the value is used for retrieval. - def get_wrapper(id : Symbol) - @values[id].wrapper - end - - # Retrieves a value. - # The symbol for the value is used for retrieval. - # The value's type must be provided so that the wrapper can be cast. - def get_value(id : Symbol, value_type : T.class) : T forall T - get_wrapper(id).as(TypedValueWrapper(T)).value - end - - # Iterates over all values and yields them. - def each - @values.each_value do |entry| - yield entry - end - end - - # Represents a single value in the set. - private struct Entry - # Human-friendly name for the value. - getter name : String - - # Wrapper for the value. - getter wrapper : ValueWrapper - - # Creates a new value entry. - def initialize(@name, @wrapper) - end - end - - # This must be after `Entry` is defined. - include Enumerable(Entry) - end -end diff --git a/src/spectator/test_wrapper.cr b/src/spectator/test_wrapper.cr deleted file mode 100644 index 79d5265..0000000 --- a/src/spectator/test_wrapper.cr +++ /dev/null @@ -1,42 +0,0 @@ -require "../spectator_test" -require "./source" - -module Spectator - alias TestMethod = ::SpectatorTest -> - - # Stores information about a end-user test. - # Used to instantiate tests and run them. - struct TestWrapper - # Description the user provided for the test. - def description - @description || @source.to_s - end - - # Location of the test in source code. - getter source - - # Creates a wrapper for the test. - def initialize(@description : String?, @source : Source, @test : ::SpectatorTest, @runner : TestMethod) - end - - def description? - !@description.nil? - end - - def run - call(@runner) - end - - def call(method : TestMethod) : Nil - method.call(@test) - end - - def call(method, *args) : Nil - method.call(@test, *args) - end - - def around_hook(context : TestContext) - context.wrap_around_each_hooks(@test) { run } - end - end -end diff --git a/src/spectator/typed_value_wrapper.cr b/src/spectator/typed_value_wrapper.cr deleted file mode 100644 index 8131d82..0000000 --- a/src/spectator/typed_value_wrapper.cr +++ /dev/null @@ -1,17 +0,0 @@ -require "./value_wrapper" - -module Spectator - # Implementation of a value wrapper for a specific type. - # Instances of this class should be created to wrap values. - # Then the wrapper should be stored as a `ValueWrapper` - # so that the type is deferred to runtime. - # This trick allows the DSL to store values without explicitly knowing their type. - class TypedValueWrapper(T) < ValueWrapper - # Wrapped value. - getter value : T - - # Creates a new wrapper for a value. - def initialize(@value : T) - end - end -end diff --git a/src/spectator/value.cr b/src/spectator/value.cr new file mode 100644 index 0000000..dd81de7 --- /dev/null +++ b/src/spectator/value.cr @@ -0,0 +1,29 @@ +require "./expression" +require "./label" + +module Spectator + # Represents a value from a test. + # This is typically captured by an `expect` macro. + # It consists of a label and the value of the expression. + # The label should be a string recognizable by the user, + # or nil if one isn't available. + class Value(T) < Expression(T) + # Raw value of the expression. + getter value : T + + # Creates the value. + # Expects the *value* of the expression and a *label* describing it. + # The *label* is usually the Crystal code evaluating to the *value*. + def initialize(@value : T, label : Label) + super(label) + end + + # Creates the value. + # Expects the *value* of the expression. + # It can be nil if it isn't available. + # A label is generated by calling `#inspect` on the *value*. + def initialize(@value : T) + super(@value.inspect) + end + end +end diff --git a/src/spectator/value_wrapper.cr b/src/spectator/value_wrapper.cr deleted file mode 100644 index 170951d..0000000 --- a/src/spectator/value_wrapper.cr +++ /dev/null @@ -1,7 +0,0 @@ -module Spectator - # Base class for proxying test values to examples. - # This abstraction is required for inferring types. - # The DSL makes heavy use of this to defer types. - abstract class ValueWrapper - end -end diff --git a/src/spectator/wrapper.cr b/src/spectator/wrapper.cr new file mode 100644 index 0000000..21bd0ec --- /dev/null +++ b/src/spectator/wrapper.cr @@ -0,0 +1,63 @@ +module Spectator + # Typeless wrapper for a value. + # Stores any value or reference type. + # However, the type must be known when retrieving the value. + # + # This type is expected to be used like so: + # ``` + # wrapper = Wrapper.new("wrapped") + # value = wrapper.get(String) + # ``` + struct Wrapper + @value : TypelessValue + + # Creates a wrapper for the specified value. + def initialize(value) + @value = Value.new(value) + end + + # Retrieves the previously wrapped value. + # The *type* of the wrapped value must match otherwise an error will be raised. + def get(type : T.class) : T forall T + value = @value.as(Value(T)) + value.get + end + + # Retrieves the previously wrapped value. + # Alternate form of `#get` that accepts a block. + # The block must return the same type as the wrapped value, otherwise an error will be raised. + # This method gets around the issue where the value might be a type (i.e. `Int32.class`). + # The block will never be executed, it is only used for type information. + # + # ``` + # wrapper = Wrapper.new(Int32) + # # type = wrapper.get(Int32.class) # Does not work! + # type = wrapper.get { Int32 } # Returns Int32 + # ``` + def get(& : -> T) : T forall T + value = @value.as(Value(T)) + value.get + end + + # Base type that generic types inherit from. + # This provides a common base type, + # since Crystal doesn't support storing an `Object` (yet). + # Instances of this type must be downcast to `Value` to be useful. + private abstract class TypelessValue + end + + # Generic value wrapper. + # Simply holds a value and inherits from `TypelessValue`, + # so that all types of this class can be stored as one. + private class Value(T) < TypelessValue + # Creates the wrapper with the specified value. + def initialize(@value : T) + end + + # Retrieves the wrapped value. + def get : T + @value + end + end + end +end diff --git a/src/spectator_test.cr b/src/spectator_test.cr deleted file mode 100644 index a0a202d..0000000 --- a/src/spectator_test.cr +++ /dev/null @@ -1,25 +0,0 @@ -require "./spectator/dsl" - -# Root-level class that all tests inherit from and are contained in. -# This class is intentionally outside of the scope of Spectator, -# so that the namespace isn't leaked into tests unexpectedly. -class SpectatorTest - include ::Spectator::DSL - - def _spectator_implicit_subject - nil - end - - def subject - _spectator_implicit_subject - end - - def initialize(@spectator_test_values : ::Spectator::TestValues) - end - - # Prevent leaking internal values since their types may differ. - # Workaround for: https://gitlab.com/arctic-fox/spectator/-/issues/53 - def inspect(io) - io << self.class - end -end