bunny-test


This is a simple unit-testing framework. It isn't the most powerful thing in the world, but it is (relatively) cross platform. It has been tested on PLT-Scheme and LispMe. This may give (other) beginners a look at how (not) to program simple things in scheme.

Using Bunny-Unit

It's pretty simple. You define a test with the test procedure. It takes one argument, and that is a quoted expression which returns a boolean. For example:

 (test '#t) ;; Test passes... Notice the test predicate is quoted 
 (test '#f) ;; Test fails 
 (test '(or #t #t #f)) ;; Test passes 
 (test '(and #t #t #f)) ;; Test fails 
 (test '(my-complex-procedure-that-returns-true arg1 arg2 arg3)) ;; Test passes 
 (test '(equal? (some-string-procedure) "my string thing")) ;; Test passes 

To actually run the tests, you pass each test to the test reporter like so:

 (test-report                   ;; Launch reporter for tests 
   (test '(equal? 'foo 'foo))   ; passes   
   (test '(my-foo arg1 arg2))   ; passes 
   (test '(equal? 'foo 'bar)))  ; fails 

This outputs:

..
Failed: (equal? 'foo 'bar)

For every passed test a dot is outputted. For every failed test it spits out the code that actually failed.

Test Suites

Sometimes its handy to package up tests in a test suite. You can package individual tests inside of a suite, or you can nest suites, like so:

 (test-report 
   (test-suite 'My-Suite-Name   ; Name could be quoted 
     (test '(equal? 'foo 'foo)) 
     (test '(catch-field-mice?)) 
     (test-suite 'Bop-Suite     ; You can nest suites if you like 
       (test '(run-in-forest?)) 
       (test '(is-field-mouse? thing))))) 

Why Quoted Expressions?

Because the expressions are quoted, it allows the test-report procedure to let you know exactly which test failed. It isn't enough to know that a test failed, you should at least know which test it was that did fail.

Note, that because of the way tests are executed (through eval) you could easily feed in the expression (test (some-predicate? (some arg) (some-complex-foo))) and it will run the test for you, but if it fails, you would only see #f as failure information. (This is because the some-predicate? procedure (and its args) are evaluated before the test is executed).

The Future?

I want to give each suite the ability to execute setup and teardown code. Setup should probably allow one to introduce new variable bindings. Both should follow the usual convention of executing on every test in a particular suite.

I also want to work on the reporting, so that the the number of passed and failed tests is shown. Also, it might be useful to show the location of the Failed test. (e.g., Test number 4 in suite "foo" failed with code bar.)

The code

 ; Bunny-Unit 
 (define (test code) 
   (list 'bunny-test 
         code 
         'bunny-test:un-tested)) 
  
 (define (bunny-test? t) 
   (and (list? t) 
        (eq? (car t) 'bunny-test))) 
  
 (define (get-code t) 
   (if (bunny-test? t) 
       (cadr t))) 
  
 (define (get-test-result t) 
   (if (bunny-test? t) 
       (begin 
         (if (not (test-execed? t)) 
             (exec-test t)) 
         (caddr t)))) 
  
 (define (test-execed? t) 
   (not (equal? 'bunny-test:un-tested 
                (caddr t)))) 
  
 (define (exec-test t) 
   (set-car! (cddr t) 
             (eval (get-code t)))) 
  
 (define (test-failed? t) 
   (if (bunny-test? t) 
       (not (get-test-result t)))) 
  
 (define (test-passed? t) 
   (and (bunny-test? t) 
        (get-test-result t))) 
  
 (define (test-suite name . tests) 
   (list 'bunny-suite 
         name 
         tests)) 
  
 (define (bunny-suite? b) 
   (and (list? b) 
        (eq? 'bunny-suite (car b)))) 
  
 (define (get-suite-name b) 
   (if (bunny-suite? b) 
       (cadr b))) 
  
 (define (get-suite-tests b) 
   (if (bunny-suite? b) 
       (caddr b))) 
  
 (define (display-suite s) 
   (newline) 
   (display "Suite: ") 
   (display (get-suite-name s)) 
   (for-each (lambda (t) (test-report t)) 
            (get-suite-tests s))) 
  
 (define (display-passed-test t) 
   (display ".")) 
  
 (define (display-failed-test t) 
   (newline) 
   (display "Failed: ") 
   (display (get-code t)) 
   (newline)) 
  
 (define (test-report item . depth) 
   (cond 
     ((bunny-suite? item) 
      (display-suite item)) 
     ((and (bunny-test? item) 
           (test-failed? item)) 
      (display-failed-test item)) 
     ((and (bunny-test? item) 
           (test-passed? item)) 
      (display-passed-test item))   
     (else 
      (error "No test or suite passed to report!")))) 
  
 (define (test-tests) 
   (test-suite 'Test-Tests 
     (test '(bunny-test? (test '#t))) 
     (test '(not(bunny-test? #f))) 
     (test '(equal? (get-code (test 'some-code)) 'some-code)) 
     (test '(get-test-result (test #t))) 
     (test '(not(get-test-result (test #f)))) 
     (let ((t (test '#t))) 
       (test (not (test-execed? t))) 
       (test (get-test-result t)) 
       (test (test-execed? t))))) 
  
 (define (suite-tests) 
   (test-suite 'Suite-Tests 
     (test '(bunny-suite? (test-suite 'dummy (test #t) (test #f)))) 
     (test '(equal? (get-suite-name (test-suite 'dummy (test #t))) 'dummy)) 
     (test '(bunny-test? (car (get-suite-tests (test-suite 'dummy (test #t)))))) 
     (test '(bunny-suite? (cadr (get-suite-tests (test-suite 'dummy (test #t) (test-suite 'inner-dummy)))))))) 
  
 (define (all-tests) 
   (test-suite 'All-Tests 
     (test-tests) 
     (suite-tests))) 

Riastradh

Why not use SRFI 9's record types for test cases & suite objects and macros for test? Test could just duplicate its sub-form to construct a thunk and a quoted representation of the code. It would eliminate the necessity of quotation or eval, which are both rather silly in this instance.


Jonathan-Arkell

Actually, the initial implementation did use a macro for test, I chose the quoted expression for compatibility sake. I may go back to the macro method instead.

As for SRFI 9, I don't believe it is available under LispMe, so I didn't use it.


TonySidaway

Running this under LispMe, I found that I had to load bunny-unit after all symbols to be used in the tests had been defined. This is because exec-test is a closure that evaluates a test in its own environment. After a few tries, I've come up with the following solution:

  1. Rename the function test-report to test-report+
  2. Change the reference to the procedure test-report to test-report+ in display-suite
  3. test-report now becomes the following macro, and remove the old definition of exec-test, and replace it with #f as shown:
 (define exec-test #f) 
  
 (macro (test-report args) 
   (let ((t (cadr args))) 
     `(begin 
       (set! exec-test 
         (lambda (t) 
           (set-car! (cddr t) 
             (eval (get-code t))))) 
       (test-report+ ,t)))) 

However, this depends on LispMe macro syntax.

Using a macro here causes the exec-test closure to be defined in the calling environment, so that it's inner eval has access to all functions defined at the time of the compilation of the test-report macro.


LispMe specific

This little fragment of LispMe specific code is to help facilitate you the test writer in writing tests. The idea is that you may be testing with the REPL, and want to turn your most-recent expression into a test. Hist->test takes the topmost expression off of the history stack, and outputs a test to the output field.

Hist->test+equal? is similar, but it will build an equal? test and prompt you for the expected output. It builds a test in the form of (test (equal? ...your-test... ...your-expected...)) where your-test is the top of the history stack, and your-expected is what you enter into the input-prompt.

 (define (hist->test) 
   `(test ',(car *hist*))) 
  
 (define (hist->test+equal?) 
   `(test '(equal? ,(car *hist*) 
     ,(input (string-append 
          "(equal?  " 
         (object->string (car *hist*)) 
         " ... ) "))))) 

TonySidaway

Running this on my palm pilot under LispMe 3.21, hist->test-equal? seemed to fail, inserting hist->test+equal? in place of the item on the history in the output (presumably the *hist* variable had been updated in the meantime). I fixed it by binding "((expr (car *hist*))" in a let around the quasiquote and substituting expr for both occurrences of "(car *hist*)".


SRFI-64

It is a portable r5rs syntax-rules based basic test


category-code