Scripted Tests

[Cameron McCormack]

Jeff Schiller:
> I'm trying to get a handle on this, because I've wondered about how to
> structure a really robust SVG test suite too.
> 
> About JavaScripted tests:  I know your *svgpoint-transform-1.svg *example at
> the bottom of http://www.svgtest.org/structure/ was for illustration
> purposes, but will real tests have to be much more careful?  Or what is the
> scope of a given test?

Yes, keep in mind that it's just a strawman example, really.

> For instance, should *svgpoint-transform-1.svg*:
>    a) check if document exists
>    b) check if document.documentElement exists
>    c) check if document.documentElement.createSVGMatrix exists
>    d) ensure that createSVGMatrix doesn't throw an exception (for instance,
> it may exist in the DOM but has not been implemented yet, I have come across
> this in certain DOM method calls in Mozilla)
>    e) ensure that createSVGMatrix returns the proper DOM interface type
>    f) ensure that the translate() method exists on SVGMatrix and doesn't
> throw an exception
>    g) ensure that document.documentElement.createSVGPoint exists, doesn't
> throw an exception and returns the correct DOM type
>    h) ensure that SVGPoint.matrixTransform exists and doesn't throw an
> exception
> 
> In all the failure cases above, you would also have to document a different
> failure cause in the fail() call.

I'm not sure if a different failure cause needs to be explicitly put in
the test file.  If any of these aren't implemented some sort of
exception will be thrown, and I think for scripted tests it should be
that any uncaught exception is a failure of the test.

As Ian said, in any given test there are going to be things tested that
aren't really the focus of the test.  I think as much as possible these
dependencies should be tested separately in their own test, but it won't
always be possible.  To test your 'e', for example, you can't do it from
script without using that object in other ways--there's no 'instanceof'
operator you can use.  So you will end up testing that it returned a
reasonable object by using the object's methods.

But I think it is reasonable not to test 'e' explicitly in this case.
That the object implements the proper DOM interface, especially in
ECMAScript, really is just that its properties and methods do the right
thing.

> The other option is to put a simple try-catch around the whole test and any
> failure scenario simply spits out an "Unknown Error" but each of the above
> would have to be checked by other test cases.  This means you will have a
> VERY LARGE number of tests that you will need to write.

Is there really a need to test the existence of a 'translate' function
on an SVGMatrix object and also to test that it returns an appropriate
value?  Maybe, maybe not.  If these existence tests are necessary, then
they can be easily automatically generated from the IDL.

> Another couple questions related to *svgpoint-transform-1.svg*:  Should you
> use different data types (i.e. an integer and a float) in the translate call
> and in the Point?

 From the point of view of function calling in ECMAScript, integers and
floats are the same thing, just numbers.  But your point could be valid
for, say, strings and numbers.  This is really a test of the script
engine, though, and not really of SVG functionality.  Batik uses Rhino,
which has well defined rules about how ECMAScript types get converted
into Java types for the real methods calls.  In the end, though, the
right type is passed to the method.

But this could be different for other implementations.

> Should one of the point coordinates go negative as a
> result of the transform to test that negative coordinates are handled
> correctly?  How much do you want to exercise the functionality to ensure
> that it has been implemented correctly?

Yeah, I'm not sure at that level how far it should go.  Simply testing
negative numbers is probably overkill, since (taking a reasonable guess
at the implementation) would most likely be just testing the arithmetic
instructions of the CPU rather than anything else.


-- 
 Cameron McCormack			ICQ: 26955922
 cam (at) mcc.id.au			MSN: cam (at) mcc.id.au
 http://mcc.id.au/			JBR: heycam (at) jabber.org