[Catalyst] writing a test suite for my app

[Matt S Trout]

Perrin Harkins wrote:
> On Fri, 2006-05-19 at 16:50 +0100, Matt S Trout wrote:
>> I'm afraid you're dead wrong there. TWMC uses Catalyst::Test, which as clearly 
>> documented, simulates by *default* but if a CATALYST_SERVER environment 
>> variable is present goes and does a full live test using the copy of the 
>> application running at the URL in that environment variable.
> 
> Okay, but the T::W::M::Catalyst docs clearly say that it doesn't hit the
> server, so I guess they need an update.

Fair enough; if you give me the rt.cpan ticket number I'll happily chase Leon.

>> Except that that's a complete pain for local tests, since it's difficult to 
>> find a URL you can rely on for the application without knowing your local 
>> network configuration
> 
> I do know my local network configuration :)

Then you also know how to set an environment variable :)

>> this can be seen clearly by the fact that 
>> Test::WWW::Mechanize has to be force installed on every development server we 
>> have since its won tests rely on http://localhost/ without checking to see if 
>> $ENV{http_proxy} is set.
> 
> Test in CPAN modules meant to be run by end users on all different OSes
> and networks are kind of a different beast from tests on your
> organization's own application.  When testing your in-house development,
> you can make certain assumptions, and have your tests start up a server
> or check to see if one is running.

Yep, but since the catalyst testing framework doesn't know whether you're 
planning to deploy under mod_perl2, lighttpd+fastcgi, or whatever else, I 
think defaulting to not trying to use a live environment is eminently sensible.

> I've had several problems caused by web server configuration issues when
> I skipped going through the server in my tests, especially since I often
> use separate server modules for things like auth and compression.  I
> just don't trust simulating the web server environment anymore.

I absolutely agree that it's worth testing a live staging environment before 
you sign off on a QA process, but during development having the tests run 
faster and have minimal moving parts to break is often vastly preferable.

-- 
      Matt S Trout       Offering custom development, consultancy and support
   Technical Director    contracts for Catalyst, DBIx::Class and BAST. Contact
Shadowcat Systems Ltd.  mst (at) shadowcatsystems.co.uk for more information

+ Help us build a better perl ORM: http://dbix-class.shadowcatsystems.co.uk/ +