[Catalyst] FW: Clustering catalyst apps

[Peter Edwards]

Hi Gert, I think the key here is "average small company".

My gut feel is that two relatively cheap rented servers at different
datacentres using an (admittedly) crude DNS approach is enough to run most
small companies' web services reliably and cheaply.

I'm not sure if you need more. The sort of questions I'd ask about your
customer are:
How much money are they willing to spend per month?
Is it online retail or chat-based or some other service?
How many transactions per hour are they handling?
Is timeliness critical?
Do they handle a few big customers who need perfect service, or many smaller
customers where they can afford to lose a few due to downtime?

Assuming you do need more, there are a couple of aspects
1) Scalability
2) Reliability

Scalability.
The model I suggested lets you scale up to about 150 concurrent users. Most
small companies would be delighted to have that many :)

Reliability.
High availability doesn't necessarily mean that you have to have 100% of
your system's functionality on-line immediately. For example, if you still
have the web pages coming up, can see contact names and phone numbers, but
maybe it takes 30 minutes for the latest orders to reappear (via your
recovery process) then for many users that is going to be acceptable and not
considered an outage in service.
If you run a watchdog process to flip the DNS on failure and initiate a DR
process (or use something like perlbal that Aristotle suggested) you can
have a high *perceived* uptime approaching what your customer asked for.
Write the SLA in your support contract carefully and that might be enough.

To achieve a real 99.99% uptime with 100% functionality is going to cost you
a lot more... design time, testing, hardware, network, database licences,
monitoring, support staff.

Put it another way, are you selling them a "Rolls Royce" solution or a
diesel van? I know which most small businesses are going to go for. Of
course they'd love a Rolls Royce - as long as you pay for it - but all their
similar-sized competitors are driving diesel vans.

I'm not suggesting you skimp on remote monitoring, or the use of a TPM if
you really need it, just that a combination of KISS and customer expectation
management will save you money and trouble.

Regards, Peter 
www.dragonstaff.com

-----Original Message-----
From: catalyst-bounces at lists.rawmode.org
[mailto:catalyst-bounces at lists.rawmode.org] On Behalf Of Gert Burger
Sent: 08 May 2006 13:45
To: The elegant MVC web framework
Subject: Re: [Catalyst] FW: Clustering catalyst apps

Thanks for the reply, here are some of my comments on this:

Using round robin dns still means that if 50% of the servers are down,
50% of all queries will goto the broken machines. Which will piss of
half your customers.

I have looked at the High Availability systems that have been written
for linux and they provide doubles(Or more) of everything, from load
balancers to db servers. The issue I have with them are they require a
great deal of money in hardware to get running.

Anycase, back to my issue, How do websites like slashdot and amazon, all
which use perl, keep uptimes of close to 99.999% ?

And is it possible to get to that level with lots of crappy hardware?

Cheers

PS. Excuse me for meddling with the semi-impossible.

On Mon, 2006-05-08 at 13:30 +0100, Peter Edwards wrote:
> (I've put some more linebreaks in this time)
> 
> Set up the DNS for your application to map to multiple IP addresses, one
> each for however many web server machines you need. Run your perl apps on
> those.

> 
> Have a single database server machine with RAID mirrored disks. Have your
> perl apps connect to that.
> 
> Regularly backup your database across the net to a disaster recovery (DR)
> machine at a different physical location. With mysql you can do a hotcopy
> then rsync the files across. Set up the DR server so it can also be a web
> server.

> 
> Failures:
> 
> Disk - switch to mirror until you can replace the disk
> 
> Database host - switch your web apps to the DR server for database access;
> have an application strategy on what to do with delayed transactions that
> happened since the last database synchronisation [1]
> 
> Network/Datacentre - point DNS to DR server and use its web server (poor
> performance, but at least limited access is available)
> 
> Assuming you've got your servers in a data centre with triple connections
to
> the Internet backbone, this last scenario is very unlikely.
> 
> A lot depends on how many users, how critical up-time is, what the cost
> equation is between having an alternative site and hardware versus the
> opportunity cost of lost sales and damaged reputation. The above works
well
> for 10-150 concurrent users. For more you could consider using the
> clustering and failover features that come with some databases.
> 
> [1] For example, if you manage to recover the transaction log from the
main
> db server you can merge the records in later provided your app hasn't
> allocated overlapping unique ids to its record keys.
> 
> Regards, Peter
> 
> -----Original Message-----
> From: catalyst-bounces at lists.rawmode.org
> [mailto:catalyst-bounces at lists.rawmode.org] On Behalf Of Gert Burger
> Sent: 08 May 2006 12:41
> To: The elegant MVC web framework
> Subject: [Catalyst] Clustering catalyst apps
> 
> Hi
> 
> I was wondering a few days ago how one would create a cluster of
> catalyst webapps?
> 
> Some of my early thoughts including just having multiple machines
> running apache with a load balancer.
> 
> But you then still have a single point of failure, at the load balancer.
> 
> 
> Another problem is, if you use some sort of database to store your
> sessions etc then you have another point of failure.
> 
> Therefore, how can a average small company improve their
> (Catalyst)webapps reliability without breaking the budget?
> 
> Gert Burger
> 
> 
> _______________________________________________
> Catalyst mailing list
> Catalyst at lists.rawmode.org
> http://lists.rawmode.org/mailman/listinfo/catalyst
> 
> 
> 
> _______________________________________________
> Catalyst mailing list
> Catalyst at lists.rawmode.org
> http://lists.rawmode.org/mailman/listinfo/catalyst


_______________________________________________
Catalyst mailing list
Catalyst at lists.rawmode.org
http://lists.rawmode.org/mailman/listinfo/catalyst