System design and planning for the future

How many of you have designed systems and explicitly made decisions based on predicted future needs?

Here's a good example. How many projects have you worked on where a Java Application Server was deemed necessary because "we might need to scale up and cluster for performance and robustness"? How many of those projects took on EJB's for the same reasons?

Ok. Now how many of you have actually seen those systems grow to actually utilize those type of decisions?

My point is, I've seen countless projects where expensive appservers (adds up quickly when you get one for dev, uat, prod..and possibly per seat licence for developers...and then there's support) are mandated as necessary for applications that don't really need more than a servlet engine and a database. Or how about in the early days of EJB's, using all those remote references when the application was always hosted on the same machine.

To be honest, the motives for these decisions (and planning) are honourable but in practice...the situations they were made to accomodate for, rarely materialize. I my experience most systems have a lifespan of 5yrs whereafter they are rewritten. Why not accept the fact that the system is most likely to be shelved after 5yrs and there isn't such a great urgency to plan/design the thing as if it were going to run for a lifetime? I can see the counter-argument where people will claim that with greater usage and higher traffic, the system must be able to grow to meet those needs. Well, with the abundance of testing tools and load testers, we should be able to ascertain whether the system we put into production today will handle that heavy load. If we have that knowledge today, why pay up front in design simplicity and server costs to plan for something that is unlikely to occur?

The Java blogsphere is endeared with these lightweight IoC containers such as Spring and Pico. From what I've seen in my brief experimentation, this paradigm of lightweight servers with pluggable services is the future. The large commercial appserver vendors better have something up their sleeve or open their eyes before they get decimated like dinosaurs during the ice age.

Speaking of pluggable, anyone aware of how to plug in a JMS provider to Spring?