The next big thing (no, really)
You can’t pick up a newspaper or business magazine these days without some talking head describing the “next big thing” that will revolutionize business, and perhaps life as we know it. The hype was “more or less” true about the Internet – it just took a bit longer than the breathless prose about it had predicted. The Internet has changed life as we know it – and is still changing it as we speak.

Sadly, there are far more examples of technologies that did not even come close to living up to their promise. It has created an unfortunate level of cynicism about any new technology. How do we know what is real and what is not?

As you might imagine, in our business, we get exposed to lots of new technologies – that of our clients, our partners, and with the groups that we interact with in the course of the research side of the business. No, I don’t consider myself an authority of the evaluation of technology – but I am perhaps just a bit more informed than the average pedestrian. And from this position I can tell you that there is one technology that is quietly reshaping the world of information technology: grid computing.

What? Grid computing isn’t new! What’s so revolutionary about linking together a bunch of computers on a network? Aren’t we already there with the Internet anyway?

Fair points. But let’s really get down to what we are talking about here. Grid computing is not merely linking a bunch of computers together – rather, it is creating a single environment where computers work cooperatively on a single problem. You order a book from your favorite online seller – the system breaks different pieces of the transaction up (debiting your credit card, updating your purchase history, placing an order at the distribution center) and processes them on different devices instead of handling each instruction sequentially.

Ok, fine…sounds efficient but what’s the big deal? Not much in the above example. But for those of us who build sophisticated models that make use of parallel processing, this is a HUGE deal. Supercomputers have always worked this way…the difference today is that we can build the near equivalent of a supercomputer with armies of inexpensive components, like PCs running the Linux operating system – precisely how Google is architected.

So many problems that we are beginning to see have a parallel feature to them. Agent-based models, Monte Carlo simulations, landscape search problems – all are highly suitable to a parallel processing environment. Companies have spent decades building elaborate IT systems that are housing terabytes of data – with grid computing and its parallel processing cousin we now have the means to tackle business problems of enormous scope – imagine a retailer developing a sales forecast based upon a demographic profile and shopping pattern of every one of its 3 million customers. Imagine an energy firm assessing its reserve position on every one of its 60,000 producing wells worldwide…

Grid computing is real. Believe it. Now, get back to [work, work, work, work…].
- posted by Industrial Science @ 8:08 AM