Wednesday, March 1, 2006

How come everyone has scalability problems, but no one really talks about them. People have no idea how to build a fully scalable infrastructure. Google, and Akamai seem to do this very well in general, but for the rest of us we aren't so gifted.

As things, especially in market and financial data scale up, we have a hard time keeping up with it. Look at the breakout of options data, and new regulations which require us to track more data, and shove it all down the pipes we have. We need some smarter burstable technology to handle the mass updates. Someone at work was talking about pulsing data. Cool ideas, minimize the updates per stock to 3 per second versus updating all of the changes to a ticker.

No comments: