Interested in how the Web's major players achieve scale and IT agility? I know a blog just for you.
The blogger's credo is always to give credit where credit is due. If, as a blogger, you happen upon someone else's blog that strikes you as particularly stimulating or that provides the seed for an item or two on your own blog, then the right thing to do is to name that blog prominently and offer a hyperlink for your readers to follow.
Which is exactly what I am doing here, citing a most fascinating blog that's called High Scalability.
High Scalability, as you might imagine, looks at the ins and outs of "building bigger, faster, more reliable Websites." Writes Todd Hoff who, except for the occasional invited guest, appears to be the blog's sole writer:
"This site tries to bring together all the lore, art, science, practice, and experience of building scalable Websites into one place so you can learn how to build your Website with confidence.
"When it becomes clear you must grow your Website or die, most people have no idea where to start. It's not a skill you learn in school or pick up from a magazine article on a plane flight home. No, building scalable systems is a body of knowledge slowly built up over time from hard-won experience and many failed battles."
And experiences, or what Hoff calls "real life architectures," are what his site offers in abundance. If gaining a better understanding of what goes on under the hood at the Web's biggest, most-visited, and most-active sites is your thing, then Hoff is your man. A software professional since 1985, when he graduated from the University of Oregon, Hoff knows his stuff, and he evidently loves sharing insights into how the Web's big boys do what they do.
Where else, for instance, can one read about how an outfit called blekko, for instance, whipped up from scratch a NoSQL datastore that serves as the heart of a full-blown Web search engine that now, three years later, comprises 1,500 servers and serves 3.5 million unique visitors per month?
How about a peek inside the infrastructure of Instagram, the mobile photo-sharing app recently acquired by Facebook for a cool $1 billion. Talk about large-scale: Instagram went from 30 million to 40 million users in just 10 days after the Facebook deal was disclosed. As with most of the super-mega-ultra sites that High Scalability delves into, Instagram's core philosophy boils down to staying lean, simple, and agile.
"Ideas are disposable," Hoff quotes from an Instagram presentation. "If one doesn't work, you quickly move on to another."
In looking into the technical guts of another high-volume site, Hoff finds a setup that handles 300,000 queries per second and dishes out well north of 100 million video streams per day -- all of them, as you've no doubt guessed, of the so-called "adult" variety.
As an outsider looking in, I find these case studies and Hoff's insights into the technical strategies that these largest of large-scale sites rely on all quite entertaining. These operations are working close to, if not exactly on, the bleeding edge of Web technology -- though it's probably safe to say that outfits like Amazon and Google are considerably further ahead, even if they don't talk much in public about what they're up to, infrastructure-wise. (In fact, Hoff does take a stab at describing Google's setup, based on various data he has found here and there.)
Most enterprise IT departments won't need to consider the kinds of extreme infrastructure that the High Scalability blog revels in, not yet, anyway. But it's good to know that someone this knowledgeable and enthusiastic is striving to keep an eye on the road ahead.
Tweet This
