I looked at LiveJournal's count of active weblogs today. Their servers report
547,157 blogs updated in the last 30 days. NITLE's census reports
331,550 active blogs at LiveJournal. Why does NITLE miss 40%?
First, I'm going to trust that LJ isn't in error or lying.
Second, NITLE's collection grows from a combination of:
- ping records (blogs updating a public ping server)
- suggestions (you can recommend a link)
- following links from one blog to another (web bots spidering)
- classifying pages as either a blog or not a blog
There may be errors or inefficiencies on any of these, but I'll assume for the moment that LJ blogs ping by default, the NITLE spiders are thorough, and the blog recognizer is accurate.
If both services are telling the truth, what gives?
Most of the LJ 40% unreported are active blogs that are private to public scrutiny. One of the attractions of LJ is the
ability to control who reads your blog. For each post you can choose to keep it private (blogging to yourself), to share it with a friends list and/or an individual, or to make it public. Of the 60% that NITLE found, any of those blogs may have some hidden posts. The 40% are just completely opting out of Google scrutiny.
So, roughly a third of LJ blogs are invisible to the public. By choice.
A technical point: this kind of control only works with dynamic blog services, services that produce each page as demanded, customized for each reader. Most, like Blogger and Moveable Type, produce static pages, their html written once for everyone. As a result, most of the blogging APIs (instructions for programmers) don't include access control options. This is an area for future work.
As intranet and extranet blogging takes off, look for similar behavior within firewalls and strong demand for access control.
