I really want a standalone autodetection tool. As I surf, it will:

• live in the Windows system tray
• parse pages for urls pointing to syndication formats like RSS and Atom
• verify those feeds exist and collect their metadata
• write a log file of the detection and verification info, in OPML
• display the number of new discoveries when hovering over the system tray icon
• push the file to a server, periodically and optionally.

By being a separate application from the RSS newsreader, the autodetective will be:

• Smaller, consuming fewer system resources than a newsreader
• Focused on the craft of detection, becoming smarter about finding things on the pages I read
• Independent of a newsreader, so I can have more than one newsreader (including browser-based ones) without having every page I read parsed for each tool.
• Diverse, detecting tidbits in my emails, chats, IRC sessions, etc.

If we wanted to get fatter about the client, it could spider to discover deeper (crawl this site) or discover wider (crawl the blogrolls you see). Less relevance than pages you've actually seen, but more context - especially as you revisit favorite blogs and services.

I'd also like the detective to discover more kinds of things and make sense of them:

• Contact information (emails, phone numbers, postal addresses)
• Physical locations (postal addresses, city names, geocoding)
• Calendar events (dates, times, durations, descriptions)
• Rich media (sound, video, flash files)  

so I can review and bring them into other software.

There should be programming specs, so they know how to find the detective's journals, and check if they've been updated with fresh discoveries. I didn't include a "new headlines" balloon or ticker in the detective's features. The detective isn't a newsreader.

The detective should listen to your newsreaders too. Your newsreaders should also push the locations of your subscription lists ("you can find what Phil is reading at http://...") to the detective. This way the detective can optimize its reports by checking your subscriptions, then excluding them from discoveries. 

Let me browse and edit my discoveries in a human-usable form. I may want to delete items from my history before sharing them with a newsreader.  

I have an identity that lives across multiple computers and cell phones. I'll have detectives on each. My detectives should be able to confer and harmonize their discoveries. I may have multiple users on any computer, so detection prefs and journals should be aware of user profiles.

What's the business case?

• Strategy: Environmental Awareness. What's the cost of missing that a trusted feed has moved? That a key customer/competitor/regulator has a new feed? What if we made our collective surfing of the Internet into a competitive analysis tool, each person contributing their view of the world? With detectives on everyone's desk, we're less likely to be surprised, more likely to catch new opportunities, and be smarter as a group than our competitors.
• IT: Enterprise System Integration with Newsreaders. We're creating feeds of all sorts of information, including RSS of our SAP transactions. Many of these feeds will be customized for a specific context ("here's the RSS for orders Mary should approve.") The detective does away with error-prone cutting and pasting, automating the process of "I want to follow up on this". These feeds will drive attention to workflow and process. Some of the feeds will trigger people to write about specific items in team and project weblogs, improving communication.

I'll pay $20 retail for this.

Assuming you have an intranet blog server and either a server based news aggregator or desktop newsreaders, what would you pay for a 100 user site license?

Do you want one?

A commercial service using the ForPhotos Palm program. PDA to blog. 

[a klog apart]

Community Services for Enterprise Blognets

While your firewall protects you from intrusion, it also cripples the community software that keeps the blogosphere hopping. Here's are some of the services you might want to bring inside to help your blognets grow and prosper. The list grows, changes, and is not complete.  

I've grouped these services, arbitrarily, into three categories: Discovery, Reading, and Writing. Discovery services help you find stuff and navigate, and understand blognets and the blogosphere as a whole. Reading services help you keep up with relevant information. Writing helps you author and publish. Basic blogging service is extra.

In your workplace:

1. Which 3 are mandatory for a blognet pilot?
2. What risks do you assume if you don't provide these services?
3. Which services might you be better off operating in support of public employee and customer weblogs, even though they are the open blogosphere's services?
4. What policy and IT operations issues do these services raise?

Service	Description
Discovery
Intranet search	Covering the intranet and DMZ, your private search engine must update its index frequently. Best is if they re-index within a few minutes of fan update server being pinged. Engines which work well in public, because they use hypertext links to establish relevance, may not work as well in the intranet, where there are fewer links or other cues. For example, the Google appliance.
Location tagging and search service	Find blogs physically near me; find posts related to a location or system. For example, Geourl.org.
Referral logs	Who's sending traffic to me? It's sometimes useful to understand your readership. Other times you discover people with similar interests.
Weblog neighborhood	Who is like me? Who writes about things like me? Who else is cited like me? For example, Technorati link cosmos.
Topic service	Find posts related to this one within my weblog, across the intranet, and perhaps across a collection of partner blognets. See K-collector and Easy News Topics.
Realspace	Generate live meetings using information from blogspace. For example, Meetup or Evite.
Random walk	Manufacture serendipity. Sample the intranet, get a bigger picture. See also wanderlust.
Directory	So you have an employee directory, maybe even a yellow pages for services and departments. How about extending the yellow pages to people, by topic, updated automatically? For example, see blogarama, Eatonweb, Oblix.
Advertising	Text ads for internal announcements. Think of it as the new bulletin board.
Cemetery	A directory of abandoned weblogs, because of personnel actions, lack of interest, or because their focus or relationship is completed. See Fucked Weblog.
Product or object watch	Analyze weblogs for well-understood references, store and analyze the results, and notify subscribers. For example, seeing what books people mention in their weblogs. Or people. Or competitor products.
Peopleroll and social network	I'm sharing some of my friends, and friends of friends. See FOAF, Friendster, Ryze.
Reading
RSS portals	server side directories of RSS feeds, aggregation and browser presentation of those feeds
Updates	What's new? A central service that writing tools notify when a blog is updated. Sometimes called a "ping service". Like weblogs.com and blo.gs.
Blogroll & WebRing services	May be linked to enterprise directory services, the better to provide automatic maintenance of blogrolls that match the formal org chart. Of more value, giving users the ability to create their own blogrolls. This reveals informal and temporary social networks. Blogrolling.com is an example.
Blog distribution gateway	Distributes blog posts by email, SMS or other channels.
Buzz watcher	What's hot on the intranet? What's hot in my circle? Services that answer this include Blogpulse, DayPop News Burst, Popdex, and Blogdex
News Readers and Aggregators	Aggregators collect a user's selection of RSS feeds, keeping them current, formating them for reading, and making them available for users to cross-post. News readers do the same thing, except from a user's desktop. Server side aggregators have the effect of concentrating traffic (they pick an RSS feed only once, instead of each user picking it up) so publishers don't experience "slashdot effects". They also hide the level of attention from publishers, useful if the publisher is a competitor or industry insider. Syndic8 is an example.
Re-aggregation service	These services combine content from multiple sources into a more focused feed. This can be fully automated or humans may approve contributions to a feed. Moreover is an example.
Machine translation	Do you span countries? Machine translations of posts and RSS feeds helps people get the gist of what their colleagues write. Systrans is an example.
Writing
Posting Gateways	Use these services to write to your weblog using non-browser devices or software. Post from voicemail, your phone's SMS/MMS, email, calendar, or IM.
Comment Service	Manages posted comments like the blog server manages weblog posts.
Conversation Threading	Tracks the flow of conversation across weblogs using methods like trackback and link analysis.
Render Services	These convert blog posts to RSS, and to document formats like PowerPoint .ppt, Flash .swf, Adobe Acrobat .pdf, or Microsoft Word .doc.
Template Farm, Widget Library	Stores styles, templates, graphics and other ways to customize the look and feel of your blog.
Weblog Medic	Checks your blog for dead links, broken images, speed, accessibility, valid RSS and html, language encoding, etc. For example, BlogCheckup.
Blog Fodder	Actively provoke blogging by suggesting themes or topics. For example, blogfodder and The Friday Five.

I'll be updating this page for a while.

[a klog apart]

Checking in with Nico Lumma of noch'n blogg and a fellow survivor of the Blogtalk conference. A few items:

He and his wife have their first baby. Congratulations!

Thinking about what blogging's official theme song should be

• recognize that blogging ring tone
• stop a blogger on the street
• to play at awards ceremonies

Nominees:

• Abba, just because they capture the spirit so well
• Johnny Cash, whose death was widely blogged
• No fleetwood mac
• The Greatful Dead, but hard to blog to

The new rage in Germany: Karaoke Blogging. The site, Karaokeblogging.de, has been climbing the blogg.de and blogcensus charts in the last month. Building on the richness and immediacy of audioblogging, karaoke blogging is a higher form of social software (unless you don't like karaoke). The first time I heard a fully karaoked blog, I was blown away by the sonified experience. Update: Now in English at Karaokeblogging.com.

I'm a fan of 1-click karaoke blogging, great usability. But they should not file for a patent. The backlash might stifle karaokeblogging in its infancy.  

A few technical concerns:

1. I am hoping for ENT 1.0 support soon, the better to navigate both the songbook libraries and karaokebloggers' mp3 recordings.
2. They must add Atom support to exploit integration and syndication of the Karaoke midi xml format with the blog item data model.
3. Move from their poorly formed RSS feed to a valid RSS 2.0 feed, to support the KAI xml namespace. How are we supposed to adjust our newsreaders to support the feeds if they aren't well formed? Also, RSS enclosures would help stream new karaoke, midi and mp3 files in the background, so I can wake up to a newsreader full of the latest from my favorite performers.
4. There is some debate over whether you should permit more than song per blog post. I support Nico's idea that more karaoke is better.
5. Not sure if it works in Opera.

The business applications are obvious. So I'll be covering them in my BloggerCon session next weekend. See you there.

- phil

Should there be blog lifecycle capability built into the Atom API?

For example,

• Create a new weblog
• Freeze, retire, suspend a weblog
• Merge weblogs x, y, and z
• Delete a weblog
• Copy or move an entire weblog from this server to that server
• Create a new weblog by extracting some content from weblog x

This would enable programmatic control over a weblog by an authorized system or person.

For instance, you may want your HR system to automatically generate a weblog for each new employee, and freeze it when the employee leaves the company. Or to manage multiple blog servers (made by different vendors) using one admin tool. [PhilWolff]

This would allow the inclusion of ads within RSS (not ad boxes, but simple text with the label advertisement).  I really don't want to see ads in my RSS, but it is inevitable.  For qualified content, like what I am working on with the Weblog Network, it is a must.

Won't you have greater matching precision by looking at all the posts in a given feed?

[a klog apart]

Steve Kirks reimagines the future of blogging. It is a beautiful, elegant vision. I want the drugs he's taking.

Create a different kind of aggregator, one that's a browser first and RSS reader second.

Read the rest...

About four weeks' ago I wrote about Scaling Echo: P2P and Cached Feeds. I drew a few conclusions:

1. The syndicated blogosphere will reach 300 million feeds in 3 years
2. Feed payloads will grow 100 to 10,000 times
3. Each reader may consume 1000 feeds
4. Syndication Growth = Denial Of Service
5. Two architectures will support this scale:
   • Peer-To-Peer (P2P)
   • Caching by intermediaries (communal aggregators)

Jevon wrote about scaling new post pinging.

We need to have aggregators that send their subscriptions into a cloud and get pings back that have gone in to that cloud at any other point. It could be called a "peer to peer subscription and message delivery system", or "distributed redundant notification network", or whatever. But as soon as we can build a message network layer, Microcontent will really be ready for the next level: integration into our active lives.

Phrases like "community ping relay servers", "supernode", and "ping cloud" make my day. 

[a klog apart klogs]

I've been following two things very closely for many years: content syndication and labor markets. Last week RSSJobs was announced, bringing the two together. Here's my interview with Steve Rose who built RSSJobs.

What inspired or provoked you to create RSSJobs?

It was a combination of things. First was the frustration with my own job hunt. Like many IT professionals, I was unemployed for 6 months. When I did finally find a job, it was for half my previous pay, and in a environment I never would have considered otherwise. Even after starting that job, I was still job hunting. Every morning I was greeted with emails from Monster, and Dice, and several others with the results of my saved search agents. They were pretty useless. Monster only allowed 5 agents, and the emails only had up to 5 jobs per agent. I had to go to Monster's web site to see all the results. Then there was Dice. It gave me up to 50 jobs for each agent every day. Most of them were the same as the previous day's results! They were supposed to be just the new ones. I was spending all my morning time before work weeding through these, and I rarely had time to check any other sites that I didn't get emails from. Sites that didn't get updated every day went un-checked for weeks or months. Who knows how many potential jobs I missed out on because I didn't have time to check all the sites I wanted to check for updates.

Second was exposure to RSS. I started reading all my web based news using NetNewsWire earlier this year, and was amazed at how much easier it was to keep up. So I stared playing with the RSS format, creating some feeds for my own personal use, and I thought this would be useful for checking a local University's job board. I wrote a quick java servlet to parse the new job listing and return the results as RSS. It was so cool! Not long after that, I added Dice and Monster to the mix.

At this point, it was all just for my own use. About 2 weeks later, I went on a job interview, and when asked what kind of personal projects I had, I mentioned this and described the benefits of RSS. One of the developers interviewing me knew about RSS, and thought it was very good idea. He said I should market it. So I came up with a simple business plan, adapted my servlets to a subscription-based model, and built a web site around it.

How would you describe what RSSJobs does?

RSSJobs is simply a search agent for other job boards. It takes search parameters from the user, searches the job boards they want, and returns the results to them in RSS.

Who is it for?

Ideally, RSSJobs is for anyone looking for a job on the internet. It is well suited to individuals who have jobs, but want to keep their eyes open to other positions, and don't have the time to do an exhaustive search every day.

That being said, the average person out there doesn't know about RSS yet, and has a hard time understanding the benefits. It's a paradigm shift for most people, making adoption of RSS more difficult. Web browsers are comfortable, and people don't want to give them up, despite their limitations.

So at this point, I don't expect most job hunters out there to "get" the benefits of using RSSJobs, so I am not targeting them just yet. Right now I am focusing on those who are already using RSS. As RSS use becomes more widespread, the target audience will expand.

When did it go live?

The official live date was August 1, 2003. The site has been up for a few weeks, but only myself and a few friends knew about it.

What's your day job? What's your technical background?

I am a Software Engineer. As a Software Engineer, I have done a little bit of everything. My strongest language is Java, but I also work in C/C++, as well as various 4GL type languages. I've done application, database, web, and multimedia development, sometimes all on the same project.

What programming tools did you use to construct RSSJobs? What platform are you running the apps on?

It was developed using Java 1.4.1, and currently hosted on Mac OS X Server 10.2.

What version(s) of RSS do you produce?

RSS 2.0

What do you think of the Echo project? Will you be supporting the new syndication formats?

I don't know much about the Echo project, but I plan to closely follow the market for RSS content. If other formats gain popularity, I will consider supporting them as well.

Most of the job boards bar "reverse engineering" and other screen scraping, concerned over theft of data by rivals and disintermediation. How does your design work around or through these concerns?

I have considered this, and I don't expect there to be an issue. The site clearly states that the user is searching other job sites. The job listings from the various boards are accessed on demand, and nothing is cached by RSSJobs. There is no attempt to mask the origin of the content. If the user wants more information about the job, they are sent to the job board, where they can apply for the job if they like. Users should still register and upload their resumes to the job boards being searched for maximum efficiency.

I liken what RSSJobs does to a personal assistant or agent who does the research requested by a client, and presents the results. For example, say my friend doesn't have internet access, but wants to use Monster.com in his job search. He asks me to search for jobs for him. Is there anything wrong with me typing in his keywords, downloading the results, and putting a summary of the listings in an Excel spreadsheet on a floppy disk for my friend to look through? It seems perfectly reasonable to me. RSSJobs does essentially the same thing.

Many employers use HR information systems that output job listings in an HR-XML format for bulk uploading to Monster and most of the big job boards. What kind of information is lost between employer and candidate?

I have no idea.

What's on your wishlist for news reader features?

I would like to see an RSS Reader that could manage the items from an RSS feed as individual items. A user could archive specific items for viewing later after it is no longer included in the feed. Adding locally-stored comments to an item would be a nice feature too.

Is there anything employers could do to make your job easier when searching jobs.Acme.com?

Yes, when they post jobs, keep the content simple. No embedded HTML tags, or other things that RSSJobs has to filter to keep the XML valid.

Where do you think the other bottlenecks are in getting work to workers?

I think the biggest problem is getting the word out about available jobs. There are so many different ways jobs get announced, between Job Boards, classifieds, and company web sites, it is hard to keep track of them all. RSSJobs is trying to help with that.

Where do you see RSSJobs going?

For now, RSSJobs is just a part-time endeavor. If it helps people out, and provides enough revenue to cover the hosting costs, I will be happy. It will expand slowly, adding new features and more search sites on an ongoing basis. Ideally, I'd like to grow it large enough to become a full time job, and maybe even provide a few jobs as well. But this is not going to be another .com flame-out, trying to become too big too fast. I've been part of that already. If the demand for RSSjobs is there, it will grow to meet that demand. If not, no-one is going to loose money over it.

What kind of feedback have you been getting from new users? What have you been learning from the RSSJobs experience?

Surprisingly, I have received very little direct feedback about it. What I have received has been positive, even excited, with a few requests for features I have already considered for the future. But the loudest statement has also been the quietest one. People are using the site! The site is still in its early stages, and I don't want more volume than I can handle, so I haven't done much to promote it yet. The little bit I have done has drawn more traffic than I could have expected, and people are actually using the site as it was intended. that says everything.

What have I learned? I'm not sure I have learned anything yet. It is all happening so fast, and things have gone remarkably well, almost too well. It's when things go wrong, particularly very wrong, when you learn the most. I'm sure that will come. Hopefully sooner rather than later.

###

Brad Wilson's I Want Something New thread.

I want new blog software.

Now, there's nothing wrong with the crop of blog software out there, but it all pretty much works the same. I'm not even sure what I want different... I just want something different. What I want, really, is something so radically different that it's hard to even call it "blog software".

My contribution to the thread...

1. Blog anything.
• On request, blog more richly structured things (like sports scores, recipes, SAP transaction approvals). [xml packages, schema for structure, css for formatting, xul for edit layouts]
2. Syndicate anything.
3. Make it easy to define new kinds of things. And share them by blogging them.
4. Have your blogging tools discover and learn to use new kinds of things. On the fly.