When the world's dominant software company runs into a problem with the webfeeds for its developer network, lots of hackers go apeshit and start cooking up solutions. Sam Ruby reports on FeedMesh, a grassroots effort that broke out at Foo Camp to start building a P2P webfeed content distribtion network. Not a bad kernel of folks/organizations to start out with: blo.gs, Feedster, Yahoo!, and Bloglines.

The group has decided to initially focus on update notification distribution. There's actually enough working knowledge out there to just go for it, and try to knock out the content distribution problem too, in my humble opnion.

I wonder what would happen if a couple of big publishers added a few nodes to Coral and started issuing redirects for those feeds into that cdn? Presuming aggregators actually correctly honored HTTP redirects, publishers could selectively join as needed, aggregators wouldn't have to change much, and everybody would win.

Sounds too good to be true.

Update: The commentary on Scoble's posts is a quite entertaining microcosm of many technical discussions. Everything from, "of course it's broken", to "it was stupid server implementation", to "aggregators are implemented wrong", to "no it's not broken", to "we saw and solved all those problems in the 90's", to "let economic forces take care of the problem". And that doesn't even include all of the other weblog commenatary out there that's hard to round up in one place.

New Media Hack is of the opinion that they're all right and all wrong. Except maybe those who start off with, "the solution is simple". It always is if you only consider your perspective.

After kvetching about not being able to directly refer to an MT Entry in a template tag, Brad Choate e-mailed to let me know that in fact there is a tag, <MTLink> , that does what I want.

Great, must have snuck into 3.11 when I wasn't paying attention.

What? It was in 2.64? Gack!!

I'll just shut up now.

Between Flickr's photo shows and Laszlo's photoblox, slideshows are becoming of the Web.

I recently did a new Movable Type 3.11 install for a project. Relatively painless if you've been through the process before, or are an experienced sysadmin. This version has improved comment management and dynamic page generation. I'm on record as being against using Php for MT dynamic pages, but it's sort of a neat trick how they do it. mod_rewrite and error pages are used to forward non-existent pages to Php. Then a Php library that reimplements the MT Perl API takes over.

As I said, I'm not a big fan, but I think this does raise an issue that's going to become more important in weblog tools and cmses in general. Allowing the smooth transition between statically and dynamically generated pages. There are all sorts of tradeoffs given the demands of popular vs unpopular sites, frequent vs infrequent authors, and sophisticated vs naive publishers. How do you design a gentle slope system that let's folks intelligently select parts of their site for dynamic or static generation?

Apropos of nothing, one thing about MT that really bugs me, is the inability to actually refer to one lone entry. Say you have one outstanding post that you'd like to refer to in a template. There literally is no way to do so using the MT template tags. Sounds like a useful plug-in opportunity to me.

I may have to eat my words about Findory. Greg Linden announced some new Findory features and growth at the company.

If I had copious spare time, I'd really kick the tires hard on Findory, Topix, and PubSub.

Perrin Harkins and Bill Hilf thoroughly documented how eToys was built, using Apache and mod_perl. This is a detailed description of how to build a multimillion page view site, with a lot of good insights.

The kicker is that the paper was written circa 2001, about 3 years ago. And in fact, being a retrospective, many of the ideas had been deployed a bit earlier. Of course, a lot of it is reiteration of stuff Philip Greenspun wrote about. In any event, this is now pretty much mature technology.

Via Ryan Tomayko.

I find that the majority of my post clipping, retaining posts for later perusal or research, is falling into Bloglines. It's close and convenient to the place where I do a lot of reading. Also, while I'm not taking advanage of it, Bloglines supports folders for your clippings. However, Bloglines clips only services posts in feeds I read.

del.icio.us is the tool of choice for plain old URLs on the web. This means I have two places where I stash Web info I care about. That's one place too many.

WIBNI Bloglines had a posting API for clips? I've concocted the following somewhat Rube Goldbergian solution, but I think it's relatively workable and efficient. I'll start using a tag clipz for URLs I want to stash. Then in Bloglines I'll subscribe to the clipz tag feed.From there it's one click to clip it. A potential benefit is that I actually have to review the tagged URL before clipping it.

The more interesting point is that I'm using tags to drive downstream processing. Of course it's me doing the processing, but the communication could later be with an autonomous process working on my behalf.

Neil Turner's "9 steps to a quicker MT3.1x installation" is about making a Movable Type installation serve pages faster, not how to get the installation process done in less time.

C'mon, tell me you didn't read it the same way.

In any event, I found the majority of the tips relatively obvious, e.g. excise lots of conditional parts of default templates, but there are one or two gems in there like trimming down .htaccess

Thought parkin': Someone needs to start a blog to cover deployment of folksonomies in general, and applications like social bookmarking and photo sharing in particular.

Yet another webfeed aggregator with AI inside: Patrick Roberts egoclip. No mention of how it learns to prioritize.

Interesting thought, will the adaptation be disconcerting due to variability? For example, I expect most people think the ordering of items in an aggregator has some chronological basis. If the aggregator orders on an opaque priority will uses be comfortable with that? Not harshing, just wondering.

Apropos of nothing, proact should be a verb so the above phrase would be proper English. Then again verbing weirds language.

I guess "proactive not reactive" will have to suffice.

Don't quite know what to make of it, but getting del.icio.us to regularly post into Movable Type, seems like a sweet hack. It's the marriage of one of lightest interfaces on the Web with one of the better Web authoring tools. MT is pretty lightweight, but del.icio.us already has about five different interfaces, three of which are tightly integrated with the browser (sidebar, right click, bookmarklet).

The combination may also help relieve load on del.icio.us since a bunch of functions folks have been asking for can be outsourced to their favorite local content management system, presuming it supports the MT API.

Going completely meta, someone could outsource the regular reading of del.icio.us RSS feeds and posting them into registered cmses.

Update: Gave Joshua his just due and put the second h in schachter.

Jeremy Zawodny has probably written the best capsule summary of what's currently making Flickr the bee's knees. Good commentary too.

It's not just you Jeremy, Flickr is now good stuff all the way around.

Photo sharing: next year's social software?

Link parkin': seoradio.com produces and archives interviews with folks in the search engine optimization industry, those folks dedicated to getting web sites high rankings on Google et. al.

Nobody crucial from the search engine industry, to my outsider eye, but worth keeping an eye on.

Memo to self: create a linkparkin del.icio.us tag

Recently, I started engaging in a bit of Firefox promotion. Adam Kalsey rejected the same offer arguing that Firefox is not ready for casual users. Subsequent commentary suggests that casual users aren't really the target of the campaign.

Frankly, I can see both sides of the argument although I think it's a good idea that the Mozilla/Firefox folks start getting their advocacy act together. Microsoft may be closing the current window of opportunity on IE's dominance, but another one may arise soon. Luck is opportunity combined with preparation.

Headshift, an English social technology consulting firm, teases some firm benefits of social tagging. Instead of, as many IA folks seem to do, latching onto how folksonomies (sp?) fail, they look at the glass as half full. Social tagging is one way to get some relatively reliable content classification out of a large number of people. Supported by a basic and uncontroversial official taxonomy, IA types actually have a hope of getting organizations to reveal their underlying classifications.

Paul Graham departs from his usual programming bailiwick to ponder, in a rather meadnering fashion, writing essays. The key is to go back to the roots of the definition of "essay", which I take as simply attempting to work through your personal hurdles.

Graham's advice is actually relevant for many creative endeavors in life besides writing. Pursue surprises. Work out loud, but be ready to revise your thinking. Start small to grow something big.

Work out loud. I sort of like that term.

Apropos of nothing, just recording the origination and meaning of the following quote:

"Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified bug-ridden slow implementation of half of Common Lisp."

You can almost say the same thing about Web applications and the ArsDigita Content System.

Michelle Levesque is rigorously investigating a number of Python based Web application frameworks. She (presumably) has a well defined app that's being implemented in all of the systems, and is using CGI as a control. Still a work in progress, but I'm enjoying the coverage of Quixote.

I have to concur that Quixote takes some work to wrap your head around. One key lesson I learned is that every module or object must have a _q_exports attribute. A second lesson is that PTL is way cool.

Adam Kalsey is a pretty serious Movable Type developer. MT has gone through some recent upgrades and Kalsey adroitly breaks down who new features are aimed at: developers and businesses. He's going to be moving his raft of plug-ins to the new system on an as needed basis, essentially choosing a conservative path. He's right though, that SixApart has stopped innovating.

Frankly, no one's really innovating on the authoring and publishing side of blogging. We've still got the same crappy forms on the Web side, and there really aren't even that many desktop blogging tools. And since the trackback, name a new feature on the published page side?

Cleaning up some old tabs. Why The Lucky Stiff ponders integrating del.icio.us and Bittorrent. Sounds like a good idea, although as a del.icio.us user monitoring the system's development, creeping featuritis is perilously close. There are users demanding all sorts of things, many of which would detract from from del.icio.us's Zen like functionality.

About a year ago, I did a lot of noodling about with Mason, a Perl based templating system that has a lot of neat features. Like JSP, Mason compiles pages into executable code, Perl, and then runs the code to handle responses. Perl code can be easily embedded into templates, templates have a sophisticated inheritance mechanism, you can actually get at internal request data structures, and you don't have to restart the server to see changes to the templates. The only downside is that installation is not for the feint of heart. I also found Mason to be somewhat finicky if used with mod_perl, occasional hard to track down crashes and all that.

Having moved to Python, I've kicked the tires on a few page templating systems. Suffice it to say I was left pining for Mason, although Quixote is quite nicely Pythonic.

Enter Michael Bayer's Myghty, a knockoff of Mason in Python. Right now I'm giving it a test drive, and it feels pretty nice. Combined with mod_python it looks like you can have Python, flexible templating, access to server and protocol internals, and high performance all in one.

Meandering about from Richard MacManus's strategizing about e-books, I chanced upon Erik Benson's discussion of why he gets more mileage out of LiveJournal, then broadcast publishing. SixApart's keynote for Blogtalk 2.0 touched upon this as well. There's a place, even big demand, for small scale communication tools. LiveJournal deliberately focuses on that realm and has built up a huge following.

Benson does pine for a more integrated toolbox, but I think focused tools, supporting narrow but universal interconnections, are the way to go. All the buckets Benson dumps content into, except for his paper notebook, can support webfeeds out and in. Add an integration toolbox, my mythical Emacs of aggregation, and the right tool for him could be built.

Webfeeds, UNIX pipes for Web 2.0?

Two months and counting until full papers are due for the International World Wide Web Conference in 2005, located in scenic Japan. The track breakdown is entertaining , with lots of overlap to my untrained eye. Makes picking the right track for submission tricky, and will probably lead to a certain detectable inconsistency in the program. But I sympathize with the organizers who have to wrangle a big event at the relatively big end of the computing conference range.

James Clarke, a CS grad student at Sheffield University, has released a first cut of a Python module for accessing the Flickr API.

With folks like Jon Udell pumping del.icio.us hard it was only a matter of time before the social gamers, a.k.a spammers, showed up.

Definitely straight out of the school of juvenile hacks, for grins I took a peek at a few user names that might be available, strictly for entertainment purposes. gwbush, georgebush, algore, johnkerry and billclinton seem available. Bill Gates was until I swiped it. I promise to use it for good. For a small fee ;-)

del.icio.us squatting, wave of the oughts!!

Mark Nottingham has been considering the usage of Python dictionaries as interfaces to computing services. First, obliquely, is the usage of dicts for navigating XML. Secondly, he's actually putting the Web in a dictionary. All right, so I'm a little late to the party. Sue me.

For the uninitiated, in most scripting languages it's quite easy to hijack the associative array/hash table syntax and hide other datatypes or procedures behind it. So for example it becomes easy to say something like web['www.cs.northwestern.edu'] to retrieve a Web page.

I did get a tad bit inspired and started putzing around with putting the del.icio.us API in a dictionary. No code as of yet, but the results look promising. Just to be a bit more of a tease, for another project, I combined dictionaries and Python's shelves for persistent storage. Putting it all together, I think there's a relatively elegant way to provide a Python module which makes accessing del.icio.us quite Pythonic, but also provides caching and persistence. The whole package would be very convenient for building friendly del.icio.us clients on top of.

Of course I think the Flickr API is the bee's knees. Just as a challenge though, what would a photo management application that exposed itself using the Atom API look like?

I got sucked into thinking about this in considering how RESTful the Flickr APIs really are, especially in light of point 4 in Paul Prescod's Common REST mistakes. Prescod recommends avoiding putting actions in URIs which the Flickr API seems to do in spades.

The key feature I'm pondering is how to retrieve metadata about a photo. Two separate URIs would solve the problem, at the expense of increased network traffic and increased API complexity. It would be nice if the same resource/URI could be queried for different representations. Yeah, I know clients are supposed to specify this in the HTTP headers, but really how many folks actually do. He Hao's REST Web Services : Best Practices article indicates the potential for a meta query word, but then tags its usage as "arguably RESTful". Given the nature of the Atom wonks, my guess is that the spec is pretty doggone RESTful from the start, so one wouldn't have to worry about this issue.

Thinking out loud, an Atom enabled photo gallery system would be a nice alternative application to weblogs. Easy enough for mere mortals to understand what it's supposed to do, practically useful, yet probably exercising all of the APIs power.

Here's my six word summary on Macej Ceglowski's manifesto:

Audioblogging is not of the web.

But listen for yourself. Brilliantly executed and I totally agree. I'm not as fanatic about videoblogging, as video streams have the potential to have embedded links, which I think is the major lose of audio posts. Still, I think an even smaller percentage of people will be technically and artistically talented enough to make videoblogging worthwhile. Then again, we were saying that about maintining Web sites roughly 5 years ago and a cohort of crafty hackers came up with the right set of tools to really make it work.

Let me say there is one upside to audioblogging, the same one that radio has. It's quite easy to tune out the speaker and get some hacking done. I've punched up exactly one of Dave Winer's audio posts, and by about a third of the way through it had become pleasant background noise.

Funny sidenote. When I first clicked the MP3 link, all I got was silence for whatever reason. Given that Ceglowski needed 4 minutes or so to get his point across, I thought it also a nice homage to John Cage. I'm not sure it wouldn't have been a more devastating critique although too subtle for most of the blogosphere.

Speaking of standing queries, somehow Asa Dotzler at the Mozilla Foundation, found me waxing poetic about the capabilities of Firefox. Really, all I do is ping blo.gs and Weblogs.com, and the latter is about to be booted as it slows down rebuilds.

Anyhoo I've been asked to add a badge or two, and I'm happy to oblige. Heck, the Moz Foundation even hosts the images for you. Like this:

The small one to your right will stick around for a while. Hey, if you use Firefox, and if you do I assume you have a Web site somewhere, why don't you pile on.

I've been kicking the tires on Gush, mostly for the IM. Geez, it's a gorgeous app. There's also a webfeed aggregator baked in. Apparently they have some deal with PubSub.com, and to help grease the skids a bit, they have a primer on how to take advantage of PubSub's features.

I haven't really jumped on the PubSub bandwagon, but this howto reminded my of a latent thought I had. At the end of my CMS class, I gave out a few technology predictions. I went out on the steel limb and predicted that canned standing queries would increasingly be accepted as a new first class data type on the Web. Taking a keyword query, turning it into a URI, and than having that URI generate an RSS feed is the first iteration. I don't quite know how to fit such dynamic resources into the REST model, but once there all sorts of goodness would seem to ensue.

Just thinking out loud.

10 reasons to love Flickr, from Marc Canter. +1, especially reason #1: the Flickr API. To see all the Flickr API hot action, check out the developer's mailing list. There's neato stuff coming down the pike for Flash, Python, Perl, ColdFusion, etc. etc.

Glassdog's Ponyboy rants on lack of imagination in webfeed readers. Nicely supports the previous post. Any regular reader of this fine site knows that we here at New Media Hack are already so there.

The commentary against is of the usual sort. "Only infopornorgrapher's want that." "It's harder than it looks." "Put up or shut up." All of these points are valid concerns, but don't invalidate the premise. Emacs isn't for everybody, and it was one hell of a thing to build, but it makes a certain class of hackers better, who subsequently make lots of good stuff. An overall win for society. Plus, some of the fundamental design principles of Emacs, e.g. damn good regular expressions all over, embedded scripting language, have withstood the test of time.

While the Ponyboy's writing blaming the problem on inhuman programmers is quite entertaining, the whole dustup argues for a major experiment. If there was an Emacs of aggrgegators, 1000 plugins/flowers could bloom. Then we'd really find out what people wanted.

I think I'm starting to hone in on the real question underlying all of my recent ponderings. Once upon a time, I jumped on Awasu's bandwagon because they had a semblance of extensibility, but it never felt quite right. What is and isn't worth being pluggable and extensible in an aggregator? And then how do you design the aggregator to support that?

Jeremy Zawodny's sparked a blogosphere burst by musing about how the big boys have left webfeed search to little fish like Feedster, and Technorati, not to mention the "newsey" types like PubSub, Findory, and Topix. John Battelle has a nice roundup of the overall discussion, including a pointer to Rich (Topix) Skrenta's spin that the blogosphere is USENET 1.7 for better or worse. Comments on many of these posts are really useful as well.

Folks are fishing around for why this space might be big. It's dependent on a value proposition that gets lots of people into using aggregators. At that point users are providing way more personal information than just 2.x search terms. Also, aggregation provides a mechanism to deliver unsolicited information in a more personalized context, but hopefully without as much spam. And one bonus is that users are in browse mode, so a commercial interruption is not as devastating as when it interrupts the task focused drive (according to Jakob Nielsen) of search mode.

But there needs to be a demonstrated mass of people using aggregators. Enter Bloglines, best of breed web based aggregator, with a search engine baked in. Adoption of Findory, Topix, and PubSub also points to healthy growth. These guys are where the feed search story is going to really be played out.

I'll just throw in three other words that could make this really juicy for the geeks at Google, Yahoo!, MSN, IBM, etc. Personalized focused crawlers.

Diego Doval's atomflow is more than just a Java based, queryable, persistent store for Atom. It's a nascent and promising attempt at applying the UNIX philosophy to syndication and aggregation. Write one excellent, but focused, tool and then rely on the surronding integration atmosphere to put it to use. In this example it would be UNIX style pipes to connect things together, and thus melt away monolithic blogging tools and aggregators.

Doval's partners in crime, Matt Webb and Ben Hammersley, make the case more eloquently than I can. Of course the overall surface ideas aren't that new, e.g. Kimbro Staken's Syncato covers much of the same ground. The difference is that atomflow seems like a microkernel to Syncato's monolithic kernel, to use an OS analogy. We'll see if it has legs.

Engaging in some lightly supported speculation, Jason Kottke imagines a browser kitted out and branded by Google. Basically, they'd be stepping somewhat into Netscape's (cement?) shoes, but at least they have a bit of an enterprise background to give them cred, and some other business to fall back on. And if Google stayed away from calling it a product, IT managers could secretly adopt it as an alternative standard. Besides, a little browser competition is definitely Not Evil (TM).

Kottke even hypothesizes a browser based application suite. Not one consisting of the typical desktop suspects, but a Web-centric set including, search, e-mail, blogging, and IM. This is a bit heavier than my vision of each user tricking out their browser with smaller extensions, but the two are not incompatible. Even better, major advancements could be made if Google showed how to integrate every aspect of a the browser UI with serious global scale computing.

Actually, instead of an official endorsement, working on Mozilla/Firefox hacks wouldn't be a bad way for a few Google engineers to spend their hacking time.

Earlier this month, a few information architecture folks got together and started discussing all this Flickr and del.icio.us tagging. In the process a cute little term, folksonomy, was coined. While it's really a noun, it seems to capture the spirit of what's going.

Folksonomizing would be trying too hard though.

I'm not sure the IA guys, quite get the point though. In these environments, there is no One True Taxonomy to bind them. Not only is it a 90/10 argument, it's The Web, so global coordination is out to start.

I wonder if there's anybody left in Scheme land ( put away your knife, I know there is) other than the scattered remnants of the Rice PLT group? Still, these guys do good stuff. Sometimes the hammer guy can get pretty ingenious turning the world into nails.

Shriram Krisnamurthi's team at Brown University has put together a toolbox for program monitoring called MzTake. Pitched as a debugging tool, they've devised some primitives for announcing program events, and a domain specific language, FrTime, for reading and reacting to those events. FrTime can alter program execution.

Forget debugging!! This is an open extensibility mechanism. Imagine your browser or aggregator instrumented with a set of events, including UI behavior, that can be externally monitored within a convenient programming environment. All sorts of gluing and mayhem would ensue. Infrastructure like AppleScript and COM sort of support this, but they skimp on the event driven end of things acknowledging the fact that help is needed from the applications to start with.

Seriously, what if every application was written with MzTake style applications in mind? It might not be that far fetched in a world where logging is growing more popular.

Hat tip to Lambda the Ultimate.

The previous post was number 700. Not only do we here at New Media Hack own the term on Google, we own "media hack" to boot. It may be a very small pond, but sometimes it's good to be the big fish.

Take that Wired.

And since there seems to be a slight uptick in readership around here (7 to maybe 9), here's my thought on comments for this site.

Weblog comments are USENET done infinitely worse. It's like taking every thread and distributing it to a different host. With no protocol for monitoring and threading them to boot!! So you can't even read 'em in a halfway decent client. I don't have time to be constantly visiting your site and/or sifting through your comment feed to track a conversation. Although I agree with Phil Ringnalda that mouthing "get yer own blog" isn't sufficient as a solution either. And don't get me started on trackbacks although I thought they were sort of neat when they first emerged. Nor do I really want to deal with pruning comment spam no matter how good Movable Type, Wordpress, or any other blog tool gets at dealing with it.

Needless to say, if you want to engage me in a conversation on material on this blog, just send me an e-mail.

What was that about "if you can't stand the heat?"

Dan Gillmor linked to GoSkokie. Hooray!!

This apparently set off those blog hatin' blokes at The Register. Boooo, trolls!! No excuse, but they caught the site at a particularly vulnerable moment, dog days of summer and all that. (Aren't all those folks across the pond supposed to be on holiday anyway?)

Also, Dave Ascher dropped private e-mail, mildly protesting my disappointment with his Dynamic Languages article. He rightly points out that to get CIOs and CTOs over the mental hump of adopting these languages (because of course all real hackers know the utility of languages like Python, Perl, Ruby, et. al.), new terminology and a new framing might do the trick. Maybe that could be a strategic initiative for these languages. Seeing how many Fortune X00 executives can be gotten to admit on record that dynamic languages are core to their business, and by how much. Bezos, Omidyar, Brin & Page, maybe get Sun on board, round up an IBM advocate, find someone at Apple. Then start working the big boys. For a while, in the Tcl community there was major effort to collect and catalog usages in the field, but this seems to have fallen by the wayside as these languages have become more popular in the open source community. Mainly, Ascher is rightly promoting a new era of advocacy.

My suggestion: just stay away from 4GL.

Okay, I'll admit that The Reg's headline was funny.