Yahoo Pipes | // Internet Duct Tape

Yahoo Pipe: Sub-Reddit Feed Filter

Posted in Delicious, Reddit, Ruby on Rails, Technology, Yahoo Pipes by engtech on January 28, 2008

Popular social bookmarking site Reddit has announced a great new feature: users can create their own sub-reddit. What does this mean in English? Users and communities can create their own social bookmarking sites around specific topics: blogging, wordpress, specific programming languages, etc but still use their regular reddit account for submitting links and voting.

You can see a full list of all the new reddits here, sorted by popularity. Of particular interest to me is the new Reddit created for Ruby/Rails related posts.

Of course, it’d be nice to be able to subscribe to a filtered version of these links. I’ve created a modified version of Dave S‘s “reddit popular on delicious” Yahoo Pipe that works with Sub-reddits.

http://pipes.yahoo.com/engtech/subredditpopularondelicious

Click on the link
Enter the name of the sub-reddit you’re interested in
- ie: ruby, see full list of all the new reddits here
Enter the minimum number of saves on a delicious before a link is included in the feed
Enter keyword inclusion/exclusion filters if you want to limit what you get
- ie: include only rails-related posts or exclude all rails-related posts
Click Run
Click on the subscribe to RSS button

I’m using the Ruby sub-reddit as an example, but this is a great way to track links based around any topic there is a sub-reddit for. Even lolcats.

I’m looking forward to when this Reddit feature comes out of beta and it’s possible to create a few new sub-reddits like blogging, wordpress and lifehacks.

Tagged with: Reddit, rss, ruby, Ruby on Rails, rubyonrails, Yahoo Pipes

2 comments

The Attention Age: Accelerando, Software Agents, Filters and Gatekeepers

Posted in Book Reviews, Digital Culture, RSS Syndication, Software, Technology by engtech on October 17, 2007

Last night I finished reading Accelerando by Charles Stross. Like many of the books I read these days, I heard about it from another blogger. It feels like a spiritual sequel to Alvin Toffler’s Future Shock, John Brunner’s the Shockwave Rider and Warren Ellis’ Transmetropolitan. It is about information overload to the nth degree and too much change in too short of a time.

Accelerando is broken into 9 fragmented stories with decades passing in between them. This is too bad because it was the initial segment, only a few years in the future, that I found most interesting. Our protagonist is hooked up to a portable computing network of software agents that he uses to continually data mine and plug-in to a “river of news”. As he communicates with other people he spawns off parts of his “distributed brain” to research more information and get back to him.

The greatest inventions usually come from seeing the possible connection between two separate things (eg: peanut butter and chocolate). Like in the Shockwave Rider, our protagonist is successful because of his ability to gather and process information is so far beyond an average person’s. Being immersed in the information stream he sees the connections and trends that other’s can’t see.

These connections lead to so many successful ideas, that he can’t possibly execute on them himself – because the time it takes to implement them would take away from the information processing that is his true talent. He makes a career of giving away his ideas and surviving off of the reputation gain and support of his sponsors he’s made so successful. Very much like Doctorow’s concept of whuffie – reputation as currency.

The book progresses to talking about the post-human experience after digitization has reached the point that we can successfully digitally encode human personalities. Post-death society, heads in jars and living bodiless on the internet. There’s a really good bit on how the next major species will be intelligent corporations and artificial spam intelligence. But what really interested me was the initial chapters so close to the beginning 21^st century: how do we use technology to deal with information overload?

(You can get a copy of Accelerando for free online – which is very useful because the copy I borrowed from the library was missing the last page – now that’s frustrating)

It’s Getting Harder to Find Information

We’re in the middle of a great revolution where anyone can become a self-publisher. But that’s the crux of the problem, isn’t it? Anyone can become a self-publisher. The low barrier to entry makes the competition for attention fierce. At some level we’re all on par with the lowliest spammers, trying to compete for other people’s attention. There is so much new content being created all the time at the only way old content stays in the public record is if the Great Google God returns it in a search result.

This is only going to get worse because Google has created a new caste of blogging serfdom. People create content and splash Google ads on it with the hope of that it will do well in Google search results so they can get paid.

There’s many a “business model” that relies completely on Google-Google Search for traffic and Google AdSense for revenue. And there’s an even larger amount of so-called business models that rely almost completely on Google for traffic, even if the money comes in via other means.

I think you know what happens to the money when the traffic stops.

I use the term “business model” above loosely, because a model that is entirely dependent on an outside company, for either traffic or revenue or both, is not really sound. You’re not in charge and you have very little control, because if Google decides to change the rules, you’re out of luck. Based on that, I would argue that relying on Google is not a business at all.

I’d say you work for Google.

From the Teaching Sells e-book

Where are the Smart Filtering Agents?

One of the things I remember clearly about the idea of intelligent agents in the early 90s was how it was going to revolutionize how we consume information. Instead of having to *gasp* pick up a newspaper, autonomous software agents would search the net finding tidbits of information what we were interested in and adapting and learning from how we interact with the results. Sci-fi books like John Varley’s Steel Beach dealt with the relationships between humans and these evolving artificial intelligences.

Take a moment to glance at the Wikipedia page on software agents; it’s quite good.

The 90s hope for intelligent agents has congealed. RSS has gotten us part of the way; now we can pick voices out of the chaos that we allow to push information to us. We can subscribe to alerts on search subjects that interest us. But aside from custom recommendation engines like Netflix and Last.FM there isn’t really a bot out there for finding information for us.

The Future: RSS Filtering

I see the fledgling baby steps of software agents delivering news. There are several sites competing for being able to filter through a list of RSS feeds and recommend the best news items to you.

AideRSS
Feeds 2.0
Persai (by the guy who does UnCov)
Particls

There’s also the “build your own” filtering agent approach.

And let’s not forget the ability to monitor search terms.

One of the more enlightened concepts I’ve come across is FaveBot that wants to bring you the custom information you want about your favorite actors, authors and musicians.

Is the Answer Better Gatekeepers?

Is having an intelligent software agent the right approach or is it better to let humans do the filtering? The past year has seen an incredible rising in using crowdsourcing to decide what is the best information available. This is how digg, reddit, stumbleupon and the delicious popular page find interesting information by using the wisdom of mobs. Unfortunately when the user-base grows too large it becomes watered down to only common denominators.

The other approach is to find human editors to act as your gatekeeper. I’m not talking about hiring your man in Mumbai, but rather niche news sites like Slashdot, BoingBoing and Fark, and to a greater extent using the network of blogs you enjoy to act as your information gate keepers.

The last.FM music service is an amazing tool for finding new music to listen to. What makes it even stronger is its ability to find your “neighbours” – people you don’t know who have similar musical tastes. Listening to your neighbourhood radio is like having a friend who’s a DJ and always pushing new and interesting songs at you.

I don’t know any of these people, but I like their musical tastes.

Maybe instead of software agents we need software that connects us to other people who have similar interests? I read LifeHacker because I know the editors have very similar sensibilities to what I find interesting. Jon Udell shares my same love for information organization and manipulation. Jeff Atwood has perhaps one of the most engaging blogs for general geekery and love of programming, and his twitterstream is always full of interesting links.

The only downside to filtering information is that restricting your input to the people you already agree with creates a reinforcing feedback loop and destroys your patience and your ability to be around people with differing outlooks.

Digital Obsolescence

8 comments

Blog Tip: Create a Link Post in 3 Seconds

Posted in Becoming a Better Blogger, Delicious, Technology, Yahoo Pipes by engtech on October 03, 2007

This is the successor to my post on how to build a weekly digest in 3 seconds.

One question I’m frequently asked is “how do you build those Best of Feeds weekly links?” The way I do it is pretty complicated, but I’ve found a much simpler solution that I want to share with you all. Building a list of links is something every blogger does at one time or another, and it doesn’t have to be hard.

Why Create a Link Post?

Link posts are great ways to share and acknowledge interesting links. Linking to other blogs is what makes the blogosphere tick. If you don’t routinely read and link to other bloggers then your using your blog as a one-way soapbox instead of as a medium for sparking communication and building relationships.

Link posts can be used for a variety of reasons:

Weekly Round-up
List of resources about a subject
List of group writing participants
List of contest participants

Here are some more tips from the experts on why create a link post

Step #1: Use Delicious to Save Links

I’m a delicious power user and it’s my favourite site for bookmarking interesting links. It integrates nicely with whatever web browser you are using.

This video explains how to use Delicious to bookmark sites

Delicious already comes with a way of posting a daily link report, but I don’t like it because I feel like I’m spamming my regular readers if my blog is filled with “links for 2007-10-02” instead of stuff I wrote myself. I much prefer posting once a week, or having full control over when I post my list of links.

But the delicious tagging system is so useful for building a list of links around a specific subject, and for attaching short descriptions around each link. For instance, I used the ‘project3’ tag when I was picking out my favorite posts from the Project 3 group writing project on Daily Blog Tips.

Delicious also integrates nicely into your web browser, no matter what it might be.

*new* firefox bookmarks extension
firefox extension *this is the one I use*
internet explorer buttons
bookmarklet buttons for any browser

Step #2: Use Delicious Link Builder

I’ve created a Yahoo Pipe that builds a list of your del.icio.us links that you can cut-and-paste into a blog post.

Put in your delicious username
Optional: Filter your links by a tag
Optional: Filter your links by date
Optional: Limit the number of links (maximum is 31, this is a limit from del.icio.us)
Click ‘Run Pipe‘
Cut-and-paste the results into a blog post using your WYSIWYG editor

The Results

This is an example of a list from my delicious saved bookmarks.

[CODE] Software Is Hard
- *Excellent* article about software estimation and Rosenberg’s Dreaming in Code
[ESTIMATION] Web Worker 101: Estimating Basics
- Nothing new, but good round-up for people who have trouble estimating.
[LIFEHACKS] The Printable CEO
- Collection of PDFs for task/hour tracking.
[RAILS] Build Your Own Ruby on Rails Web Applications
- Free e-book for next 60 days. Probably not as good as Agile Web Development.
[WORKHACKS] Cover Your Butt At Work with Thorough Notes
- If you have to CYA that much then you might want to say cya to that job. But there is a lot of be said for indoctrinating the people around you that you are coordinated and correct because everything is tracked.

That’s all there is to it. Bookmark web pages with delicious, then go into Delicious Link Builder when you want to make a list of them.

You can start by bookmarking this post. :)

Advanced Users – Pretty Cut-n-Paste

I use a Greasemonkey script in Firefox to make the output of Yahoo Pipes a little bit nicer.

Advanced Users – Clone Your Own Pipe

If you’re logged into Yahoo then you’ll have the option to ‘clone’ my Pipe (Delicious Links Builder). This means you have your own copy of it and you can change the default values for the fields to whatever you want, eg: always default to your username, and to 7 days worth of links.