// Internet Duct Tape

Yahoo Pipe: Sub-Reddit Feed Filter

Posted in Delicious, Reddit, Ruby on Rails, Technology, Yahoo Pipes by engtech on January 28, 2008

Hacking RSS with Yahoo Pipes

Popular social bookmarking site Reddit has announced a great new feature: users can create their own sub-reddit. What does this mean in English? Users and communities can create their own social bookmarking sites around specific topics: blogging, wordpress, specific programming languages, etc but still use their regular reddit account for submitting links and voting.

You can see a full list of all the new reddits here, sorted by popularity. Of particular interest to me is the new Reddit created for Ruby/Rails related posts.

Of course, it’d be nice to be able to subscribe to a filtered version of these links. I’ve created a modified version of Dave S‘s “reddit popular on delicious” Yahoo Pipe that works with Sub-reddits.

  1. Click on the link
  2. Enter the name of the sub-reddit you’re interested in
  3. Enter the minimum number of saves on a delicious before a link is included in the feed
  4. Enter keyword inclusion/exclusion filters if you want to limit what you get
    • ie: include only rails-related posts or exclude all rails-related posts
  5. Click Run
  6. Click on the subscribe to RSS button

I’m using the Ruby sub-reddit as an example, but this is a great way to track links based around any topic there is a sub-reddit for. Even lolcats.

I’m looking forward to when this Reddit feature comes out of beta and it’s possible to create a few new sub-reddits like blogging, wordpress and lifehacks.

Related Posts

The Attention Age: Accelerando, Software Agents, Filters and Gatekeepers

Posted in Book Reviews, Digital Culture, RSS Syndication, Software, Technology by engtech on October 17, 2007

Last night I finished reading Accelerando by Charles Stross. Like many of the books I read these days, I heard about it from another blogger. It feels like a spiritual sequel to Alvin Toffler’s Future Shock, John Brunner’s the Shockwave Rider and Warren Ellis’ Transmetropolitan. It is about information overload to the nth degree and too much change in too short of a time.accelerando charles stross

Accelerando is broken into 9 fragmented stories with decades passing in between them. This is too bad because it was the initial segment, only a few years in the future, that I found most interesting. Our protagonist is hooked up to a portable computing network of software agents that he uses to continually data mine and plug-in to a “river of news”. As he communicates with other people he spawns off parts of his “distributed brain” to research more information and get back to him.

The greatest inventions usually come from seeing the possible connection between two separate things (eg: peanut butter and chocolate). Like in the Shockwave Rider, our protagonist is successful because of his ability to gather and process information is so far beyond an average person’s. Being immersed in the information stream he sees the connections and trends that other’s can’t see.

These connections lead to so many successful ideas, that he can’t possibly execute on them himself – because the time it takes to implement them would take away from the information processing that is his true talent. He makes a career of giving away his ideas and surviving off of the reputation gain and support of his sponsors he’s made so successful. Very much like Doctorow’s concept of whuffie – reputation as currency.

The book progresses to talking about the post-human experience after digitization has reached the point that we can successfully digitally encode human personalities. Post-death society, heads in jars and living bodiless on the internet. There’s a really good bit on how the next major species will be intelligent corporations and artificial spam intelligence. But what really interested me was the initial chapters so close to the beginning 21st century: how do we use technology to deal with information overload?

(You can get a copy of Accelerando for free online – which is very useful because the copy I borrowed from the library was missing the last page – now that’s frustrating)

It’s Getting Harder to Find Information

We’re in the middle of a great revolution where anyone can become a self-publisher. But that’s the crux of the problem, isn’t it? Anyone can become a self-publisher. The low barrier to entry makes the competition for attention fierce. At some level we’re all on par with the lowliest spammers, trying to compete for other people’s attention. There is so much new content being created all the time at the only way old content stays in the public record is if the Great Google God returns it in a search result.

This is only going to get worse because Google has created a new caste of blogging serfdom. People create content and splash Google ads on it with the hope of that it will do well in Google search results so they can get paid.

There’s many a “business model” that relies completely on Google-Google Search for traffic and Google AdSense for revenue. And there’s an even larger amount of so-called business models that rely almost completely on Google for traffic, even if the money comes in via other means.

I think you know what happens to the money when the traffic stops.

I use the term “business model” above loosely, because a model that is entirely dependent on an outside company, for either traffic or revenue or both, is not really sound. You’re not in charge and you have very little control, because if Google decides to change the rules, you’re out of luck. Based on that, I would argue that relying on Google is not a business at all.

I’d say you work for Google.

From the Teaching Sells e-book

Where are the Smart Filtering Agents?

One of the things I remember clearly about the idea of intelligent agents in the early 90s was how it was going to revolutionize how we consume information. Instead of having to *gasp* pick up a newspaper, autonomous software agents would search the net finding tidbits of information what we were interested in and adapting and learning from how we interact with the results. Sci-fi books like John Varley’s Steel Beach dealt with the relationships between humans and these evolving artificial intelligences.

Take a moment to glance at the Wikipedia page on software agents; it’s quite good.

The 90s hope for intelligent agents has congealed. RSS has gotten us part of the way; now we can pick voices out of the chaos that we allow to push information to us. We can subscribe to alerts on search subjects that interest us. But aside from custom recommendation engines like Netflix and Last.FM there isn’t really a bot out there for finding information for us.

The Future: RSS Filtering

I see the fledgling baby steps of software agents delivering news. There are several sites competing for being able to filter through a list of RSS feeds and recommend the best news items to you.

There’s also the “build your own” filtering agent approach.

And let’s not forget the ability to monitor search terms.

One of the more enlightened concepts I’ve come across is FaveBot that wants to bring you the custom information you want about your favorite actors, authors and musicians.

Is the Answer Better Gatekeepers?

Is having an intelligent software agent the right approach or is it better to let humans do the filtering? The past year has seen an incredible rising in using crowdsourcing to decide what is the best information available. This is how digg, reddit, stumbleupon and the delicious popular page find interesting information by using the wisdom of mobs. Unfortunately when the user-base grows too large it becomes watered down to only common denominators.

The other approach is to find human editors to act as your gatekeeper. I’m not talking about hiring your man in Mumbai, but rather niche news sites like Slashdot, BoingBoing and Fark, and to a greater extent using the network of blogs you enjoy to act as your information gate keepers.

The last.FM music service is an amazing tool for finding new music to listen to. What makes it even stronger is its ability to find your “neighbours” – people you don’t know who have similar musical tastes. Listening to your neighbourhood radio is like having a friend who’s a DJ and always pushing new and interesting songs at you.

last.fm music neighbourhood

I don’t know any of these people, but I like their musical tastes.

Maybe instead of software agents we need software that connects us to other people who have similar interests? I read LifeHacker because I know the editors have very similar sensibilities to what I find interesting. Jon Udell shares my same love for information organization and manipulation. Jeff Atwood has perhaps one of the most engaging blogs for general geekery and love of programming, and his twitterstream is always full of interesting links.

The only downside to filtering information is that restricting your input to the people you already agree with creates a reinforcing feedback loop and destroys your patience and your ability to be around people with differing outlooks.

Related Posts

Blog Tip: Create a Link Post in 3 Seconds

Posted in Becoming a Better Blogger, Delicious, Technology, Yahoo Pipes by engtech on October 03, 2007

Bloggin Tips and Tricks

This is the successor to my post on how to build a weekly digest in 3 seconds.

One question I’m frequently asked is “how do you build those Best of Feeds weekly links?” The way I do it is pretty complicated, but I’ve found a much simpler solution that I want to share with you all. Building a list of links is something every blogger does at one time or another, and it doesn’t have to be hard.

Why Create a Link Post?

Link posts are great ways to share and acknowledge interesting links. Linking to other blogs is what makes the blogosphere tick. If you don’t routinely read and link to other bloggers then your using your blog as a one-way soapbox instead of as a medium for sparking communication and building relationships.

Link posts can be used for a variety of reasons:

  • Weekly Round-up
  • List of resources about a subject
  • List of group writing participants
  • List of contest participants

Here are some more tips from the experts on why create a link post

Step #1: Use Delicious to Save Links

I’m a delicious power user and it’s my favourite site for bookmarking interesting links. It integrates nicely with whatever web browser you are using.

This video explains how to use Delicious to bookmark sites

Delicious already comes with a way of posting a daily link report, but I don’t like it because I feel like I’m spamming my regular readers if my blog is filled with “links for 2007-10-02″ instead of stuff I wrote myself. I much prefer posting once a week, or having full control over when I post my list of links.

But the delicious tagging system is so useful for building a list of links around a specific subject, and for attaching short descriptions around each link. For instance, I used the ‘project3′ tag when I was picking out my favorite posts from the Project 3 group writing project on Daily Blog Tips.

Delicious also integrates nicely into your web browser, no matter what it might be.

Step #2: Use Delicious Link Builder

I’ve created a Yahoo Pipe that builds a list of your del.icio.us links that you can cut-and-paste into a blog post.

  1. Put in your delicious username
  2. Optional: Filter your links by a tag
  3. Optional: Filter your links by date
  4. Optional: Limit the number of links (maximum is 31, this is a limit from del.icio.us)
  5. Click ‘Run Pipe
  6. Cut-and-paste the results into a blog post using your WYSIWYG editor

Delicious Link Builder

The Results

This is an example of a list from my delicious saved bookmarks.

That’s all there is to it. Bookmark web pages with delicious, then go into Delicious Link Builder when you want to make a list of them.

You can start by bookmarking this post. :)

Advanced Users – Pretty Cut-n-Paste

I use a Greasemonkey script in Firefox to make the output of Yahoo Pipes a little bit nicer.

  1. How to Install Greasemonkey
  2. How to Install a Greasemonkey Script
  3. Install Yahoo Pipe Cleaner

Advanced Users – Clone Your Own Pipe

If you’re logged into Yahoo then you’ll have the option to ‘clone’ my Pipe (Delicious Links Builder). This means you have your own copy of it and you can change the default values for the fields to whatever you want, eg: always default to your username, and to 7 days worth of links.

Advanced Users – StumbleUpon

If you’re using delicious to save bookmarks, you can also use another handy Greasemonkey script I created that lets you save web pages to StumbleUpon at the same time you’re saving them to Delicious.

Related Links

There’s Plenty More

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

This post was written as part of the Geeks Are Sexy Ultimate “How-To” contest.

Follow

Get every new post delivered to your Inbox.

Join 288 other followers