// Internet Duct Tape

How to Get an RSS Feed for your XBOX 360 Gamertag

Hacking RSS with Yahoo Pipes

My geek want of the day is getting an RSS feed of my Xbox 360 game activity so that I can use it with lifestreaming services. For once I’m not the only person who feels this need. There’s at least two of us! :)

badger-tweet.png

I’m not sure why Microsoft doesn’t make an RSS feed of your Xbox Live activity available. The information is all there, they publish it as a gamercard. But they don’t give you access to the raw data to do with as you please unless you’re a member of the Xbox Community Developers Program. Here are the various ways you can access your Xbox 360 Gamercard to use with other websites.

(more…)

Building an RSS Templating System

Hacking RSS with Yahoo Pipes

The blog posts might have been slow lately, but that’s only because there’s been an accumulation of interesting projects piling up on my hard drive. Here’s a few things that are in the pipe (which I’m talking about to stop my procrastination and force me to release them):

  • Sandbox Theme for Tumblr so that you can use Sandbox CSS themes on tumblr.com
  • Tumblr Theme Templates to make it easier to develop themes for tumblr.com without having to upload your theme to tumblr.com
  • Tumblr automatic backup + restore
  • WordPress.com automatic backup
  • Twitter There Will Be Followers – program to automatically follow back anyone who is following you on Twitter
  • PostMaster html2blog – automatically post formatted HTML to Tumblr, Blogger or WordPress
  • rss2html – powerful templating system for converting an RSS feed into HTML

It’s the last one I want to talk about. I’ve gotten tired of using Yahoo Pipes + Pipe Cleaner to build digest posts. It’s kind of a pain in the butt. So I want something that can take an rss feed, convert it to html so that I can use another program for automatically posting it to the blog. I’m not going the plugin route because of WordPress.com’s inability to support javascript or PHP plugins.

This is what I’ve come up with.

(more…)

Tagged with:

Yahoo Pipe: Sub-Reddit Feed Filter

Posted in Delicious, Reddit, Ruby on Rails, Technology, Yahoo Pipes by engtech on January 28, 2008

Hacking RSS with Yahoo Pipes

Popular social bookmarking site Reddit has announced a great new feature: users can create their own sub-reddit. What does this mean in English? Users and communities can create their own social bookmarking sites around specific topics: blogging, wordpress, specific programming languages, etc but still use their regular reddit account for submitting links and voting.

You can see a full list of all the new reddits here, sorted by popularity. Of particular interest to me is the new Reddit created for Ruby/Rails related posts.

Of course, it’d be nice to be able to subscribe to a filtered version of these links. I’ve created a modified version of Dave S‘s “reddit popular on delicious” Yahoo Pipe that works with Sub-reddits.

  1. Click on the link
  2. Enter the name of the sub-reddit you’re interested in
  3. Enter the minimum number of saves on a delicious before a link is included in the feed
  4. Enter keyword inclusion/exclusion filters if you want to limit what you get
    • ie: include only rails-related posts or exclude all rails-related posts
  5. Click Run
  6. Click on the subscribe to RSS button

I’m using the Ruby sub-reddit as an example, but this is a great way to track links based around any topic there is a sub-reddit for. Even lolcats.

I’m looking forward to when this Reddit feature comes out of beta and it’s possible to create a few new sub-reddits like blogging, wordpress and lifehacks.

Related Posts

Blog Tip: Create a Link Post in 3 Seconds

Posted in Becoming a Better Blogger, Delicious, Technology, Yahoo Pipes by engtech on October 03, 2007

Bloggin Tips and Tricks

This is the successor to my post on how to build a weekly digest in 3 seconds.

One question I’m frequently asked is “how do you build those Best of Feeds weekly links?” The way I do it is pretty complicated, but I’ve found a much simpler solution that I want to share with you all. Building a list of links is something every blogger does at one time or another, and it doesn’t have to be hard.

Why Create a Link Post?

Link posts are great ways to share and acknowledge interesting links. Linking to other blogs is what makes the blogosphere tick. If you don’t routinely read and link to other bloggers then your using your blog as a one-way soapbox instead of as a medium for sparking communication and building relationships.

Link posts can be used for a variety of reasons:

  • Weekly Round-up
  • List of resources about a subject
  • List of group writing participants
  • List of contest participants

Here are some more tips from the experts on why create a link post

Step #1: Use Delicious to Save Links

I’m a delicious power user and it’s my favourite site for bookmarking interesting links. It integrates nicely with whatever web browser you are using.

This video explains how to use Delicious to bookmark sites

Delicious already comes with a way of posting a daily link report, but I don’t like it because I feel like I’m spamming my regular readers if my blog is filled with “links for 2007-10-02” instead of stuff I wrote myself. I much prefer posting once a week, or having full control over when I post my list of links.

But the delicious tagging system is so useful for building a list of links around a specific subject, and for attaching short descriptions around each link. For instance, I used the ‘project3’ tag when I was picking out my favorite posts from the Project 3 group writing project on Daily Blog Tips.

Delicious also integrates nicely into your web browser, no matter what it might be.

Step #2: Use Delicious Link Builder

I’ve created a Yahoo Pipe that builds a list of your del.icio.us links that you can cut-and-paste into a blog post.

  1. Put in your delicious username
  2. Optional: Filter your links by a tag
  3. Optional: Filter your links by date
  4. Optional: Limit the number of links (maximum is 31, this is a limit from del.icio.us)
  5. Click ‘Run Pipe
  6. Cut-and-paste the results into a blog post using your WYSIWYG editor

Delicious Link Builder

The Results

This is an example of a list from my delicious saved bookmarks.

That’s all there is to it. Bookmark web pages with delicious, then go into Delicious Link Builder when you want to make a list of them.

You can start by bookmarking this post. :)

Advanced Users – Pretty Cut-n-Paste

I use a Greasemonkey script in Firefox to make the output of Yahoo Pipes a little bit nicer.

  1. How to Install Greasemonkey
  2. How to Install a Greasemonkey Script
  3. Install Yahoo Pipe Cleaner

Advanced Users – Clone Your Own Pipe

If you’re logged into Yahoo then you’ll have the option to ‘clone’ my Pipe (Delicious Links Builder). This means you have your own copy of it and you can change the default values for the fields to whatever you want, eg: always default to your username, and to 7 days worth of links.

Advanced Users – StumbleUpon

If you’re using delicious to save bookmarks, you can also use another handy Greasemonkey script I created that lets you save web pages to StumbleUpon at the same time you’re saving them to Delicious.

Related Links

There’s Plenty More

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

This post was written as part of the Geeks Are Sexy Ultimate “How-To” contest.

Greasemonkey Script: Yahoo Pipe Cleaner

Posted in Firefox and Greasemonkey, Technology, Yahoo Pipes by engtech on August 13, 2007

Hacking RSS with Yahoo Pipes

I’m a very big fan of Yahoo Pipes. It’s an amazing service that lets you take information from websites (using RSS, XML, JSON) and then do all kinds of filtering and manipulation with it. It is all done with a slick graphical user interface but it is not for the faint of heart — it is much easier to create new pipes if you have a programming background. But once a pipe is created it is simple for other people to use it. For example, this is how you can create a blog digest post using a Yahoo Pipe I’ve created for you.

Yahoo Pipes can create automated lists that you can cut-and-paste into blog posts. My only real complaint is with the HTML markup they create. It doesn’t look good when you cut-and-paste it into a WordPress blog. This is where Yahoo Pipe Cleaner comes in. It is a Greasemonkey script for Firefox that fixes the Yahoo Pipe output so that it looks nicer when you cut-and-paste it into a WordPress blog.

  • removes any H1, H2, H3, H4, H5, H6 headers
  • dofollows the links (removes rel=nofollow)
  • replaces paragraphs with list elements
  • removes all class/id CSS selectors

Without Yahoo Pipe Cleaner

 

  • Facebook Tip: Broadcast your Facebook status as RSS

    RSS is one of the most useful tools out there for moving information around on the web. Recently the concept of “micro-blogging” status updates has become very popular with applications like Facebook, Twitter and Pownce. The only problem is that it is a pain to update many sites at the…

  • How Do People Use Google Reader with Internet Explorer?

    Any computer geek worth his salt has been through the drill: you go to visit a family member for dinner and eventually they mention some arcane problem they’ve been having with software you long ago expunged from all of your computers. Common culprits are the Unholy Triad: Microsoft Outlook,…

With Yahoo Pipe Cleaner

  • Facebook Tip: Broadcast your Facebook status as RSS
    • RSS is one of the most useful tools out there for moving information around on the web. Recently the concept of “micro-blogging” status updates has become very popular with applications like Facebook, Twitter and Pownce. The only problem is that it is a pain to update many sites at the…
  • How Do People Use Google Reader with Internet Explorer?
    • Any computer geek worth his salt has been through the drill: you go to visit a family member for dinner and eventually they mention some arcane problem they’ve been having with software you long ago expunged from all of your computers. Common culprits are the Unholy Triad: Microsoft Outlook,…

Get Yahoo Pipe Cleaner

You can find installation instructions for Yahoo Pipe Cleaner here.

Facebook Tip: Broadcast your Facebook status as RSS

Posted in Facebook, Technology, Twitter, Yahoo Pipes by engtech on August 08, 2007

RSS is one of the most useful tools out there for moving information around on the web. Recently the concept of “micro-blogging” status updates has become very popular with applications like Facebook, Twitter and Pownce. The only problem is that it is a pain to update many sites at the same time. It is better to pick one and broadcast RSS to the others. I’m going to show you how to broadcast your Facebook status to Twitter.

IMPORTANT UPDATE: The Twitter Facebook App now lets you control your Facebook status from Twitter. This tip lets you posts your Facebook status in Twitter. Using them together is a very bad idea.

How to Find the RSS Feed for Your Facebook Status

This is actually the hardest part.

  1. Login to Facebook
  2. Click on Profile tab
  3. Under the Mini-Feed heading click on See All
  4. Click on Status Stories from the right hand column
  5. Right click on My Status and copy the link

facebook rss status

Filtering Your Status with Yahoo Pipes

I’ve put together a Yahoo Pipe that filters your status. This isn’t necessary, but it makes the status updates look a little bit better in other applications like Twitter. It removes your name, and changes the link to go to your profile instead of the individual status. Feel free to clone it and tweak it some more.

Eric is washing his cat.

becomes

is washing his cat (from Facebook status).

and the feed link is set to your Facebook profile.

  1. Go to this Yahoo Pipe
  2. Copy your Facebook status RSS feed
  3. Click Run Pipe
  4. Click on Subscribe
  5. Right click on Get as RSS and copy link

You can now put this filter RSS into TwitterFeed, your blog sidebar, etc.

Special Thanks

Related Posts

WordPress Tip: Create a Digest Post in 3 Seconds

Posted in Becoming a Better Blogger, Technology, Yahoo Pipes by engtech on July 13, 2007

It’s a good habit to post a summary of your recent posts once or twice a month, but like all blog maintenance it can be a pain in the butt if you don’t make it as easy as possible. Here’s a hack that’ll let you create a summary of all of your posts over X number of days using a handy-dandy Yahoo Pipe.

How to Create a Digest Post

  1. Click on this link to go to the Yahoo Pipe
  2. Change “Truncate feeds older than 7 days ago” to the number of days back you want to go
  3. Change “Enter RSS URL” to the feed address to match your blog
  4. Click on the Run Pipe button
  5. Cut and paste the output from the pipe into a new blog post using the WordPress rich text editor

Ta-da! Now you’ve made a digest post. You can edit the text and summaries as necessary, or adjust the date and re-run the pipe if the amount of time is not correct.

Advanced users can make a clone of that pipe and change it to have your feed url and the date range you want by default.

Sample Digest Post

This was cut and pasted from the pipe output with no modifications.

 

Advanced Users

It doesn’t look great when you cut-and-paste the code from the Yahoo Pipe to a WordPress blog post. You can fix that by using my Yahoo Pipe Cleaner script with Greasemonkey.

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

More Pipes

Here’s a list of more Yahoo Pipes I’ve created.

Be My Friend (on social network sites)

Unless you’ve had your head in the sand (or *gasp* you aren’t obsessed with Internet culture), then you’ve noticed that we’re seeing more and more web service startups over the past few years. Last year I flamed the Bubble 2.0 soundly in “Web Too.Many.” Earlier this year I tried to get an idea of what websites people actually use by starting the What’s Your Web 2.0? meme.I think I’m past breaking when it comes to my attention span and the number of services that I use. But that doesn’t mean there aren’t some I heartily recommend. If you use some of these services as well, then please go ahead and “friend” me. And yes, there is a Facebook account in the list.

Active

Software Hosting

Autopilot

  • Twitter – sharing, republishing
  • Tumblr – aggregating my online presence
  • Jaiku – sharing, republishing

Deadpool

These are sites that I used to use often, but I’ve given up on:

  • MyBlogLog – too much spam
  • Flickr – it’s easier to share photos with friends on Facebook
  • RottenTomatoes – it’s easier to share movie reviews with friends on Facebook
  • Technorati – never indexes me properly
  • Blogcritics – got some books, but they aren’t a very good source of traffic
  • LinkedIn – until the next time I’m looking for work
  • … more than I can possibly remember.

What got me thinking about this is trying out Pownce for the first time and seeing how horrible it is at re-discovering my friends.

I just sent out some invites to Kevin Rose’s Pownce to my FeedBurner subscribers. (Thanks for hooking me up, Adam)

Pownce First Impression

I’ve seen other complaints that the biggest problem of Pownce is “what do I do with it?” It’s probably the most powerful web-based instant messaging client out there.

I was very surprised that they Pownce doesn’t have an “import contacts from address book” feature. That is rapidly becoming the only way to easily import the list of your friends from one web app to another. I was trying out Blue Swarm the other day and they are using a very slick widget from Plaxo that does easy address book imports. All web startups should use this, since email address contacts are the only universal data format for identifying your friends on the web.

This is a perfect example of why the Facebook application experience is so powerful… signing up and maintaining a user account is the major barrier that prevents most web startups from gaining a massive user base. “Social” web sites have an even bigger barrier in that you have to move or re-find your network. Facebook apps allow for any application to have the same user account and social network.

Obviously I think it would be pretty awesome if that Plaxo contact importer also supported Facebook as well as Gmail, Hotmail, Outlook, and Yahoo.

What Others Have to Say

Scoble: “I see many of the same people in my friends list on Twitter, Jaiku, Facebook, and now Pownce. Pownce is growing faster than the other ones right now, though. 728 people have already added me on Pownce. I can’t take many more social networks.

Memoirs of a Bystander: “My question is this: Is there any value actually garnered from adding an obscenely large amount of random people as your friend on various social sites? Honestly, if a social networking site it meant to enhance you life through discovery of new interests, music, recommendations, etc…, is that easily done by wading through thousands of people?”

Mashable: “And it is this: on Pownce, you can send a message, or a file, or a song, or an event, to one person; or three of your friends; or only your family; or everyone.”

SocialHam: “Now more than ever Email seems to becoming a dead medium so can micro blogging sites fill in the gaps?”

JetPacked: “Can’t decide between pownce and twitter? This should help. Here’s how to post your pownces to twitter.”

Daily Grumble: “Social networking is a very difficult area for a new service to break into. How on earth are you going to persuade users of other, more established social networks to come to your service?”

Greg Verdino: “Are social media mavens living inside a bubble of our own making, artificially inflating the impact that most of these nascent technologies are having on the population in general, and ultimately getting our companies and our clients riled up over something that will, over time, turn out to be, well um, nothing?”

The Programmable Web – Yahoo Pipes

Posted in IDT Labs Software Development, Perl, Technology, Yahoo Pipes by engtech on May 30, 2007

Duct tape is a great tool because it is so shiny and sticky. You can use it to glue so many things together, even if they end up looking like Frankenstein by the end of it. All you need is a camera, a cellphone, an MP3 player and a piece of duct tape to get yourself the only mobile convergence device worth having. The programming language Perl has quite often been called “duct tape for the internet” because it lets you easily transform text and interact with web sites.


(photo by philgarlic)

Here is a simple Perl script that downloads an RSS feed and bookmarks each entry to del.icio.us:

my $delicious = Net::Delicious->new(
				    {'user'=>$user,
				     'pswd'=>$password},
				    'updates'=>'.',
				    'xml_parser'=>'simple',
				    'debug'=>1);
my $feed = XML::FeedPP::RSS->new($rss_url);
$feed->normalize();         # Sort by pubDate and remove non-unique
foreach my $item ($feed->get_item()) {
  my $description = $item->description();
  $description =~ s/<.*?>//g; # remove HTML
  my %args = ('url'=>$item->link(),
	      'description'=>$item->title(),
	      'extended'=>$description,
	      'tags'=>"from_feed",
	      'replace'=>'no');
  my $retval = $delicious->add_post(%args);
}

The problem with Perl is that you have to either run it on your own machine, or buy web hosting that lets you run your own Perl scripts (or Python/Ruby). This is a real pain in the butt.

Enter Yahoo Pipes

Yahoo created one of the most innovative web tools I’ve ever seen. Yahoo Pipes lets you do all kinds of conversions and filtering on the web without requiring a web host to host your programs. If you want to convert XML/RSS data to other XML/RSS then look no further.

There’s still room for improvement:

  • Scraping web sites that do not have information in XML/RSS. There are other companies that let you do this, but they’re even harder to use than Pipes (IE: dapper.net).
  • Notifying you when your pipes don’t work. That makes finding existing pipes and mashing up multiple pipes tricky as best. It’s hard to use a tool when things constantly change underneath you.
  • It would also be nice if allowed HTML in the Pipes descriptions as it is hard to describe how to use them sometimes.
  • Better debugging messages when developing your own Pipes

Yahoo Pipes is targeted towards programmers, not casual users, but there is still a million and one things you can do with it. Here are some of my pipes that are free for other people to use.

Simple

Social Sites

Personal

Got Pipes?

Are there any specific RSS feeds mashups you’re looking for but don’t have the Yahoo Pipes expertise to create? Leave a comment on this post and I’ll see what I can come up with.

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

Get RSS Updates when your Site is Submitted to Digg or Reddit (Yahoo Pipes)

Posted in Digg, IDT Labs Software Development, Reddit, Technology, Yahoo Pipes by engtech on March 28, 2007

I’ve put together an RSS widget that I like to call Social Site Submission Watchdog. It creates an RSS feed for when people submit your site to digg.com or reddit.com. The reddit results are tweaked so that they link to the voting page on reddit instead of the page on your site. The link title indicates which site the source is coming from. I’ve also created a Reddit-only version of the Pipe.

This is an essential tool for building a dashboard for your blog.

This is an essential way to keep track of your site so that you can prep a post for the potential wave of traffic heading towards it by doing things like adding a Digg This widget. It’s also a good way for a reader to keep track of specific sites they want to support.

digg reddit rss mashup yahoo pipes

Any feature requests? Leave a comment.

Digg Pipes

Related Posts

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

Hack – Display Your Technorati Rank in a Sidebar RSS Widget (Dapper + Yahoo Pipes)

Posted in IDT Labs Software Development, Technology, Technorati, Yahoo Pipes by engtech on March 26, 2007

technorati rank imageWordPress.com bloggers can’t use nifty WordPress plugins or Javascript to doing something as simple as displaying their Technorati rank in their sidebar. Dapper.net and Yahoo Pipes both give the users the ability to create their own mashups of existing web services. Neither of them quite get it right — Dapper doesn’t give you enough output / programming options and Yahoo Pipes doesn’t give you an interface for scraping any page the way Dapper does. Which is my long-winded way of saying I got Dapper to create RSS feeds that contain nothing but your Technorati rank.

Of course, the interface to the Dapper RSS feed is less than stellar, which is where Yahoo Pipes comes in. Yahoo Pipes also lets me create several flavours depending on what people want.

See the full list of free software I have created.

You can get frequent updates about all of my new software, tools or blog themes by subscribing to IDT Labs by RSS or by email. Or you could just subscribe to my main blog, Internet Duct Tape.

Subscribe to feed

(more…)