So I updated my pet project last Friday, after deciding that I had been dragging my feet on it for long enough. The Infinitube revamp was essentially brought on by Two Major Issues. The First Issue was that the site was getting 120,000 average pageviews per day over the past week and was getting noticeably slow. (On the day I started rewriting it, it peaked at 230,000 pageviews, i.e., more than 3 pages per second over a 24-hour period. If that wasn’t a motivation to start rethinking the architecture, I don’t know what is.) The Second Issue meanwhile was that I still hadn’t addressed one of the really important specs I listed down back when I first launched this thing in mid-July, i.e., having full-screen viewing.
With those two things firmly in mind, I set about putting together the 2nd version of Infinitube on Thursday the 7th. The first issue was whether the whole 120,000-daily-pageviews thing was even really warranted. I mean, it’s a nice number to bandy around and all, but if there were a way to maintain or improve the Infinitube user experience while generating fewer pages (thus reducing the overall load on Morph’s servers), then that would be the better strategy overall. Remember that we were running the cheapest Morph package ($1/day) to host this thing, and it looked like the two mongrels handling the site were getting the shit kicked out of them. But I digress.
The first question we needed to answer was: why were the pageview numbers so bloated anyway?
Let’s break the problem down further. At this point in time, Infinitube still hadn’t broken more than 3,500 unique visitors per day. Except for that one weekend when we were massively Stumbled, my daily numbers were always within 2,500 - 3,000 uniques. So if you’ve only got 3,500 visitors on any given day, generating 150,000 pages for them seemed a bit excessive. Even on an active community site like Highfiber, the pageview-to-visitor ratio is usually only about 10:1. Here, we were talking more like 30:1, on a site that had near-zero interaction.
Of course, the site concept itself, coupled with my initial architecture, were the real culprits here. You probably already know by now that Infinitube’s concept is essentially “unattended Youtube viewing,” so interactivity is pretty minimal. The problem was that in order to keep new videos constantly streaming to the browser, my solution was to refresh the page to replace the previous video with the new one. It was this architectural approach that was causing the massive bloat in terms of pageviews, because if you happened to leave your browser on Infinitube for an entire day, you’d probably rack up literally hundreds of pageviews all by yourself.
This page-refreshing silliness was largely unnecessary from a technological standpoint, and as it turns out, my solution for the Second Issue (full-screen viewing) ended up solving this First one. I reasoned that if you could feed a list of Youtube video URLs to a generic Flash player, you wouldn’t need to refresh your page at all. The player would simply take the next URL on the list, load it up, and play it. The tradeoff is that now you’ve off-loaded all of your navigational logic to a Flash player that you may or may not know how to fix. Or for that matter, build.
Thankfully, a little bit of research showed that many open-source Flash players like this already exist on the web. The one I eventually went with was Jeroen Wijering’s JW Player, which had everything I needed, and then some. Fullscreen mode, playlist mode, extensive API, skinnable interface, the works. (Yes, I paid a small commercial license for it. Considering that he saved me probably a week’s worth of futzing around in ActionScript, it seemed only proper that I gave him some money for it.)
Armed with my new Flash player, all that was left for me to do was figure out how to feed it a list of Youtube videos. JW supports a whole boatload of playlist formats, so I really only needed to pick the one I liked. I was just about to start rendering my search-results in the RSS format when I discovered this Youtube/GData announcement from late last year. It turns out that Google was already generating dynamic RSS feeds for any Youtube-related keyword or search-method you could think of. You didn’t even need a developer key to access it. All I had to do was forward the visitor’s search terms to the appropriate GData url, and it would come back with a nice, JW-compatible RSS feed.
At that point, both of my Major Issues were pretty much solved. I had no more processor-intensive searching to think about, the database was freed up from having to sort hundreds of thousands of cached video records, and I had the improved Flash functionality that I didn’t have access to with Youtube’s official player. There were a few minor bits that still needed to be figured out though — there’s really no such thing as a free lunch with these things — but they were nothing that a little reading didn’t solve quickly enough. (How to catch errors from outside the JW player was one of my primary concerns; it’s still only partially there at the moment.)
So what’s left to do on Infinitube? Within the next month, I’m planning on adding user accounts so visitors can squeeze some more lazy automation out of the app. Essentially, I want you to be able to go to http://infinitube.net and have a video start playing automatically. It would, of course, match whatever you previously searched for. Ideally, it would even pick up where you previously left off. (To a certain extent, I can pull this off right now just by tracking session data, but it wouldn’t work if you had multiple machines.)
I also want to see if there’s any way to pre-load data so that the transition from video to video is shorter. I’d also like to have some way to download the videos that I happen to like. That’s potentially a very expensive operation though, so unless users are willing to pay for the service of encoding, I doubt I could pull that one off. And of course, I’m pretty sure I’d be breaking some kinda Youtube/Google terms of service by doing that. (Although, what exactly? Worth looking at, at the very least.)
Lastly, keyboard shortcuts in combination with iRed Lite would allow me to control Infinitube using my Apple Remote. That would be crazy awesome.
So after about a year of that, I started reading about 
