For those looking for a replacement

Older work, still valid

A lot of people have been sending me e-mails due to the current state of MetaHypem. If you’re looking to find alterative ways to download the music I can recommend the following.

  1. Just follow the blog link for each song. The blogs let you download the music.
  2. I’ve written previously a Chrome Extension that injects a download button into the web page.e web page.

    Don’t forget you need to right-click and select Save-As however…

  3. I wrote an example Python script outlining how to download songs that you can use or help build other tools with.

MetaHypem has been temporarily shut down

The game resumes

Over the course of this weekend, my latest pet project MetaHypem has been shut down but by HypeMachine and not through us. HypeMachine seems to have added either our domain or instance running our server onto a black-list.

Here is a nice screen capture of what current data HypeMachine is returning my scraper. The MP3 Artist/Title was a friendly sting I guess.
hypemshutdown

I haven’t done much publicizing of the server and daily hits to the site was only ~40 visitors (most returning). However clearly there was a need/love for having access to the music (~50k downloads). All the blogs that HypeMachine scrapes allow users to download the music. The purpose of a music blog is to help promote bands and new music. Keeping the music locked onto their platform seems going against their goal.

Where to go from here

I’m currently investigating (alongside my partner) different ways to resolve this issue. I’d love feedback from people expressing perhaps which they would prefer or even have experience with!

  • Move the server to EC2. Perhaps they only block the IP at which I can just tear down the server to get a new IP.
  • Change the scraping of HypeMachine to be distributed and have other instances perform the scraping.
  • Release the source code to the project and let others run local instances.
  • Figure out other inventive ways!

Small Update + More to Come!

Happy New Year!

I’ve finally gotten back into development for MetaHypem (alongside Alex) and most of it has been spurred/inspired by the amount of songs downloaded! It figuratively blew my mind to have 30,000 downloads…

I’ve just released a small update that includes some fixes and 1 specific feature that has been asked about! We have some cool stuff planned for the site so I hope you keep checking it and send me feedback! We are using the feedback to drive what to work on next!

Changelog

  • 28/12/2012 – You can now search for a specific url (i.e. http://hypem.com/artist/Noosa )
  • 27/12/2012 – Added an Expire time for each song record (currently 1 hour). This is incase HypeMachine updates the download URL.
  • 27/12/2012 – Generic Bug Fixing for cases where no songs are returned from HypeMachine

As always, please send me feedback and if anyone wants to contribute (especially in terms of design/HTML/CSS contact us!)

Someone must be using the site!

Milestone Baby!

Update Since posting (~1 month) we generated 30 000 songs. Awesome!

I’ve only told few people (not that many read my blog) about Metahypem, largely because it is still a huge WIP and a learning experience. Nevertheless, the few that have been checked it out seem to be using it quite extensively.
I haven’t managed to grep the logs yet to see some interesting statistics, however we’ve recently passed the milestone of over 10 000 songs liberated.

That’s a pretty awesome number! That many times someone wanted to hear a particular song while not at their computer or over an internet connection and we aided them in that! I’ll follow up with some interesting tidbits from the request logs when I get a chance.
Thanks <3

Downloading Songs from Soundcloud

The next task

I’ve been fiddling around with HypeMachine for quite a while now even though my personal music taste has taken me elsewhere in search of music. Lately, I’ve been enjoying subscribing to individual artists on Soundcloud as a way of finding more niche electronica.

My experience so far with Soundcloud has been great. They even let you download some songs based on settings set by the artist! I however want the music while I’m on the go; especially on my jogs. I’ve now moved onto the task of seeing how I can liberate much of my beloved music from Soundcloud similarly to what I’ve accomplished with HypeMachine.

Investigation

Once again most of the information gleamed was from saving the raw HTML of a request to SoundCloud outside the browser (i.e. via python or perl). They look to be doing something similar to HypeMachine in removing certain HTML elements because I cannot find them when I view source through my browser.

We can see one of the track elements is the streamURL:

"streamUrl":"http://media.soundcloud.com/stream/WtquUHiysWgk?stream_token=H7TPl"

Node.JS Style

Grabbing the bufferTracks variable can easily be accomplished in Node.JS with the help of the jsdom. I’d simply need to build the DOM according to the HTML file and access the JavaScript variable! I’m curious if there is even an easier way…. Drop a comment if you know of one!

New Domain Name

More information to come

tl;dr; I’ve moved the download site to metahypem.com

Me and my friend Alex have been hard at work revising my original concept for hypemfinder.

Out with the old and in the new. I’ve purchased a new domain name, metahypem.com which will be the domain name for the application for now!

The site is redesigned in Node.JS and I will be providing some cool information regarding it in the near future. For now we have deployed the instance on a small server. Please don’t murder our traffic 🙂

Getting things to play nice

The move to Nginx

It’s been only one month on my new Linode machine and I’ve only ever known of Apache as a front end HTTP server. Apache sadly had to get the boot as I was preparing in deploying me and Alex Hortin’s new Node.JS version of the site to facilitate downloading songs from HypeMachine.

Node.JS comes with a complete event-driven web server however I needed a front-end server so that I can host multiple sites on my single instance (such as this blog!). Having a blocking HTTP front-end such as Apache proxy requests to Node? Sounds retarded.

The whole appeal of Node.JS is that it’s event-driven and non-blocking which hopefully results in really speedy responses. Luckily Nginx (pronounced EngineX) is also an event-driven web server and is a perfect match as a proxy server for a Node.JS application.

Apache is like Microsoft Word, it has a million options but you only need six. Nginx does those six things, and it does five of them 50 times faster than Apache. — Chris Lea

Continue reading “Getting things to play nice”

Python Download Script Updated

I’m so slow

I’m been so caught up in my new website project (downloading songs from HypeMachine as well), that I’ve really left the extension and python script fall behind.
In an attempt to try and rectify this, I’ve made the first step by fixing/updating the python script. I’ve already documented many of the changes that needed to occur in my previous blog post but now here is the full source for those who are curious!

You can download the script here: https://github.com/fzakaria/HypeScript

I’ve substantially cleaned up the code and removed the sqlite (database) aspect to it. This has resulted in a nice clean barebone downloader which does the one job it needs to do; download’em songs!. I’m always looking for advice, comments and feedback.

[BETA] Download HypeMachine Songs Website

Seriously. Beta

For those in a rush and want a tl;dr; navigate to http://hypemfinder.fzakaria.com

Hey everyone! I decided to throw up what I had so far for the site. It’s working but a lot of functionality is missing however the most important part (downloading the songs) is extremely functional 🙂

Instructions

  1. First navigate to http://hypemfinder.fzakaria.com
  2. Enter a valid hypem.com url (i.e. http://hypem.com/popular or http://hypem.com/track/1qb1t)
  3. Click Search
  4. It may take a few seconds (if searching a lot of songs), but the list of found songs will apear
  5. Click the download button and enjoy!

Help, Feedback, Donations and Criticism Appreciated

This was honestly a test toy project to learn Django so don’t expect too much however I’d love some user feedback on features, usability or even get a hold of me if you’d like to contribute!
There’s a few things left to still do to make it polished and I’d love some help with it.

To do list

  • Figure out URL scheme and remove the /hypemfinder/ app name at the end
  • Buy domain name (figure out domain name)
  • Figure out how to properly setup logging. Right now when I perform manage.py actions it creates a logfile that Apache cannot write to!
  • Add Contact/About/Tutorial page
  • Add detail page for songs
  • Add extended search functionality to make use of Hypem.com’s search bar!
  • Cleanup CSS and make prettier

Hello Linode and New Subdomain

All Aboard Linode

I’ve just finished transferring my blog and files in general from my previous shared hosting solution at bluehost.com to linode.com. The amount of freedom and boostrapping needed to move my website from bluehost to linode has been amazing and annoying

Linode offers amazing beginner guides which were invaluable to the migration however this blog post this blog post outlines a lot of information on moving a WordPress blog.

New subdomain!

You might also notice that my blog is now hosted on a new ‘blog‘ subdomain. I hope I’ve fixed a lot of the links however if you notice some broken ones, please send me an e-mail!

Continue reading “Hello Linode and New Subdomain”