unnnhhh!

DJs generally have a large music collection and I am no different: 15,000+ vinyl records, a shelf full of cds (both store bought and burned) and 500+GB of digital music. As a Mac person, all of my digital music is organized in iTunes. I like iTunes. Sure, it has a few faults like the incessant checking for gapless playback information on a few tracks that I can’t seem to get it to turn off, but, in general, it is pretty robust and does the job. iTunes has been my program of choice for 8 years now, both for my personal music collection, the tracks I dj with and to organize the music for my digital distribution company, AMP Collective.

One of the things I use most in iTunes is Playlists. I use them to organize groups of music to burn the cds I use to dj (I am not a laptop dj). Using Playlists serves several purposes: when I get a lot of new music, I go through and rate the tracks and then group the best tracks in a ‘master’ group by giving it a playlist. Ratings is another feature in iTunes that I just started using in earnest in the past couple years. I only rate the tracks I will play out and those either get 4 or 5 stars – tracks below that threshold are not worthy for me to play (that is part of my job as a dj). From the master playlist, I then make smaller groups by playlist that will each fit on a cd. I always burn two copies of each cd to put in my wallet so that I can either play two copies of a track, or another track on that cd even if one is playing in the other CDJ.

As any dj will tell you, cds get scratched, left in the cd players or given away. Thus, my playlists are vital documents that keep my cd wallet organized and up to date. After a dj trip, I can look through my wallet and see which cds are missing and go back to my playlists and burn them. I use Toast to burn my cds, mainly because it supports CD TEXT which grabs the artist and title information and displays it on the cd players we use (as well as many car and home stereos). Recent versions of Toast grab all your playlists from iTunes making it much easier to organize your tracks in iTunes and then burn with Toast.

When I got back from Miami about ten days ago, I could tell something was amiss with the external drive that stores my entire iTunes library. Even if nothing was wrong, the drive would run out of space in a matter of weeks and it is always good practice to leave approximately 1/3 empty space on any drive for maximum performance. Now, I was getting the dreaded Mac beach ball every time I wanted to play a track. It would eventually play, but I just knew this drive’s days were numbered. It was important that I got everything off this drive and moved it to a new drive before it completely died. First, I did the usual checks in Disk Utility and the drive started to unmount itself before Disk Utility could finish any repairs. This is never a good sign. When the drive would stay mounted long enough to complete a repair, Disk Utility would say the drive was OK, but I knew this just wasn’t the case.

I went out and bought a new larger drive and tried to go the normal route of changing the location of my iTunes library folder to the new drive, then checking the ‘Consolidate my iTunes music’ box in iTunes > Preferences > Library > Organize Library… This should copy all of the music on the current drive (the one that was failing) to the new drive, all the while keeping the folder structure, playlists and ratings. So, I hit ‘OK’ and went about my business as I knew copying over 485GB of music would take up most of the day. It would copy for a couple hours, then just stop with a warning that my ‘Device has been improperly removed’. Basically, the drive was unmounting itself. Not good.

Using an old trick I learned from some audio email groups I belonged to, I put the failing drive in the freezer for a while which sometimes can help a troubled drive (especially if it tends to overheat). I took it out, and started the copying process all over again. The drive would be really responsive right out of the icebox, but as soon as the drive would heat up, the same thing would happen. I quickly realized that I would never be able to copy all the files this way. Then, I remembered that I had backed up my iTunes library not too long ago (it happened to be October). So, I dragged my current iTunes Library file, as well as my iTunes Music Library.xml file which should be stored in Users/~me/Music/iTunes/ to the desktop. Then, I opened up iTunes and it was like it had never been opened – completely blank but with fresh library files. Next, I went to Add to Library… and added all the music I backed up in October. This took 9 hours. With this, I could see exactly where the old library had left off, and what I needed to copy from the old drive. I could see exactly which track was the last track I had copied when I organized my iTunes library by ‘Date Modified’ (check this box in View Options).

Luckily, using the freezer trick, the old drive would mount and work for a short period. I dragged the newer iTunes library files to a safe place and copied back my old ones. Then, I launched iTunes again and grabbed all the tracks I had added since the October backup – 1210 tracks/54GB. I tried to copy these to a third drive so I could then add them to my new library on my new drive. But, I was running into the same problem as before as the drive would unmount as soon as it was getting hot. Using the freezer, I could copy about 75-100 tracks at a time so this is what I did. All in all, to copy 1200 tracks took about 3 days as after each group I would have to unmount the drive and put it back in the freezer for about 30 minutes. Finally, I had copied all of the tracks.

From here, I dragged my old iTunes library files back to the desktop, copied over the newer files and added all of the 1210 tracks to my new library. With my old drive unmounted, I could now name my new drive the same name as the old drive. This is important, because now I could use the older iTunes Library file to keep all of my playlists and ratings! Since my new drive was the same name and folder structure as the old drive, all of the playlists would point to the right place!

The obvious lesson from this is to have your library backed up often. Yet, with huge libraries, this is not always easy (or possible). Still, there are a few features that Apple could implement in iTunes that could save a lot of time if the same thing happens to someone else (or me again for that matter): 1) add the ability to archive all or selected playlists. Now, you can only export/import one at a time. Thus, if your drive dies, but you have the music backed up, you can have the playlists stored somewhere else and recover them. 2) With this, there should be a preference to update the playlist files to tell where the tracks are stored. 3) add the ability to archive your entire iTunes library (similar to the ‘Archive Mailbox’ feature in Mail). Using compression, I’m sure one’s whole iTunes library, playlists, ratings, iPhone apps, etc. could be shrunk down considerably. I would totally back this up once a week if I could get it down to say 50% of the size.

Still, with iTunes (and peoples digital music collections approaching a decade), perhaps it is time for a ‘Pro’ music app á la Aperture that is built to handle large libraries, have easier support for multiple drives and even have basic editing features. Until then, I’ll just be happy I saved 6 years of playlists.