Oh my. I had no idea about what was running my music. Now I know quite a bit more, and I thought I would share. My collection is very convoluted, but I didn’t know it. No, I had no idea.

How did I get to the messy state I’m at today? I started converting my CDs to MP3s back in 2000. I got a fast IDE controller, 4 Maxtor 80 Gig drives, and built up a file server on Red Hat with RAID 5. The players at the time (it turns out) just went by song name order. Back in 2000 and 2001, ripping your CDs to MP3s was mostly command line, and the results varied. I don’t believe it created any kind of tags in the file at all. I had a Windows 98 laptop with Winamp on it that connected to my Samba share and an FM transmitter to broadcast to the stereo, and this worked until I moved in 2007. For a time, between 2003 and 2006, I had an MP3 ripper that ran on my XP workstation, and I converted much of my CD collection with this (I don’t remember the program). In my car or truck I had an MP3 player that used a CD, so I could fit 20 or so albums at a time. Now, around 2006 I got a MacBook. I used iTunes to rip the CDs. In 2007 I got a Logitech SlimServer (now Logitech Media Server) to replace the laptop console. The server piece is a perl script, and I’ve got the server to run on Windows, Mac OS X, and several GNU/Linux distributions. I still use it. The SlimServer also goes by song order, and so everything worked fine. Then I had my first frustrations…

My wife bought me an iPod one year, 2008, I believe. It was great, I loved the device, but iTunes completely screwed up my songs. I bought various utilities that cleaned them up. They downloaded album covers and “fixed” the files. I figured it wasn’t hurting anything, but it never completely worked right. The SlimServer continued to work fine, but it was going by song name order. I’ve always had numbers in front of my songs. The bad thing is that I ripped my favorite bands first. So, all of my old music didn’t play correctly. Some of the rippers I used early on created mp3 files that couldn’t be identified. I always run a Windows workstation at my job, and Windows Media Player started to get the song order wrong. Also, I have a player in my car that has an SD card. I thought it would be so cool to have lots of cards. 2 gigs on an SD card is a pretty decent size. I can fit all of Joni Mitchell, The Moody Blues, and Kate Bush I have on 1 2 gig card (and I have most of their albums), but none of the song orders were right except for a few albums that I bought from Amazon MP3 (great service). Also, for work, I can now fit everything I own on one iPod, but, again, the songs don’t play in order. And as a final horrible thing, my wife makes beautiful mix sets that have album art, but I can’t listen to them in my car. Not only weren’t the songs in order, but they skipped around. My world is falling apart.

A couple weeks ago I decided that I would fix this problem once and for all. I was going to make my MOODK8JONI SD card (10 character volume name), and it would play everything in order. Further, it would play on iTunes or Windows Media. Here is how I did it.

First off, regarding the skipping. There is something wrong with my car player where it needs the FAT file system to have sequential blocks or something. There are two things you need to do to fix this. First, you have to really format the file system. It seems that if I just format as ext3 and then format as vfat, it works fine as long as I use a sync command after every file copy. It isn’t supposed to matter, but it does. I’m using Ubuntu 11.10. I can plug my iPod into it, and my SlimServer (server) runs on it too. So, the technical details are using a GNU/Linux perspective. I converted my Macs to Ubuntu when I realized what Apple was doing with Lion. I want my data.

The next problem is the iPod. Well, the iPod uses ID3. ID3v2 is just some tags tacked to the top of an MP3 file, and ID3v1 tacks it to the bottom. I was able to fix up my files to full ID3v2 using MusicBrainz Picard. I can transfer to my iPod via Amarok. If I want to transfer to my iPhone, this is possible, but the easiest is to boot into Snow Leopard and import into iTunes. I can’t get Amarok to transfer correctly to my phone, and I don’t care that much. (Yes, I am surrounded by Apple devices, and refuse to use Mac OS X as my main OS any more. This isn’t place for a long philosophical rant about cloud and data and lock-in, but I could go on and on. I was fine until Lion… and I even started using the web publishing in me.com, and they pulled that out from under me.)

When I tried to use my shiny new ID3v2 files, they still wouldn’t play in order on my car player. I so much wanted to use my SD cards. It is so perfect. Alas. I tried EasyTAG, and it would strip out the ID3v2 tags and leave ID3v1 tags, which the car radio manufacturer said would work. It didn’t. It turned out that the only thing that worked was the file write time. It may be that it uses ID3v1 tags for artist name or something like that, but it doesn’t use the tags for song order. Oh, and the player adds folders to a freshly formatted card, so you can’t add files without the filesystem being slightly fragmented, and I believe this causes some skipping. Now, dear reader, my struggle at this point is likely not yours. But, some of you probably have the older players. Regardless, some of the ideas might help.

So, at this point, I need to create a volume, format the filesystem twice, and copy files in order of filename one at a type and sync between each file copy. Well, I did futz around a bit looking for some combinations of *NIX tiny, sharp tools, but particularly because of the nasty reserved characters in the directories and files, I ended up writing a perl script that takes a specially formatted file and spits out a shell script. This gives me the ability to hand-tweak it a bit and exclude greatest hits albums and single songs if I want. Here is my final system. The core is this perl script:

print ("umount /dev/mmcblk0p1\n");
print ("mkfs.ext3 /dev/mmcblk0p1\n");
print ("mkfs.vfat /dev/mmcblk0p1\n");
while (<>){
	if (/^label(.+)$/){
		$volname=$1;
		print("mlabel -i /dev/mmcblk0p1 ::$1\n");
		print("umount /dev/mmcblk0p1\n");
		print("mount /dev/mmcblk0p1 /sdcard/\n");
	}
	if (/^artist(.+)$/){
		$artist=$1;
		print("mkdir '/sdcard/$1'\n");
	}
	if (/^rootpath(.+)$/){
		$rootpath=$1;
	}
	if (/^album(.+)$/){
		$album=$1;
		@files=();
		opendir (DIR, "$rootpath/$album");	
		while ($file = readdir(DIR)) {
    			push (@files, $file);
		}
		@files = sort {$a cmp $b} @files;
		foreach $file (@files) {
			if ($file=~/mp3$/){
    			print('mkdir "/sdcard/'.$artist.'/'.$album.'/" -p');
			print "\n";
    			print('cp "'.$rootpath.'/'.$album.'/'.$file.'" "/sdcard/'.$artist.'/'.$album.'/"');
			print "\n";
			print "sync\n";
			print "sync\n";
			}
		}
		close DIR;

	}
}
print ("umount /dev/mmcblk0p1\n");

If you wonder why I run sync twice, it is in honor of Dennis Ritchie:

From: Dennis Ritchie (dmr_AT_bell-labs.com)
Subject: Re: why sync twice?" (was Re: PDP-11 emulation and Unix v7)
Newsgroups: alt.folklore.computers
Date: 1999/05/12
markh_AT_usai.asiainfo.com (and previous askers of the question) wrote:
> 
> Sarr J. Blumson  wrote:
> : Just run "sync" twice and shut if off IIRC.  fsck would run when you rebooted,
> : but it wouldn't find any problems and the disks were small...
> 
> I was always curious why the unix manuals would always recomment
>         "sync; sync; halt"
> 
> My mentor of sorts (this was in 1984) had worked on the Nixdorf
> unix port and said it was unnecessary.  "If you don't trust it
> the first time, what makes you think it's going to work the
> second time?"
> 
> Anything I missed?
If you actually typed "sync; sync; halt" then I can see the possible
timing issue.  However, if you were told to type "sync" and then do it
again, this had to do with giving you a chance to ponder whether
you remembered to do it at all the first time.
Dennis

The data file looks like this:

labelMOODK8JONI
artistThe Moody Blues
rootpath/home/myhome/media/music/mp3/Moody Blues
albumMoody Blues - A Question of Balance
albumThe Moody Blues - Days Of Future Passed [Remaster]
.
.
.
artistJoni Mitchell
rootpath/home/myhome/media/music/mp3/Joni Mitchell
albumBlue
albumCourt and Spark
albumHejira

I can create an SD card that works in my car with this:

perl create.pl < MOODK8JONI > MOODK8JONI.sh

I run this on the machine that serves up my SlimServer. Since I have proper ID3v2 tags now, I can play the songs on any iPod, and it works in the car. Finally, I can recreate the card repeatedly and expand easily without having to go through a complicated sequence of commands. Note that this is specific to Ubuntu 11.10 running on my Mac mini. The label only works on FAT 16 (type 6) partitions, not FAT 32. Anyway… I hope this is helpful.