So, now I want to buy P!nk’s Greatest Hits…So Far!!!. Usually, I purchase my music at Amazon’s MP3 store, or, if I’m really feeling nostalgic, Second Spin; but iCloud presents me with a new opportunity: the idea that if I just buy my music with iTunes I will have access to it wherever I am.

My first concern is space. You see, I’ve ripped about half of my CDs so far and I’m taking up 22GB of space on my iPod with music. As time goes on, I’ll probably reach 45GB in CD music alone. I’m in no danger of hitting my upper limit today, or next year even. My iOS devices (iPod, iPhone, iPad) all have 64 GB of space. I don’t put music on the iPad, preferring to use that device for movies and books, so the space taken up by the different forms of media shouldn’t interfere with one another.

I don’t have a lot of music with iTunes. I bought one album because it was only available there. If I continue to buy MP3s via Amazon, then I will continue to take up more and more space on my iOS devices.

I’m not sure where Moore’s Law will taper off in terms of more and more storage being available in these consumer devices. I doubt this trend will continue for consumer devices because more and more people are moving their storage to the cloud. My theory is that this will lead to fewer folks buying hard drives (whether they be magnetic or flash, it doesn’t matter), thus driving the demand down and the price back up again. With the price up, Apple (or Amazon, or Samsung, or whoever makes your portable music device) will not put as much storage into your portable device. This means that I will hit an upper limit on how many MP3s I can put onto my iOS device in the future. Apple wants me to use iCloud instead, because they see this future coming. More importantly, they are helping that future come.

So, the question is, do I continue to use Amazon’s MP3 store and manually move songs onto and off of my iOS device in the future, or do I just bite the bullet now and move to iTunes altogether? iTunes has the largest collection of music out there and I can always store a few MP3s for those artists and labels that I have to buy on Amazon.

Then there is an issue with preservation. The music on iTunes is (as most of you know) not stored in the MP3 format, but AAC (with various file extensions). As of 2009, it is supposedly DRM free. I opened one of my iTunes songs in VLC player, and it played fine, without asking me for a password like the DRM-full songs do. If this is the case, and AAC players are available in the future, then my concerns about preservation may be unfounded, as I will always be able to play the songs I’ve purchased.

iTunes is big, and with that many customers, I wouldn’t be surprised if AAC stayed around for a long time, so there should be no issue with finding an AAC player if iTunes goes away.

To recap:

So, there really is no reason I can’t just jump on the iTunes bandwagon and enjoy the benefits of iCloud.

I found several articles describing how one creates the plist files used by launchd. These XML files contain the information on the process you wish to run, including when to run it, how to run it, and who is allowed to run it. Here I will detail how I used it to merely run a scheduled job every night at midnight.

To start with, I have a simple script that backs up the database on the server:

#!/bin/sh
 
BACKUPDIR=$HOME/Backups/lawrence.littleprojects.org/db
REMHOST=lawrence.littleprojects.org
OUTPUTFILE=$REMHOST.sql
 
# if we don't have a backup directory, make it
if [ ! -e $BACKUPDIR ]; then
  mkdir -p $BACKUPDIR
fi
 
cd $BACKUPDIR
 
# run the remote database dump command
ssh $REMHOST "mysqldump --verbose --user=root --password='XXXXXXXXXXXX' --all-databases" > $OUTPUTFILE
 
# if the zipped output file already exists, move it before zipping the new one
if [ -f $OUTPUTFILE.bz2 ]; then
  mv $OUTPUTFILE.bz2 old-$OUTPUTFILE.bz2
fi
bzip2 $OUTPUTFILE
 
chown smj:staff $OUTPUTFILE.bz2

This script is creatively entitled backup-database. I want it to run every night so I have an exact copy of this website’s database in case the web server goes down and I have to reinstall everything.

Under cron, I would run crontab -e and then put the following line into the editor that is brought up:

0 * * * *    $HOME/bin/backup-database

Apple made this simple step a lot more complicated than, in my opinion, it needed to be; but they gave me a tool that is a lot more powerful that mere cron.

To use launchd, I needed to make a file in the directory /Library/LaunchDaemons named org.littleprojects.backup.lawrence.db.plist that contains the following XML code:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>org.littleprojects.backup.lawrence.db</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Users/smj/bin/backup-database</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>0</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
</dict>
</plist>

I’m a little annoyed at how Apple handled the whole key-value syntax, but I’ll let that be. The file reads as follows: create a job named org.littleprojects.backup.lawrence.db that will run the program /Users/smj/bin/backup-database at the 0th hour and 0th minute of ever day (midnight). I’m not sure if this is any less difficult to read than the cron syntax, but it is more difficult to write to be sure.

Once I created the file, I then needed to load it into launchd, which could be done by running the following commands:

sudo launchctl load org.littleprojects.backup.lawrence.db.plist

If you do this you, do not need to reboot as others claim. This is Unix and reboots should only be necessary for major operating system changes. Rebooting in order to create a scheduled job is not only ridiculous, but time-wasting and leads to error-prone behavior like making many untested scheduled jobs at once in order to save on reboots. I always try to find a way around reboots to avoid error-prone behaviors!

So, I did the same thing for other scripts I wanted to run at certain times. I understand why apple decided to use launchd, I just wish they would have made it easier to configure. Lingon exists in the app store for the express purpose of helping you create these plist XML files, but it will not load them for you and instead recommends you reboot after each edit, which I noted my distaste for above.

In the future I will be exploring how to best use launchd for other purposes, like restarting services, and scheduling scripts to run when other system events occur.

References:

• http://macdevcenter.com/pub/a/mac/2005/11/15/terminal5.html?page=5
• http://www.activecollab.com/forums/topic/4342/
• http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man5/launchd.conf.5.html
• http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man5/launchd.plist.5.html
• http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man8/launchd.8.html
• http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/launchctl.1.html

The day it arrived, I set it up and spent hours pouring over Apple’s documentation. I got file sharing, SSH, WebDAV, and other services set up easily, but realized that I had to download Server Admin Tools separately in order to get access to DHCP and DNS. Some admins speculate that this is a separate download to prevent would be admins from accidentally turning on DHCP in a network that already had it. Screen sharing worked flawlessly. I had it using my 46″ television as a monitor, so watching online video and iTunes was awesome!

The next day I went back to my new toy, and… it had locked up. I was not pleased. Murmuring curses and threats of sending the server back didn’t bring the server back from its limbo. This was UNIX! UNIX was not supposed to do this! I felt betrayed by a company I’ve been growing in support for. Why did they do this to me?

I think I’ve solved the problem. Nothing useful was in the logs, but I came across a forum post where someone had suggested that there were bugs [1][2] in Lion’s display drivers that caused some Macs to lock up when returning from sleep.

I went into System Preferences, then chose Energy Saver. I slid the slider next to Display Sleep all the way to the right, choosing Never. I have not had a lockup since.

I can’t recommend this for everyone, because they don’t have a display that is either off, or using another input, like my TV, but it might work in a pinch if you don’t mind manually turning your display off. Also, I’m a little surprised that it’s affecting my brand new Mac Mini Server, because the articles I’ve found refer to older Macs.

Update: This may have been fixed by the 10.7.1 patch. I haven’t tested it yet.

Later, on my wifi network, I started the download of a book. It took 10 minutes. I’ve noticed that books in the Amazon Kindle app and iBooks generally take about 5 minutes to download, over 3G. I was quite disappointed with the Safari download speed.

Also, the first attempt was unsuccessful, for the iPad’s desire to sleep during inactivity had forced me to “babysit” the iPad, occasionally touching it to ensure that S2G didn’t terminate the download. Other iPad apps are able to override this inactivity setting so they can finish their downloads without issue, even finishing the download while the user is running other apps (you know, that whole multitasking thing that’s sweeping the nation).

Once I did get the book to download, the resulting product wasn’t fully available.

Some of the pages in the resulting book vanish when you try to read them. They’re quite the tease. It might be related to a watermark that vanishes after a picture is loaded. If you do a quick print screen you might be able to capture the missing page.

All in all, it’s not a pleasant experience.

The Safari web site works fine, but I was looking forward to the offline storage for when I didn’t have or want to use an internet connection.

At the moment S2G sucks. What the hell were they thinking releasing it like this?

Update (2011/06/08): The updates haven’t helped, crashing often and appearing to fix none of these issues. In fact, the latest version doesn’t even load, presenting the user with a black screen and no functionality.

Perhaps Web Science and Digital Libraries is really my thing. I spend an awful lot of time blogging about my own web sites.

Validation of the Creative Commons Markup

I also discovered that the Creative Commons license markup wouldn’t validate with the W3C validator. This was because my page was initially specified as XHTML, but the Creative Commons license link contains additional RDF metadata used by search engines and other software. A blog post suggested just removing the metadata, but I didn’t want to do that because I knew that metadata had been placed there by Creative Commons for a reason. That metadata is part of the RDFa standard, which is a way of embedding metadata into normal HTML and XHMTL.

It turns out that once you put that Creative Commons markup into your XHTML, it becomes XHTML + RDFa, which is a completely different document type. I felt good that I’d figured this out, and changed the page to use this instead.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN" "http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

Now it validates fine as XHTML + RDFa.

Nifty floating image trick

You may wonder how I got the image to stay on the left and in the same spot, even though the visitor can scroll the page.  It is based on the following CSS, shamelessly stolen from the W3C themselves.

body {
    background-image: url(http://www.cs.odu.edu/~sjone/images/oducs-lpx.png);
    background-repeat: no-repeat;
    background-position: top left;
    background-attachment: fixed;
   /* more CSS here */
}

That’s it. It’s quite elegant, and I spent more time making the image with Inkscape than I did actually writing the CSS to tell it to stay put.

Useful Links

I discovered the following useful sources for bringing that site to life:

• FOAF Vocabulary Specification
• W3Schools CSS Tutorial
• Learn CSS Positioning in Ten Steps
• RDFa Primer
• RDFa in XHTML: Syntax and Processing
• The W3C CSS Validation Service
• W3Schools HTML 4.01/XHTML 1.0 Reference
• The W3C Markup Validation Service
• Expressing Dublin Core in HTML/XHTML meta and link elements

Early in January I started to see applications crash.  VMWare Fusion became almost unusable.  I had never seen this behavior from my Mac before, so figured the new RAM had something to do with it, but wanted to verify my assumption before blindly blaming Crucial for bad RAM.  At that point I had to learn how to best test RAM on a Mac.

On a PC, I would have booted a CD running memtest86, but none of my Internet research indicated that was possible for the Mac.

Instead, I had to install memtest and boot the machine into single user mode in order to run memtest with the least amount of interference from running programs. I wanted to document what I had done in a blog post so I could remember next time.

Note: This article involves configuring your Mac in a way that could make it difficult for you to boot it normally if you don’t know what you’re doing. If you don’t feel comfortable following these instructions, take your Mac to a technician who not only feels comfortable fixing your computer, but is also competent to do so.

To tell the Mac to boot into verbose 64-bit single user mode, go to Applications/Utilities/Terminal, and type:

sudo nvram boot-args="-s arch=x86_64 -v"

You may be asked for the password you type to make changes to the system, type the password and hit enter.

Reboot your Mac.

Your Mac will boot to a prompt.  You may be surprised at all of the text that goes by.   These are kernel messages that are normally hidden by the nifty Apple logo.

At the prompt type:

memtest all

This will attempt to test as much RAM as it can.  Remember that some of your RAM is being taken up by minimal amount of software that booted your machine into single user mode (the kernel and bash), so you won’t be able to test everything, but if the flaw was in the RAM being used by the software running at this point, the machine would have failed to boot.

If your RAM is good, the test will take hours and you will see something like:

Memtest version 4.22 (64-bit)
Copyright (C) 2004 Charles Cazabon
Copyright (C) 2004-2008 Tony Scaminaci (Macintosh port)
Licensed under the GNU General Public License version 2 only
 
Mac OS X 10.6.6 (10J567) running in single user mode
Memory Page Size:  4096
System has 2 Intel core(s) with SSE
Requested memory: 7555MB (7922552832 bytes)
Available memory: 7555MB (7922552832 bytes)
NOTE:  Memory request is too large, reducing to acceptable value...
Allocated memory:  7343MB (7700721344 bytes) at local address 0x00000000101000000
Attempting memory lock... locked successfully
Partitioning memory into 2 comparison buffers...
Buffer A: 3671MB (3850360672 bytes) starts at local address 0x0000000101000000
Buffer B: 3671 MB (3850360672 bytes) starts at local address 0x00000001e67fd760
 
Running 1 test sequence... (CTRL-C to quit)
 
Test sequence 1 of 1:
 
Running tests on full 7343MB region...
  Stuck Address       : ok
  Linear PRN            : ok
Running comparison tests using 3671MB buffer...
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV          : ok
  Compare OR           : ok
  Compare AND         : ok
  Sequential Increment:ok
  Solid Bits               :ok
  Block Sequential      :ok
  Checkerboard         :ok
  Bit Spread             :ok
  Bit Flip                  :ok
  Walking Ones         :ok
  Walking Zeroes       :ok
 
All tests passed!  Execution time:  7068 seconds.

If your RAM is bad, you will likely get a response really fast, and you will see something like:

*** Address Test Failed *** One or more DIMM address lines are non-functional.

or

FAILURE: possible bad address line at offset 0x06b3a4c8.

or

FAILURE: 0xbea1ce76 != 0xbea1ce7e at offset 0x06b3a4c8.

or

*** Memory Test Failed *** Please check transcript for details.

If you get a message like:

Attempting memory lock... ERROR: Memory lock failed - reason unknown.

WARNING: Testing with unlocked memory may be slower and less reliable

hit CTRL-C and make sure you booted into 64-bit mode by typing the following:

uname -m

If the command doesn’t return x86_64 then you are not running in 64-bit mode and your Mac can’t reach the RAM above 4GB. Please reference the above command for booting your Mac into single user mode, as it includes the flags to set it to 64-bit mode, then reboot.

If the command above does return x86_64, then you are running in 64-bit mode and this message is probably a preview of test failures to come.

If you have more than one module of RAM, then take all of them out, and place one in the machine, boot, and run the test. After that one finishes test, note the results, turn the Mac off, take the RAM module out, and put the next one in. This way you can figure out which module is bad.

When done, make sure you have good RAM in the machine, boot it again, and type:

sudo nvram boot-args=""

This will give you your “normal” Apple logo screen on boot.

I hope you will find this article as useful as I will the next time we need to test RAM on a Mac.

Today, Crucial was nice enough to send me some new RAM after I sent them the bad module.  So far, it appears to be passing memtest.  Cross your fingers for me! 

An important take-away from tonight’s lecture was that merely placing “© 2010 Shawn M. Jones” at the bottom of my pages is not merely enough to protect it legally.  A notice must appear somewhere on the site for its content.  Several years ago, Lawrence Lessig created the Creative Commons license to allow those on the Internet to share their work while retaining their copyright to said work.

To provide a license for this blog, I’ve filled out the form on the site and followed the directions on the web site for linking to the chosen license, like so:

the Little Projects of Shawn M. Jones by Shawn M. Jones is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

We’re quite lucky to live in an era where folks are willing to do the legal legwork necessary to make this happen. I’m happy that it’s even held up in court.

Within a traditional library, the information about a book (metadata) is stored within some system (e.g. MARC), and this system is linked to some library classification (e.g. Dewey Decimal) for finding it on the shelf at a particular library. A whole profession exists for making this happen. Book metadata is chosen by professionals so that said book can be delivered to the person looking for the information within. These professionals (catalogers) are the gatekeepers of the whole system. Without them, the books might as well be strewn about.

On the web, there is no central authority. Every site is responsible for its own content. Search engines like Google use complex algorithms to try to find something that answers your question. Web site owners must take it upon themselves to not only ensure that their site stays consistent and correct, but also that it has metadata for these search engines to use to find them. Though Search Engine Optimization (SEO) is largely used to ensure potential customers find businesses, it is also important in helping users find the information they are looking for.

Now that I am aware of the importance of such metadata, I have installed a plugin for WordPress on my blog that generates Dublin Core metadata elements. These metadata elements are supposed to help others find articles like this one via search engines.

This plugin takes the existing metadata I was already supplying for each post, and places it in the header of the HTML at the top of the page, like so:

<meta name="DC.publisher" content="the Little Projects of Shawn M. Jones" />
<meta name="DC.publisher.url" content="http://www.littleprojects.org/blog/" />
<meta name="DC.title" content="Finding a kernel on the web" />
<meta name="DC.identifier" content="http://www.littleprojects.org/blog/?p=188" />
<meta name="DC.date.created" scheme="WTN8601" content="2011-01-19T23:52:11" />
<meta name="DC.created" scheme="WTN8601" content="2011-01-19T23:52:11" />
<meta name="DC.date" scheme="WTN8601" content="2011-01-19T23:52:11" />
<meta name="DC.creator.name" content="Shawn M. Jones" />
<meta name="DC.creator" content="Shawn M. Jones" />
<meta name="DC.rights.rightsHolder" content="Shawn M. Jones" />
<meta name="DC.language" content="en-US" scheme="rfc1766" />

I don’t really expect the search engine rankings to go up, but the real win here is that I’m helping others index my site in case I’ve actually provided exactly the information someone is looking for.  In a way, this is a form of SEO, but it gets back to that cataloging spirit originally found in the library.  There is no common list of tags or subjects for the web that we all must adhere to, but little steps like this bring us closer to finding the information we are looking for.

Take a look at the source for some of your favorite news sites, you’ll probably see the same metadata in their headers too.

For futher reading:

Metadata: The Foundations of Resource Description

If your .bashrc looks like this:

#!/bin/bash
echo "Running the .bashrc file"
 
# correct spelling mistakes
shopt -s cdspell
 
# save multiline commands in the command history
shopt -s cmdhist
 
# some useful aliases
alias c='clear'
alias ls='ls -p'
alias vi='vim'
 
echo "done with .bashrc file"

And you connect via SSH, you’ll see the following:

[me@otherhost]$ ssh myhost
me@myhosts's password:
Running the .bashrc file
done with .bashrc file
[me@myhost me]$

Which is fine for an SSH session.  In fact, you might even ask questions of logging in users, or dump an entire warning banner to the screen to indemnify you for legal reasons.  Many interactive possibilities exist.

Now say you wish to copy a file from one server to another:

[me@otherhost]$ scp myfile myhost:~/
Running the .bashrc file

Once this is done, you only see the output from the first echo.  What’s worse, the copy never happens.  Some claim this is a bug.  The fact that the bug was reported in 2000 and still exists in scp indicates to me that the scp authors don’t consider it a problem.

The solution is in testing for the existence of a terminal:

#!/bin/bash
if tty > /dev/null 2>&1; then
    echo "Running the .bashrc file"
fi
 
# correct spelling mistakes
shopt -s cdspell
 
# save multiline commands in the command history
shopt -s cmdhist
 
# some useful aliases
alias c='clear'
alias ls='ls -p'
alias vi='vim'
 
if tty > /dev/null 2>&1; then
    echo "done with .bashrc file"
fi

Now scp will work without any output to the screen. More importantly, your files will be copied:

[me@otherhost]$ scp myfile myhost:~/
me@myhost's password:
myfile                               100% 1896KB   1.9MB/s   00:00

Thanks to the forums on Ars Technica for leading me to the answer to this issue. As that post existed in 2000, I now see why the openssh folks haven’t “fixed” scp.

I wanted the site to do the following things:

• allow me to publish blog posts, which I was already doing with WordPress
• allow me to publish the occasional article, which I seem to have little time or inclination to do, so I ditched Drupal
• allow me to publish photos, for which I’m currently using Picasa Web Albums

I’m maintaining my own site on my own server for the following reasons:

• Blogger was too slow to load on many of the networks I experimented with
• WordPress has more features than Blogger.
• WordPress is open source, and I can learn about this nifty piece of software by running it myself.
• I had already moved my mail services off of Gmail and onto my rented server, and wanted to consolidate my web services there as well.
• I love messing with a Linux server in my spare time.

We’ll see how far this Intel Celeron 2.53 GHz machine with 1GB of RAM will get me.