Cleaning up iPhoto

The longer I use iPhoto, the more I hate it. When I initially started using it, I would import photos but choose not to copy the originals to the iPhoto Library. In the past year or so, I've started to just allow iPhoto to copy the originals to the iPhoto Library. So now, I ended up with photos scattered all over the place. This was a pain to maintain and figure out where my photos were. I decided that I want all photos to be in one place and if I was going to use iPhoto, I'm going to import by copying the originals to the iPhoto Library. Before I could get all of this to happen, I had to backup all of existing photos that were not in the iPhoto Library to an external hard drive. Planning on importing these photos over again from the external hard drive, I then deleted all of these photos through Finder. Going back into iPhoto, I still saw the thumbnails for the photos I just deleted. If I attempt to open any of them though, it complained it couldn't find the original. What a mess. There doesn't seem to be a way to refresh your iPhoto Library and it would remove photos it doesn't have references to any more. I figure the easiest way to get my iPhoto Library setup the way I want it is to start from scratch. So I went under my Pictures directory and renamed iPhoto Library to iPhoto Library.original. Opening up iPhoto again, you get prompted to search for your Library or create a new one. I choose create a new iPhoto Library and now I can begin importing my photos.

After making sure I have the option to copy items to iPhoto Library when importing, I can now import photos from my external hard drive. Great! Now I have all of my scattered photos in iPhoto Library. But what about the photos I had imported to the original iPhoto Library ( the ones under iPhoto Library.original)? Well, there didn't seem to be an easy way to import non-broken originals form one iPhoto Library to another from the GUI so I went to the command line. All of the original photos that are imported to an iPhoto Library are under a folder called Originals. However, due to the way iPhoto manages the photos, this also includes the broken files that reference the photos I had deleted. Basically I wanted to get rid of all of these broken references before importing all of the photos from my original iPhoto Library to the new one I am creating. Here's how I did it.
  1. Open up Terminal
  2. Change directory to the original iPhoto Library's Originals directory (~/Pictures/iPhoto Library.original/Originals)
  3. The broken references aren't actually symbolic links. They seem to be using extended file attributes to denote where the original file actually is (see xattr).  Since we deleted the actual photos, to identify these files type: 
    find . -size 0
    to get a list of all of the 0 byte files.
  4. To remove them: 
    find . -name "*.jpg" -size 0 -exec rm {}  \;
    or if you want to just move them elsewhere: 
    find . -name "*.jpg" -size 0 -exec mv {} $dest \;
    where $dest is the path to where you want to move the files
  5. To delete all of the empty directories (that represent your events) for clean up purposes: 
    find . -depth -type d -empty -exec rmdir {} \;
Now your original iPhoto Library should only contain photos that were imported by copying the originals to the iPhoto Library. To finish importing everything, open iPhoto, choosing your new iPhoto Library and import the Originals directory from the original iPhoto Library (~/Pictures/iPhoto Library.originals/Originals). You might have to copy this folder someplace else since the Import menu doesn't allow you to specify going into the iPhoto Library.originals package. After about 5 hours of battling with iPhoto, I think I finally have all of my photos reimported to a fresh iPhoto Library with no duplicates and broken file references. Having to do all of this really makes me think to just switch to something else. How do you guys feel about iPhoto? What are some good alternatives? Picasa anyone?

Moving wordpress to another host

If you've been following my blog for awhile, you might have noticed I moved this blog from blog.notedpath.com to theodorenguyen-cao.com as it was more fitting domain. I originally just registered the domain, added the DNS record, and updated my apache config to have theodorenguyen-cao.com to be an server alias to blog.notedpath.com. This allowed requests to blog.notedpath.com/* and theodorenguyen-cao/* respond with the same content. I thought I was done. I discovered this wasn't the case when I saw blog.notedpath.com as a direct traffic source in my google analytics for theodorenguyen-cao.com. To fix the screwed up analytics, I needed to make it so that all requests that go to blog.notedpath.com are permanently redirected (301) to theodorenguyen-cao.com. To do this I had to apply an Apache mod_alias redirect directive as such: 
<VirtualHost *:80>
        VirtualDocumentRoot /var/www/blog
        ServerName blog.notedpath.com
        Redirect permanent / http://www.theodorenguyen-cao.com/
        ErrorLog /var/log/apache2/wp-error.log
        TransferLog /var/log/apache2/wp-access.log
</VirtualHost>
The virtual host for theodorenguyen-cao.com looks like: 
<VirtualHost *:80>
    VirtualDocumentRoot /var/www/blog
    ServerName www.theodorenguyen-cao.com
    ServerAlias theodorenguyen-cao.com
    CustomLog /var/log/apache2/theodorenguyen-cao.com_access.log Combined
    ErrorLog /var/log/apache2/theodorenguyen-cao_error.log
</VirtualHost>
At first I thought this would only fix the simple case of blog.notedpath.com redirecting to theodorenguyen-cao.com, but blog.notepath.com/foobar not being translated to theodorenguyen-cao.com/foobar. However, this does exactly what I want. All blog.notedpath.com URLs will be replaced with theodorenguyen-cao.com URLs. Old bookmarks will simply redirect to a theodorenguyen-cao.com URL and not 404. Success! I'm still waiting to see if Google will update the search result links that point to blog.notedpath.com to be theodorenguyen-cao.com URLs.

Using validates_presence_of on a boolean field? Should use validates_inclusion_of!

rsvp.theoandpat.com had boolean flag to marked whether or not a visitor was going to be able to make it to our wedding. Unfortunately, if you selected you were not able to make it and submit the form, the application would return saying it could not process your submission because you have to say that you are going to make it. I argued, this is an RSVP form so you have to accept if you are RSVPing. That's the point of the RSVP! Only people RSVP would bother submitting the form!  Pat wasn't too happy about that and ask/told me to fix it.  Digging into it, it turns out the way  for validates_presence_of relies on Object#blank which of course when sent 
false.blank? # returns true
Reading up on the documentation, it is suggested to use validates_inclusion_of when dealing with booleans. The one line change solved the problem: 
validates_inclusion_of :accepted, :in => [true, false]

Picasa vs Flickr

Pat got me a new camera for my birthday this past October so I have been trying to take more pictures lately. One of my new years resolutions is to take more pictures! As a result, I've been trying out Yahoo's Flickr and Google's Picasa over the past couple of weeks. With Picasa recently added support on the Mac and Apple announcing iPhoto out-of-the-box Flickr integration, the feature sets from the desktop are pretty much identical. Both support (or will support) face recognition, tagging, and exporting. For the web galleries, I really liked Picasa's layout over Flickr's. The web site is a lot more simplistic and easy to navigate. Maybe it's becase I'm so familiar with other Google services. Flickr's slideshow is so awesome though. Viewing your photostream through Cooliris is such beautiful eye candy.

I started with the free accounts. Google gives you a gig of space for the free account and you can create as many albums as you want. Flickr's free account gives you 100 mb of upload per month and limits you to three sets. For me, it really came down to pricing. In order to get more disk space for Picasa, you have to purchase more storage. Starting purchase goes for $20/yr for 10gigs up to $500/yr for 400GB. This storage is actually not specific to Picasa but is shared by all other Google services that need to use extra space. Flickr offers a Pro account, which gives you unlimited storage space for your videos and photos and allows you to create any number of sets and collections, for $24.95. Given the number of photos I want to upload and my budget, I went with Flickr. Today, storage is so cheap. For $25 bucks, I should be able to get unlimited storage for a service like photo sharing. I've started to upload some photos I've had for awhile and never really shared them with those who would most enjoy them. It's been really fun going through old pictures. My photostream can be found here.

tiny URLs? HA! diminutive URLs for the win!

Inspired by a discussion of URL shortening, I took a weekend and implemented one of my own.  When thinking about tiny URLs, a quote always came to mind.
Don't use a big word where a diminutive one will suffice.
So after finding out the domain was available, diminutiveurl.com was born. Yes, it's poking a little fun at the idea of a tiny url but it was fun to hack on.  It's very minimilistic at this point but I hope to add some interesting features.

For no other reason than to build something, I hope you enjoy it! I am glad to present diminitiveurl.com! Please let me know what you think. .

Introducing theoandpat.com

Over the course of the past months in her free time between work and planning a wedding, Pat's been hard at work at creating a web site for our wedding. We felt we had enough to show it off to the rest o the world so I finally got around to deploying it.  I'm really happy with the way it turned out. I think she did a pretty awesome job.  I contributed in a small way by setting up the web server, create the RSVP site, and installing the guestbook.  The whole concept and design is Pat's creation.  She did all of the page layout and flash work.  I'm so proud of her. She's turning into a web geek and she doesn't even know it.

I'm sure there are some kinks to iron out but please check out theoandpat.com and let us know what you think.  Oh, yeah also, RSVP for the wedding. If you're reading my blog, chances are I know you and you're invited. :) -Theo

Reduce MySQL memory usage

As I am running more and more on my 256mb slice, I'm trying to squeeze more performance out of the system. After a bit of digging, I've made a few changes. First thing I did was switch out the default mysql config with one tweaked for smaller boxes. This configuration actually came with the mysql installation under /usr/share/doc/mysql-server-5.0/examples/. Just backup your existing my.cnf (mine is located under /etc/mysql) and use the following: 
# The following options will be passed to all MySQL clients
[client]
port                = 3306
socket                = /var/run/mysqld/mysqld.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port                = 3306
socket                = /var/run/mysqld/mysqld.sock
skip-locking
key_buffer = 16K
max_allowed_packet = 1M
table_cache = 4
sort_buffer_size = 64K
read_buffer_size = 256K
read_rnd_buffer_size = 256K
net_buffer_length = 2K
thread_stack = 64K

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (using the "enable-named-pipe" option) will render mysqld useless!
# 
#skip-networking
server-id        = 1

# Uncomment the following if you want to log updates
#log-bin=mysql-bin

# Uncomment the following if you are NOT using BDB tables
#skip-bdb

# Uncomment the following if you are using InnoDB tables
#innodb_data_home_dir = /var/lib/mysql/
#innodb_data_file_path = ibdata1:10M:autoextend
#innodb_log_group_home_dir = /var/lib/mysql/
#innodb_log_arch_dir = /var/lib/mysql/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 16M
#innodb_additional_mem_pool_size = 2M
# Set .._log_file_size to 25 % of buffer pool size
#innodb_log_file_size = 5M
#innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 1
#innodb_lock_wait_timeout = 50

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 8M
sort_buffer_size = 8M

[myisamchk]
key_buffer = 8M
sort_buffer_size = 8M

[mysqlhotcopy]
interactive-timeout
You can see estimated memory usage with a config if you just copy paste the configuration file here. Restart MySQL (/etc/init.d/mysql restart). The second thing I did was check out MySQLTuner. Just download the perl script and run it. It will run some analysis on your mysql setup and provide some performance and configuration tweak recommendations. Sample output looks like this: 
>>  MySQLTuner 1.0.0 - Major Hayden <major@mhtx.net>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering
Please enter your MySQL administrative login: root
Please enter your MySQL administrative password: 

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.0.51a-3ubuntu5.4
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: +Archive -BDB -Federated +InnoDB -ISAM -NDBCluster 
[--] Data in MyISAM tables: 867K (Tables: 18)
[--] Data in InnoDB tables: 704K (Tables: 40)
[!!] Total fragmented tables: 2

-------- Performance Metrics -------------------------------------------------
[--] Up for: 14h 57m 0s (17K q [0.316 qps], 699 conn, TX: 55M, RX: 3M)
[--] Reads / Writes: 96% / 4%
[--] Total buffers: 26.0M global + 824.0K per thread (100 max threads)
[OK] Maximum possible memory usage: 106.5M (41% of installed RAM)
[OK] Slow queries: 0% (0/17K)
[OK] Highest usage of available connections: 6% (6/100)
[!!] Key buffer size / total MyISAM indexes: 16.0K/381.0K
[!!] Key buffer hit rate: 88.5% (154K cached / 17K reads)
[!!] Query cache is disabled
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 4K sorts)
[!!] Temporary tables created on disk: 30% (1K on disk / 4K total)
[!!] Thread cache is disabled
[!!] Table cache hit rate: 0% (4 open / 7K opened)
[OK] Open file limit used: 0% (8/1K)
[OK] Table locks acquired immediately: 99% (17K immediate / 17K locks)
[OK] InnoDB data size / buffer pool: 704.0K/8.0M

-------- Recommendations -----------------------------------------------------
General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Enable the slow query log to troubleshoot bad queries
    When making adjustments, make tmp_table_size/max_heap_table_size equal
    Reduce your SELECT DISTINCT queries without LIMIT clauses
    Set thread_cache_size to 4 as a starting value
    Increase table_cache gradually to avoid file descriptor limits
Variables to adjust:
    key_buffer_size (> 381.0K)
    query_cache_size (>= 8M)
    tmp_table_size (> 32M)
    max_heap_table_size (> 16M)
    thread_cache_size (start at 4)
    table_cache (> 4)
I haven't made any of the recommended changes yet though. I am going to try to see how this default small config performs. I'll try to follow up with any of my findings.

Rails initial request slow with mod_rails

Following up on my previous post , I've been experience slow load times (10-15 secs) on the initial request after application restarts. This has to do with the way mod_rails manages application instances (Although, I experienced this when using mongrel_cluster and proxy balancers). It will spin up instances on page request and each instance has an idle timeout. This just means after the timeout expires, mod_rails will shutdown that instance to conserve memory allocation. While you can change timeout value (see PassengerPoolIdleTime), this will only cause all instances that get spin up to live longer. After high load times, these instances will stick around longer than neccessary. For low traffic sites (like mine), this idle timeout may be reached causing the next visitor to our website to experience a really long delay before page load. What we really want is an option to set a minimum number of instances. This would allow us to automatically spin up an instance during start up and keep it around. Unfortunately at this time, it doesn't look like there is a way to set this. As a workaround, I've setup a crontab that makes a request to my application every 5 minutes to prevent mod_rails from killing off all application instances. To do this just run: 
crontab -e
And then specific the following cron 
*/5 * * * * wget -O /dev/null http://www.myapp.com 2>/dev/null
You can verify this is working correctly but just tailing your application logs and verify every 5 minutes you get a request. You can also run 
passenger-status
You should see at least the count variable to be at least 1 instance. Note, this workaround will not immediate start up an instance upon restart. You can add an initial request as part of your post deploy capistrano task though. You probably should be making sure your application is up after deploying or restarting the application anyways.

Setting up Phusion Passenger (mod_rails) with Capistrano support

I had heard of mod_rails awhile back but never had the time to take a closer look at it.  While setting up a new rails app, I was getting frustrated with all of the configuration I needed to do to get the mongrel clusters and proxy balancers setup.  So I decided to give passenger a chance.  I'm a fan now :) The process was dead simple.
  1. Install the passenger gem 
    sudo gem install passenger
  2. Install passenger as an Apache module 
    passenger-install-apache2-module
  3. Load the passenger apache module by editing the Apache config 
    LoadModule passenger_module /usr/lib/ruby/gems/1.8/gems/passenger-2.0.5/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.8/gems/passenger-2.0.5
PassengerRuby /usr/bin/ruby1.8
  4. Restart Apache
If all things went well, you have everything installed you need. If there were some missing dependencies, you should be presented with how to install those dependencies. In the installation output, it tells you how to mod_railsify your apps by creating a vhost as such: 
<VirtualHost *:80>
    ServerName www.mywebsite.com
    DocumentRoot /home/deploy/mywebsite/public
</VirtualHost>
That's it! No more of this proxy balancer and mongrel_cluster.yml configuration. There's some magic going on in the background. As requests come in, passenger will spin up more application instances. For more tweaking your configuration options check out the user guide. Go to your website and you should see your rails app up and running. So now we have your app up and running, how do we update or restart our app? Passenger provides two ways for us to do this. The first is whenever apache is restarted, your application is restarted. The second way allows us to restart a specified application without affecting Apache. Whenever passenger detects tmp/restart.txt, it will restart the application instances for us. We can integrate this into our Capistrano deploy flow by adding the following our config/deploy.rb 
namespace :passenger do
  desc "Restart Application"
  task :restart do
    run "touch #{current_path}/tmp/restart.txt"
  end
end

after :deploy, "passenger:restart"
This will create that restart.txt after the cap:deploy task gets executed, causing the application to restart. Finally, passenger comes with some pretty useful utilities. Check out passenger-status which produces output showing current passenger server statuses. Sample output: 
----------- General information -----------
max      = 6
count    = 1
active   = 0
inactive = 1
Using global queue: no
Waiting on global queue: 0

----------- Applications -----------
/home/deploy/www.myapp.com/releases/20081206183156: 
  PID: 30784     Sessions: 0
Another utility passenger-memory-status gives you insight into how much memory is being used by apache and passenger. Sample output: 
-------------- Apache processes ---------------
PID    PPID   Threads  VMSize    Private  Name
-----------------------------------------------
12841  1      1        225.9 MB  0.0 MB   /usr/sbin/apache2 -k start
28294  12841  1        248.4 MB  21.4 MB  /usr/sbin/apache2 -k start
28300  12841  1        243.7 MB  0.5 MB   /usr/sbin/apache2 -k start
28306  12841  1        248.4 MB  4.4 MB   /usr/sbin/apache2 -k start
28357  12841  1        249.1 MB  19.8 MB  /usr/sbin/apache2 -k start
29400  12841  1        249.4 MB  3.7 MB   /usr/sbin/apache2 -k start
29788  12841  1        249.3 MB  21.7 MB  /usr/sbin/apache2 -k start
29834  12841  1        245.8 MB  18.9 MB  /usr/sbin/apache2 -k start
29836  12841  1        245.8 MB  9.3 MB   /usr/sbin/apache2 -k start
29868  12841  1        245.8 MB  2.4 MB   /usr/sbin/apache2 -k start
29870  12841  1        246.5 MB  5.2 MB   /usr/sbin/apache2 -k start
### Processes: 11
### Total private dirty RSS: 107.44 MB

--------- Passenger processes ----------
PID    Threads  VMSize    Private  Name
----------------------------------------
28031  10       15.3 MB   0.1 MB   /usr/lib/ruby/gems/1.8/gems/passenger-2.0.5/ext/apache2/ApplicationPoolServerExecutable 0 /usr/lib/ruby/gems/1.8/gems/passenger-2.0.5/bin/passenger-spawn-server  /usr/bin/ruby1.8  /tmp/passenger_status.12841.fifo
28032  2        48.7 MB   0.6 MB   Passenger spawn server
29161  1        114.8 MB  0.7 MB   Passenger FrameworkSpawner: 2.1.2
30461  1        122.8 MB  32.3 MB  Passenger ApplicationSpawner: /home/deploy/www.myapp.com/releases/20081206183156
30784  1        129.3 MB  33.4 MB  Rails: /home/deploy/www.myapp.com/releases/20081206183156
### Processes: 5
### Total private dirty RSS: 67.08 MB
Pretty sweet.

2 ways to fix transparent PNG files in IE6

I fired up Photoshop for the first time in a long time. I created a transparent PNG for an image that would be used as a CSS background-image. It keeps displaying with a gray background even though the page background color was something else. This is a pretty well documented bug. Luckily, there are a couple of ways to fix this. Here are my two favorites: The first solution is well documented: Download iepngfix.zip. The development version 2.0 Alpha 3 has support for background position and repeat. Extract the zip and copy iepngfix.htc and blank.gif somewhere. I put in under stylesheets. Add the following snippet to your CSS stylesheet: 
img { 
        behavior: url(/stylesheets/iepngfix.htc); 
}
And you're done! Hit refresh in IE6 and transparent PNG images should render correctly now. You can add apply this fix to other elements that may be using PNG images as CSS background images as such: 
img, #logo { 
        behavior: url(/stylesheets/iepngfix.htc); 
}
where logo is a div that has a background-image that is a transparent PNG. Note: If you are using v2.0 and want to take advantage of background-repeat and position support, copy iepngfix_tilebg.js to your javascripts folder and include the js file in the HTML files you need it for. The second way to fix this is to get everyone off IE6 but I guess that's just wishful thinking...