Ruby Fun -- Aaron Qian's Blog

Merb, DataMapper, Upload Progress, Nginx, Rack, Mongrel, Subversion on SilverRack VPS

Aaron Qian — Fri, 27 Jun 2008 15:23:00 +0000

Note: This guide is from my own experience, if it didn’t work for you, or you caught mistakes in this site, please feel free to leave your comments/suggestion/corrections, and I’d be more than happy to update this article.

So You Want A Production Site?

After playing with merb 0.9.4 and datamapper 0.9.3 on my local development enviorment, I was delighted with this killer combo for their innovation, ease of use, clarity, and stability. I’ve made a little project with merb and wanted to test out how hard it would be to setup as production server. This blog is a tutorial on what I did to get it working.

Get your VPS

A few of the projects I worked before are currently using SilverRack slice to host their Rails applications, and from the past experience it was a good pick.

So I went straight to their website and filled out their purchase form. I used the following options:

256MB slice
railsconf coupon code. ($5 discount, don’t know if it works with other packages)
ubuntu 8.10 hardy

Initial setup

After the purchase, you will get an email with instruction to login to your slice with SSH. You can setup your DNS setting by logging in to SilverRack as well.

Now let’s get your new shinny Ubuntu VPS some basic apps! Logging in to your VPS as root and get this puppy going.

Fix some perl locale complaints Note, if you encounter errors similar to below:

perl: warning: Setting locale failed

The remedy is:

apt-get install --reinstall language-pack-en

Update your apt repository

Update sources.list:

nano /etc/apt/sources.list

Paste the following into sources.list:

deb http://us.archive.ubuntu.com/ubuntu/ hardy main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy main restricted

deb http://us.archive.ubuntu.com/ubuntu/ hardy-updates main restricted
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy-updates main restricted

deb http://us.archive.ubuntu.com/ubuntu/ hardy universe
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy universe
deb http://us.archive.ubuntu.com/ubuntu/ hardy-updates universe
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy-updates universe

deb http://us.archive.ubuntu.com/ubuntu/ hardy multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy multiverse
deb http://us.archive.ubuntu.com/ubuntu/ hardy-updates multiverse
deb-src http://us.archive.ubuntu.com/ubuntu/ hardy-updates multiverse

deb http://archive.canonical.com/ubuntu hardy partner
deb-src http://archive.canonical.com/ubuntu hardy partner

Update the package database:

apt-get update

Upgrade to the latest version of all base packages:

apt-get upgrade

Install some basic tools

apt-get install bash man-db cron wget lynx git-core subversion build-essential

Add a default user

adduser <username>

Give sudo access

Open sudoer config file:

nano /etc/sudoers

Add the following line at the end:

ALL=(ALL) ALL

Disable root login via SSH

Edit the sshd config file:

nano /etc/ssh/sshd_config

Find this line:

PermitRootLogin yes

and change to:

PermitRootLogin no

Restart SSH

/etc/init.d/ssh restart

Install Merb and Datamapper

Setup Ruby, Gems, and Rake

Get ruby:

apt-get install ruby irb ri rdoc rubygems ruby1.8-dev libzlib-ruby libyaml-ruby libreadline-ruby libncurses-ruby libcurses-ruby libruby libruby-extras libfcgi-ruby1.8 build-essential libopenssl-ruby libdbm-ruby libdbi-ruby libdbd-sqlite3-ruby sqlite3 libsqlite3-dev libsqlite3-ruby libxml-ruby libxml2-dev

Update your gem to latest

gem update --system

If you see something like this:

uninitialized constant Gem::GemRunner (NameError)

Then open /usr/bin/gem and Insert the following:

require 'rubygems/gem_runner'

After this line:

require 'rubygems'

Install Merb and Datamapper Dependencies

Type the following into your console:

gem install erubis rake json_pure rspec rack hpricot mime-types addressable english rspec

Now you can go have some coffee while the gems download and compile…

Download the latest gems from GitHub

mkdir ~/gems
cd ~/gems
git clone git://github.com/wycats/merb-core.git
git clone git://github.com/wycats/merb-more.git
git clone git://github.com/wycats/merb-plugins.git
git clone git://github.com/sam/extlib.git
git clone git://github.com/wycats/do.git
git clone git://github.com/sam/dm-core.git
git clone git://github.com/sam/dm-more.git

Compile and install all gems

cd merb-core; sudo rake install; cd..
cd merb-more; sudo rake install; cd..
cd merb-plugins; sudo rake install; cd..
cd extlib; sudo rake install; cd..
cd do; sudo rake install; cd..
cd dm-core; sudo rake install; cd..
cd dm-more; sudo rake install; cd..

Now you should have Merb and DataMapper both installed, to check their existence, you can do the following:

gem list

You should see gems with names start with “merb-” and “dm-”

nginx for Your Front-End

grab a latest stable source from nginx

At the time of writing, I downloaded the latest stable

cd ~
wget http://sysoev.ru/nginx/nginx-0.6.31.tar.gz

h4. Untar it

tar -xvzf nginx-0.6.31.tar.gz

h4. Make a modules directory for upload progress module

cd nginx-0.6.31
mkdir modules

Download the nginx upload progress module

Download the latest source tar.gz file, and untar it. Finally, put it in your “modules” directory

cd modules
wget http://wiki.codemongers.com/NginxHttpUploadProgressModule?action=AttachFile&do=get&target=nginx_uploadprogress_module-0.2.tar.gz
tar -xvzf nginx_uploadprogress_module-0.2.tar.gz
rm nginx_uploadprogress_module-0.2.tar.gz
cd ..

Install dependencies for Nginx

apt-get install libpcre3-dev libgcrypt11-dev libssl-dev

Configure Nginx

./configure --with-openssl=/usr/lib/ssl/ --with-md5=/usr/lib --add-module=modules/nginx_uploadprogress_module

Make and Install

make
make install

Setup Init Script

cd /etc/init.d/
wget http://topfunky.net/svn/shovel/nginx/init.d/nginx
chmod +x nginx

Make Nginx start on boot

update-rc.d -f nginx defaults

Start Nginx

/etc/init.d/nginx start

link Nginx Configuration Files to ect

ln -s /usr/local/nginx/conf /etc/nginx

Setup Your Subversion Repository

Install subversion and tools

apt-get install subversion subversion-tools libsvn-dev libsvn1

Setup a repository

adduser deploy
sudo svnadmin create /var/svn/mycode
chown -R deploy:deploy /var/svn/mycode

Edit svnserve config file

Open the config file:

nano /var/svn/mycode/conf/svnserve.conf

Modify the following lines:

anon-access = none
auth-access = write
authz-db = /usr/local/nginx/conf/svn_access.rules
realm = My Code Repository

Permissions

Create an svnserve access file:

sudo nano /usr/local/nginx/conf/svn_access.rules

Paste in the following and save the file:

[groups]
developers = deploy
qc = deploy
pm = deploy
[/]

@developers = rw
@qc = r
@pm = r

Init Script for snvserve

Create a svnserve startup file:

sudo nano /etc/init.d/svnserve

Paste in the following:

#!/bin/sh
# Use the following variables to customize user, group, and paths.
SVNSERVE=`which svnserve`
SVN_USER=aaron
SVN_GROUP=aaron
SVN_ROOT_PATH=/var/svn/mycode/

if [ -f /etc/rc.d/init.d/functions ]; then
# RedHat style
. /etc/rc.d/init.d/functions
START="daemon $SVNSERVE -d --root $SVN_ROOT_PATH"
STOP="killproc $SVNSERVE"
else
# Ubuntu LSB style
. /lib/lsb/init-functions
START="start-stop-daemon --start --exec $SVNSERVE -- -d --root $SVN_ROOT_PATH"
STOP="start-stop-daemon --stop --exec $SVNSERVE"
fi

if [ ! -x $SVNSERVE ]; then
echo "Could not find ${SVNSERVE}. PATH is ${PATH}"
exit 0
fi

case "$1" in
start)
echo "Starting svnserve..."
umask 002
$START
if [ $? ]; then
echo "Started svnserve"
exit 0
else
exit $?
fi
;;

stop)
echo "Stopping svnserve..."
$STOP
exit $?
;;

restart|force-reload)
"$0" stop && "$0" start
;;

*)
echo "Usage: /etc/init.d/svnserve {start|stop|restart|force-reload}"
exit 1
;;
esac

echo "Unhandled case while trying to start svnserve."
echo "see $0"
exit 3

Make it executable:

sudo chmod +x /etc/init.d/svnserve

Make svnserve start on boot

Add it to the startup scripts:

sudo update-rc.d /etc/init.d/svnserve defaults

Start the server

sudo /etc/init.d/svnserve start

Import your merb app

Create a directories for your app:

svn mkdir -m "make my_app" svn+ssh://domain.tld/var/svn/mycode/my_app
svn mkdir -m "make trunk" svn+ssh://domain.tld/var/svn/mycode/my_app/trunk

Import your app:

cd my_app
svn import -m "initial import of my_app" . svn+ssh://domain.tld/var/svn/mycode/my_app/trunk

Checkout your app:

makedir ~/projects/my_app
cd ~/projects/my_app
svn co svn+ssh://domain.tld/var/svn/mycode/my_app/trunk

Capistrano

To be continued…

Note: This guide is from my own experience, if it didn’t work for you, or you caught mistakes in this site, please feel free to leave your comments/suggestion/corrections, and I’d be more than happy to update this article.

References:

Website Design Methodology -- The Golden Pyramid

Aaron Qian — Fri, 27 Jun 2008 06:27:00 +0000

Optimizing Design Efficiency and Cost

Building a website is similar to building a pyramid, except it is built from the top of the pyramid to the bottom. The pyramid below represents different phases of creating a website from an idea to completion. The sizes of each phase gives you an idea of how much time/money it will take for each changes made during that phase.

Idea Phase

This is the initial phase of making a website. Primary goal is to gather initial requirements of the site. However, the initial requirements is often incomplete, vague, and often contradictory. Often times driven by the vision of the boss.

Spec Phase

This is the phase where you make your initial spec complete, correct, concise, and clear. Several tools will be used to facilitate this goal:

Wire Diagrams (Schematics)
Flow Diagrams (Use Cases)

At this stage, no design is done. However, functionalities, features, and general layouts are decided at this stage. This stage should also involve the Operation, Design and Development team and hear their input for several reasons:

Is this spec operationally suitable for the operations team? (Operations)
How to maximize Usability? (Usability)
How to lead different types of users through our website to increase the conversion rate? (Design and Usability)
Is this spec possible to implement? (Dev)
How long will it take? (Design, Dev) What plans do we have for SEO? (Dev, Design, Operations)

The detailed spec should be finalized BEFORE moving into design and development. Often times this process is lengthy and chaotic. It’s important that an open exchange of ideas are tabled and discussed as a team. It’s also critical that there’s a clear decision maker reach closure.

Design Phase

This is the phase when the spec is finalized, and we start to create the prototype of the website. This time, design should only focus on the look and feel, not usability and functional issues. When this phase is done, ideally an HTML mock up should be done with links to demonstrate how the site should look like when it is operational. Also, when changes to usability and functional changes must be made, (meaning Spec is done wrong) there should be a meeting where all member should get involved.

Development Phase

At this phase, development of the backend starts. The design should have finalized the draft, and the development team should follow the design draft. (if HTML mock up is done, the develop team should use HTML mock ups)

Boohoo, My Site is Down!! -- Got Capistrano?

Aaron Qian — Thu, 26 Jun 2008 16:26:00 +0000

Servers DO Fail—Get Real

About three days after my failed attempt in persuading my contractor FooBy Company (for privacy reasons, the name is not real) to setup automatic deployment solutions for their rails online shopping service, I got a call from them and was informed that the website is down, and they can’t seems to get the site back up. I have just started working with them. Without the knowledge of the internals of the server structure, I ssh into their server(a little nervous), read their poorly written bash scripts, hack around. After about one and half hours of frustration, 2 server reboots, a dozen mongrel_rails cluster::start, a few undeleted pids, and countless French. I finally brought the site back from oblivion.

Total downtime: 3 hours 23 minutes…

A Better Solution Than French

Someone as smart as you might wonder, there must be a better solution to this. Yes, if you ask any creditable rails developer, usually you will get Capistrano.

As you probably already know, Capistrano is a automatic server deployment solution that ease the complexity of rails/merb deployment. You can find out more on their website.

How long does it take to setup Capistrano?

```
sudo gem install capistrano
```
```
cd /YOUR_APP_ROOT; capify .
```
```
mate config/deploy.rb
```
and modify to your liking
```
cap deploy:setup
```
ssh into your server, create directories, create config files
```
cap deploy:cold
```

Setup time: 20 minutes. Deployment time: 1minute. (Just run cap deploy)

Do it right => Do it fast

From my own experience in working with managers and CEOs, I have concluded some commonalities.

Speed is everything.
Short term benefit is always greater than long term benefits.
We should do it right, but we should do it later, because it cost too much
Don’t fix/change/optimize it if it works now.
We know there are potential problems, let’s fix it when the problem arises

No No No!

Do it right, and do it once!!