The command line

We have a working deployment setup. It runs a Magma image and several Seaside images. The Seaside images are all of them running copies of a single Seaside image. This Seaside image has a copy of the application we want to deploy. In this tutorial the application is the SeasideProxyTester from the SeasideExamples of squeaksource.com.

The SeasideProxyTester is very simple because it doesn’t want to show any particular Seaside feature. It is a tool to test a setup of proxied Seaside images behind a proxy webserver. It consists of three classes:

• SPTApplication. It is a Seaside application registered as seasideProxyTester.
• SPTApplicationMagma. It is a Seaside application registered as magmaProxyTester
• SPTDatabase. It is used as the root of the domain objects in the Magma repository. It is used by SPTApplicationMagma to store the number of requests made to the application

SPTApplication and SPTApplicationMagma are very simple. When each application is accessed a simple page is rendered, showing the number of requests made so far to the application. The request can be counted in three ways:

1. By Seaside session. Number of requests made as part of the same Seaside session (that is, _s is the same for all of them).
2. By Seaside image. Number of requests made to the same image, no matter if they are from different Seaside sessions (that is, the requests have the same value for the server cookie: app9001, app9002, etc)
3. By application. Number of requests made to the application, no matter if they are from different Seaside sessions or Seaside images (that is is a global counter of the requests made to any of the images in any of the created Seaside sessions).

The magmaProxyTester works exactly as the previous points explain. The Magma database is used to store the global request number and a request number for each image.

The seasideProxyTester can’t, by their nature, keep track of the third counter, that is the global request counter or application request counter. This is because each image is an autonomous entity that has no way to communicate between them to share and update the global request counter. So, although each SPTApplication in each image has a class instance variable to track the number of requests, this is scoped to the image and only can track the number of request that reached the image is part of. So each image has it very own global counter that tracks the requests that has processed. If you sum the global counter of each Seaside image for the SPTApplication, you can get the real global request counter for the seasideProxyTester application. In the SPTApplicationMagma app, the magma database is the external (to the image) holder of those counters, and is also responsible of mediating between them when updating the counters.

So with this warning out of the way. What can you get from this SeasideProxyTester. Lets begin with the magma.example.com application. The first time I access it it shows:

This shows that the request was served by the first Seaside image listening on port 9001.  This request has been correctly counted. If I reload the page I create a new Seaside session but i remain in the same Server (as my browser has accepted the cookie with value app9001). Now the page shows:

As you can see you are on the same server (the one listening on port 9001) and the global counters have been updated accordingly.

Now, I delete the cookies of my browser so that the next reload it doesn’t include the server cookie. lighttpd will use round-robin to proxy to the next available server, that is, the one on port 9002. Now I see:

You are now on a different server, the one listening on port 9002. Well, you get the idea.

Now, there are two links on the page. The first one, “Reset database”, as you can guess, will reset the global counters. But, as this is also a request, the counters won’t be 0 but 1 the next time the page is rendered.

The second one “Make a new request inside this session” will make a new request (by means of a Seaside callback) that is part of the same session (same _s parameter). This, as always, changes the global counters, but also increments the session request counter.

So far so good.

The seasideProxyTester works very similar, the only problem, as I have said is that there is no way for all the Seaside images to update a group of shared global counters (this, in the end, will need a external database, the role Magma is playing in the magmaProxyTester), so each image has a class instance variable that tries to keep track of the global counters but this is only valid inside the image scope. Repeat the same requests made with the magma.example.com using the seaside.example.com and you’ll note that each time you change of Seaside server you get a different global counter. Only if you sum all the global counters you will have the real global request counter.

Now you can put your stress loading tools to work by pointing them to your new setup and measure the results.

Or, wait for the next post and I will show you one way to do load testing on Seaside applications

You have a working squeak vm install. Now we will create the directories we’ll use.

Create directories

export WORK=/home/miguel/work
mkdir -p $WORK

export DEPLOY=/home/miguel/example
mkdir -p $DEPLOY/{pharo,magma,backup,logs,scripts,website}

We will use two directories, put them where you want. I chose to put them on my home directory but you can use any other if you wish. The important thing is to have the environment variables correctly assigned. This will ease the following steps.

The work directory is to hold temporary files as we setup the deploy directory. At the end will be discarded.

The deploy directory is what we will populate with the images and other useful scripts to host our Seaside application and data. As you can see has directories for the images (pharo), for the database (magma), for the magma backups (backups), for the logs (currently only has the output of the nohups used to start the images), for the scripts (guess, scripts!) and for the static content of your application, that you wisely have the webserver to serve and not the Seaside server (website). More on this later.

Download PharoCore

Now go to the Pharo download page look for the section “Sources files” and download the Sources zip file. At the moment is:

SqueakV39.sources.zip

Now go to the bottom of the page and follow the link that says “Pharo-core images and other files”. Find the most recent PharoCore zip file. Currently is:

PharoCore-1.0-10451-BETA.zip

but any newer will do.

Unzip this two zip files. The PharoCore will create a directory and inside it will be the image and changes files. The Sources zip contains one file. Now, copy this three files to the $WORK directory:

cp PharoCore-1.0-10451-BETA.image $WORK
cp PharoCore-1.0-10451-BETA.changes $WORK
cp SqueakV39.sources $WORK

So far so good. You have a PharoCore image with its changes file and a sources file. This, together with the virtual machine gives you a complete Pharo environment to work. You can try it:

cd $WORK
$VM PharoCore-1.0-10451-BETA.image

You should see the PharoCore image running. Quit the image WITHOUT saving.

Save scripts

Save the following scripts to the $WORK directory.

magma-image.st:

“Install Magma Server on a PharoCore image”

“Set some preferences”
Preferences enable: #fastDragWindowForMorphic.
Preferences disable: #windowAnimation.
Preferences enable: #updateSavesFile.
Preferences disable: #windowAnimation.

“Update from pharo update stream (only works/recomended for PharoCore)”
Utilities updateFromServer.

“Your name, like MiguelCoba or VincentVanGogh, dont  use spaces or accents, just ASCII”
Author fullName: ‘FirstnameLastname’.

“Install Installer”
ScriptLoader
loadLatestPackage: ‘Installer-Core’ fromSqueaksource: ‘Installer’.

“RFB”
Installer lukas project: ‘unsorted’;
install: ‘RFB’.

“Magma server”
Installer ss project: ‘Magma’;
install: ‘1.0r42 (server)’.

“Configure the packages”
RFBServer current
initializePreferences;
allowEmptyPasswords: false;
allowLocalConnections: true;
allowRemoteConnections: false;
allowInteractiveConnections: true;
connectionTypeDisconnect;
configureForMemoryConservation;
setFullPassword: ‘useyourownpasswordhere’.

“Save with a new name”
SmalltalkImage current saveAs: ‘magma’.
SmalltalkImage current snapshot: true andQuit: true.

This script when executed on a PharoCore image, will set some preferences, apply updates from the Pharo project if available and then install some packages directly from their repositories. The packages installed are RFBServer (a VNC server for Squeak and Pharo) and the Magma server. Finally it configures the packages and saves the image with a new name: magma. Be sure to change your full name and the RFBServer before saving the file.

magma-run.st:

“When a file named magma.shutdown is found on the same directory as the image
this process is triggered and the image is shutdown without saving”
[
[
[ 60 seconds asDelay wait.
(FileDirectory default fileOrDirectoryExists: 'magma.shutdown')
ifTrue: [ SmalltalkImage current snapshot: false andQuit: true ].
(FileDirectory default fileOrDirectoryExists: ‘magma.startvnc’)
ifTrue: [ Project uiProcess resume.  RFBServer start:0 ].
(FileDirectory default fileOrDirectoryExists: ‘magma.stopvnc’)
ifTrue: [ RFBServer stop. Project uiProcess suspend ].
] on: Error do: [ :error | error asDebugEmail ]
] repeat
] forkAt: Processor systemBackgroundPriority.
“To save CPU cycles”
Project uiProcess suspend.

I will explain this script later.

seaside-image.st:

“Install Seaside on a PharoCore image”

“Install packages”

“Comanche”
Installer ss project: ‘KomHttpServer’;
install: ‘DynamicBindings’;
install: ‘KomServices’;
install: ‘KomHttpServer’.

“Seaside”
Installer ss project: ‘Seaside’;
answer: ‘.*username.*’ with: ‘admin’;
answer: ‘.*password.*’ with: ’seaside’;
install: ‘Seaside2.8a1′;
install: ‘Scriptaculous’.

“Seaside Jetsam”
Installer ss project: ‘Jetsam’;
install: ‘Seaside28Jetsam-kph.67′.

“Seaside helper”
Installer ss project: ‘MagmaTester’;
answer:’username’ with:’admin’;
answer:’password’ with:’seaside’;
install: ‘Magma seasideHelper’.

“SeasideProxyTester”
Installer ss project: ‘SeasideExamples’;
install: ‘SeasideProxyTester’.

“Configure the packages”
“Start Seaside”
WAKom startOn: 9001.

“Unregister example apps”
WADispatcher default trimForDeployment.

“Unregister deployed apps”
WADispatcher default
unregister: (WADispatcher default entryPointAt: ‘/browse’);
unregister: (WADispatcher default entryPointAt: ‘/config’).

“Save with a new name”
SmalltalkImage current saveAs: ’seaside’.
SmalltalkImage current snapshot: true andQuit: true.

This script install Seaside 2.8, Magma seasideHelper and the SeasideProxyTester app that we will use to test the setup. Besides, start Seaside on port 9001, unregister unnecessary apps from the Seaside dispatcher and save the image as seaside.

seaside-run.st:

“When a file named seaside.shutdown is found on the same directory as the image
this process is triggered and the image is shutdown without saving”
[
[
[ 60 seconds asDelay wait.
(FileDirectory default fileOrDirectoryExists: 'seaside.shutdown')
ifTrue: [ SmalltalkImage current snapshot: false andQuit: true ]
] on: Error do: [ :error | error asDebugEmail ]
] repeat
] forkAt: Processor systemBackgroundPriority.
“To save CPU cycles”
Project uiProcess suspend.

I will explain this script later.

start_app.sh:

#!/bin/sh

HOME=”/srv/example”
NOHUP=”/usr/bin/nohup”
VM=”/opt/pharo/squeak -mmap 100m -vm-sound-null -vm-display-null”
IMAGES=”$HOME/pharo”
SCRIPTS=”$HOME/scripts”
LOGS=”$HOME/logs”
START_PORT=9001
END_PORT=9004

# Delete command files

[ -f $IMAGES/magma.shutdown ] && rm $IMAGES/magma.shutdown
[ -f $IMAGES/magma.startvnc ] && rm $IMAGES/magma.startvnc
[ -f $IMAGES/magma.stopvnc ] && rm $IMAGES/magma.stopvnc
[ -f $IMAGES/seaside.shutdown ] && rm $IMAGES/seaside.shutdown

# Start the Magma image
echo “Starting Magma image”
$NOHUP $VM $IMAGES/magma.image $SCRIPTS/magma-run.st >> $LOGS/magma.nohup &

# To give Magma time to open the repository
sleep 5

# Start the Seaside images
for PORT in `seq $START_PORT $END_PORT`; do
echo “Starting Seaside image on port: $PORT”
$NOHUP $VM $IMAGES/seaside.image $SCRIPTS/seaside-run.st port $PORT >> $LOGS/seaside$PORT.nohup &
done

I will explain this script later.

Prepare images

Make sure that the previous scripts are saved to the $WORK directory. Then build the images:

cd $WORK

# Build magma image from PharoCore image
$VM PharoCore-1.0-10451-BETA.image $WORK/magma-image.st

# Build seaside image from the magma image
$VM magma.image $WORK/seaside-image.st

This will take the PharoCore image and, by using the scripts given, will build the magma image. Then, using the magma image, will build the seaside image.

The build scripts are based on the scripts included in the pharo-dev and pharo-web images created by Damien Cassou.

The magma-run.st and seaside-run.st scripts are based on the ones Ramon Leon posted on his blog.

We are going to deploy a Seaside application that uses Magma seasideHelper to connect to a Magma database in a production server. The production server will have an image for the Magma server and serveral Seaside images with the application code. All the images will be PharoCore images with just the necessary packages loaded and nothing else. The users’ sessions will be load balanced and proxied by a web server. The Magma image, the Seaside images and the webserver will be hosted on the production server. For this tutorial we will use the SeasideProxyTester application from the SeasideExamples project on SqueakSource.com. You can use your own code given that you do a few minor modifications that I will detail later.

Although this tutorial host everything on a single production machine, you can use as many servers as you want. For example, you can use a server for Magma, a group of servers for the Seaside images and a hardware load balancer instead of the webserver. Is up to you. I want to keep things simple here and I show a setup that involves the minimal set of parts but that explains the way to deploy a Seaside app to production.

You can follow this instructions directly on the production server and get exactly the same results but I recommend to first do it on your development machine and, when is working correctly, stop it, zip it, copy it to the production server, unzip it and start it. But as you want. I will do it on my development machine and the last part, the copying to production server, is left as an exercise for you (but it is very simple, really).

I will work with a development machine that has Debian GNU/Linux Testing (Squeeze), but my production server has Debian GNU/Linux 5.0 (Lenny). The instructions apply to both unchanged.

You can install and configure your production server following this instructions. So to the first step.

Install the Squeak VM

The version of squeak installed on Debian Lenny and Debian Squeak with the following command:

# aptitude install squeak-vm

installs a vm that is not closure enabled (for explanations search the Squeak or Pharo mailing lists). As we are going to use the PharoCore we need VM that support closures. So we won’t install squeak from the Debian repositories (at least until they ship a closure vm, in which case I will update this instructions) but we will download the VM from the Pharo project download page. There you’ll find a section named Virtual Machines. Download the Linux/Unix VM. Currently is

pharo-vm-0.15.2d-linux.zip

but any newer version will be ok.

Now, as the root user, copy the zip file to /opt, unzip it and make a symbolic link:

cd /opt
unzip pharo-vm-0.15.2d-linux.zip
ln -s pharo-vm-0.15.2d-linux pharo

Back as your normal user, test it:

export VM=/opt/pharo/squeak
$VM -help

This will show the help of the squeak vm executable. Read it carefully to understand the options we will use to start our images.

This is the first of a several post about Seaside.

The goal: to deploy a Seaside application to the real world.

Ingredients:

• Debian GNU/Linux
• PharoCore
• Seaside
• Magma
• Magma seasideHelper
• Lighttpd 1.4.23.
• Automated image setup
• ab
• Gnuplot

I must say that this is not the only way to do this kind of deployment. Ramon Leon’s post explain other setup and also the Seaside book has a section on scaling Seaside. This is the way I do it.

Some things to discuss first. I will be using Debian GNU/Linux Squeeze but the procedure is the same for other versions of Debian and for other distributions. I think that this setup can be done in Windows too but I don’t plan to check. I will be using Seaside 2.8 because is the current stable version. When the 3.0 release is announced I will update the instructions accordingly. The Magma version will be the 1.0r42 (server) that was recently released and that since this version runs also in Pharo. I will be using Magma seasideHelper that although for the moment isn’t being mantained, the version used here works very well. This is used for easy integration of Magma and Seaside. I will use lighttpd because that is the webserver I use on my sites but the equivalent Apache configuration it is very easy and sometimes shorter that the ones I show. nginx or pound can be used too. The webserver will be used to proxy the requests from the users to the Seaside server images. It will be doing load balancing too. In order to test the setup a simple Seaside application has been developed and uploaded to the SeasideExamples project on SqueakSource. This can be used to verify that the webserver is doing a sticky session kind of load balancing. This is necessary because for a setup of proxied Seaside images a session must be routed always to the same image where it was created. This is not necessary on applications deployed to Gemstone/S or GLASS because on them the session data is persisted to the OODB and restored on subsecuent request no matter what stone the request is handled by. But, in a setup that doesn’t use Gemstone/S the proxy/load balancing server must guarantee that a session is always routed to the same image. Of course, on the first request of a session, the server choose a random image to handle the session so that the users are distributed uniformly between the images.

The setup will be tested with ab, the tool from the apache project for load testing websites. This is a very simple test, as only request the first page of the application. This shows how many new sessions can be created in the cluster of Seaside images. It doesn’t test inner navigation of the application. Of course, the same setup can be tested with tools like Selenium, JMeter, httperf, siege or jcrawler. This is left as an exercise to the reader. One cool feature of ab is that can output the results in a format understood by GNUPlot so that they can be plotted and visualized more easily.

Scripts to automate image setup are provided too. They update a PharoCore image and prepare Magma and Seaside images with just the required packages. Also, they configure an RFBServer (VNC server) so you can connect to the Magma image to do routine tasks as database backup (this at least until you integrate that tasks to your app so you can launch them from Seaside without entering the image).

Finally a script to start the images and procedures to stop them on demand are provided.

So, a lot of steps, but at the end you’ll have a fully deployed app in a server using a domain name and suitable of scaling by adding more images or physical servers behind the proxy/load balancing server. Also, with the current high availability features of Magma you can also increase the persistence performance by adding more magma servers to the setup.

Ready? Lets go…  to the Seaside.