Using the NSAPI tunnel

Using the APACHE tunnel

The Tunnel Configuration File

Using the CGI tunnel

The Locolink tunnel is a piece of software that allows a web server to send and receive information from a Locomotive. There are currently three varieties: a CGI based version, an Apache module, and a NSAPI version. The CGI program is provided for compatibility- it can be used on nearly any web server in existence. The NSAPI and Apache tunnels, however, are much more efficient and are the recommended choice.

Each tunnel provides a way to connect to multiple Locomotives from a single web server, and each provides a means of balancing load between multiple Locomotives. Read on for information on how to configure each tunnel for your particular system.

Before we get on with explanations of each of the tunnels, here's a helpful hint: If you are running multiple Locomotives on your site, we suggest that you create multiple locolinks tied to each individual Locomotive, in addition to the locolink which routes to multiple Locomotives. For example, if you're running Locomotives on ports 5421 and 5422, you can create two other locolinks besides your main locolink, called locolink_5421 and locolink_5422. We've found that having a locolink which routes exclusively to a particular Locomotive allows us to test that Locomotive specifically. For example, sometimes a Locomotive may "freeze"; it may stop responding to any requests on its port, for one reason or another, be it the Java VM, the database driver, the database itself, or code within the Locomotive which ties up a vital resource. Having a locolink which talks exclusively to that Locomotive will allow you to pinpoint, from any Web browser, which of your Locomotives is frozen, so you can stop it and restart it.

To set up extra locolinks for the NSAPI tunnel, just add an ObjectType and Service method to your obj.conf file for each locolink you you need. The NSAPI instructions will tell you just what you need to do. For the CGI, you'll have to configure and compile a CGI for each locolink you want to set up. That's explained below too.

To set up extra locolinks for the Apache tunnel, just add extra <Location> objects to the your access.conf file. You'll see how to do that in the Apache instructions below.

The NSAPI Tunnel

In this guide unless specifically noted, instructions for installation with FastTrack or with the Enterprise Server are the same. The name FastTrack can be used interchangeably with Enterprise Server.

It assumes some basic knowledge of TCP/IP networking, file systems, and startup/shutdown of Netscape's FastTrack Web Server.

Unix Installation

  Installing the NSAPI Tunnel requires four major steps:

1. configure the web server to load and invoke the tunnel
2. configure the tunnel destination list(s)
3. restart the web server
4. add your tunnel identification token to your loco.conf file

The Locomotive NSAPI Tunnel is distributed as a shared object file, named "locolink_nsapi.so". It comes in the Locomotive distribution and can be found in the Locomotive's lib directory under the sub-directory for the appropriate operating system. For instance if you're web server is running on Sun Microsystems Solaris (Sparc), you will find the shared object you need at lib/solaris_sparc/locolink_nsapi.so. You may wish to copy this file into a another location depending on your needs. For the purposes of simplicity in this guide we will leave it where it is, and will also assume you installed the Locomotive in the default recommend location /usr/local/loco. Thus, the path to the shared object would be in /usr/local/loco/lib/solaris_sparc/locolink_nsapi.so.

We need to tell FastTrack where the Tunnel software is, along with what functions to load in at startup time, how to identify a request that is destined for a Locomotive, and what function to call when a request for a Locomotive is received. This is done through the FastTrack's "obj.conf" configuration file, which it reviews during starting or restarting. See Netscape's online or printed documentation for FastTrack for a full description of this file.

The obj.conf file will be located within the FastTrack distribution, under what we call the 'web server instance' directory set up during FastTracks's installation. FastTrack sets up a different directory for each copy of the FastTrack server that is to be run. If you installed FastTrack as /usr/local/fasttrack and you named your default installation "httpd-myhost-80", you'll find the obj.conf file at /usr/local/fasttrack/httpd-myhost-80/config/obj.conf

There are a minimum of four lines that need to be added to the obj.conf file. Below is an example of each line, followed by a brief description. Note that the '\' at the end of the line means a line wrap. The Init line below should actually be one line in the obj.conf file.

Init fn=load-modules \
         shlib=/usr/local/loco/lib/solaris_sparc/locolink_nsapi.so \
         funcs="locolink-init,locolink"

This line tells the FastTrack server where the Tunnel shared object file is located and which functions should be linked in at startup using the dynamic linker of the operating system. In the above example, the shared object file is located at /usr/local/loco/lib/solaris_sparc/locolink_nsapi.so. The functions that FastTrack will look for are called 'locolink-init' and 'locolink', as specified by the 'funcs' paramter.

     Init fn=locolink-init

This line must appear after the load-modules line. It tells FastTrack to call this function once whenever it is started. locolink-init initializes static data used later by the Tunnel whenever the Tunnel is invoked.

ObjectType fn=type-by-exp exp=*/locolink* type=magnus-internal/locolink

This line configures the request routing mechanisms inside FastTrack, and works along with the "Service" line, described below. It tells the web server how to identify a request that is meant for a Locomotive. When a request is received by FastTrack, it runs through a series of rules to select the right function for handling the remainder of the request. ObjectType is used to set FastTrack's internal representation of the data type of a request. Once a data type is set, an appropriate Service function can be invoked. Here we use a FastTrack built in function called 'type-by-exp' which stands for "type by expression" to set the data type. Any URI containing the string '/locolink' will be mapped to the data type (like a MIME type) of 'magnus-internal/locolink'. Other ObjectType functions can be used also. See Netscape's FastTrack documentation for a description of others.

Service method=(GET|POST) type=magnus-internal/locolink \
         fn=locolink cfgpath="/var/tmp/locolink.conf"

This line configures a Service for handling requests to the Locomotive. Any HTTP 'GET' or 'POST' request that has been identified as a "magnus-internal/locolink" data type will be handled by the Tunnel (function 'locolink'). The 'cfgpath' parameter specifies the location in the file system of the Tunnel destinations configuration file. The contents of this file (described further below) tell the Tunnel how to connect to one or more available Locomotives.

There may be cases where you'll want to have more than one Locomotive destination list installed. For instance, you may want to be able to connect to a specific Locomotive out of a pool, rather than just any in the pool. This can be accomplished by additional ObjectType and Service entries. Using a slightly different pattern match in the URI, you can map the data type to something other than that of the primary Locomotive pool. For example, the lines below will allow the Locomotive locations specified in "/var/tmp/locolink.conf" to be accessed by any URI that contains '/locolink/'. If the URI contains '/locolink_p5421', the Tunnel will connect Locomotives specified in a different configuration file, "/var/tmp/locolink1.conf'. Note that each in FastTrack's operation, each 'ObjectType' line will get executed until one succeeds. If the first line below did not contain the second '/', neither of the two following ObjectType lines would ever get called. Because of this, and for sake of speed, you'd want your primary pools to be listed first, but not void out access to your other Locomotives. Hold on, we're getting a bit ahead of ourselves.

ObjectType fn=type-by-exp exp=*/locolink/* type=magnus-internal/locolink
ObjectType fn=type-by-exp exp=*/locolink_p5421* type=magnus-internal/locolink1
ObjectType fn=type-by-exp exp=*/locolink_p5422* type=magnus-internal/locolink2
.
.
Service method=(GET|POST) type=magnus-internal/locolink1 fn=locolink \
 cfgpath="/var/tmp/locolink.conf"
Service method=(GET|POST) type=magnus-internal/locolink1 fn=locolink \
 cfgpath="/var/tmp/locolink1.conf"
Service method=(GET|POST) type=magnus-internal/locolink2 fn=locolink \
 cfgpath="/var/tmp/locolink2.conf"

  Here is an example of a complete obj.conf file:

Init fn=flex-init access="/usr/local/fasttrack-3.01/httpd-io-fastrk3.01/logs/access" format.access="%Ses->client.ip% - %Req->vars.auth-user% [%SYSDATE%] \"%Req->reqpb.clf-request%\" %Req->srvhdrs.clf-status% %Req->srvhdrs.content-length% %Req->headers.referer"
Init fn=load-types mime-types=mime.types
Init fn=load-modules \
     shlib=/usr/local/fasttrack-3.01/lev_nsapi/locolink_nsapi.so \
     funcs="locolink-init,locolink"
Init fn=locolink-init

<OBJECT name=default>
NameTrans fn=pfx2dir from=/ns-icons dir="/usr/local/fasttrack-3.01/ns-icons"
NameTrans fn=pfx2dir from=/mc-icons dir="/usr/local/fasttrack-3.01/ns-icons"
NameTrans fn=pfx2dir from=/cgi-bin dir="/usr/local/cgi-bin" name="cgi"
NameTrans fn=document-root root="/usr/local/www"
PathCheck fn=unix-uri-clean
PathCheck fn="check-acl" acl="default"
PathCheck fn=find-pathinfo
PathCheck fn=find-index index-names="index.html,home.html"
ObjectType fn=type-by-extension
ObjectType fn=type-by-exp exp=*/locolink* type=magnus-internal/locolink
ObjectType fn=type-by-exp exp=*/locosrv1* type=magnus-internal/locolink1
ObjectType fn=type-by-exp exp=*/locosrv2* type=magnus-internal/locolink2
ObjectType fn=type-by-exp exp=*/locosrv3* type=magnus-internal/locolink3
ObjectType fn=force-type type=text/plain
Service method=(GET|HEAD) type=magnus-internal/imagemap fn=imagemap
Service method=(GET|HEAD) type=magnus-internal/directory fn=index-common
Service method=(GET|POST) type=magnus-internal/locolink \
     fn=locolink cfgpath="/var/tmp/locolink_pool.conf"
Service method=(GET|POST) type=magnus-internal/locolink1 \
     fn=locolink cfgpath="/var/tmp/locolink1.conf"
Service method=(GET|POST) type=magnus-internal/locolink2 \
     fn=locolink cfgpath="/var/tmp/locolink2.conf"
Service method=(GET|POST) type=magnus-internal/locolink3 \
     fn=locolink cfgpath="/var/tmp/locolink3.conf"
Service method=(GET|HEAD) type=*~magnus-internal/* fn=send-file
AddLog fn=flex-log name="access"
</OBJECT>

<OBJECT name=cgi>
ObjectType fn=force-type type=magnus-internal/cgi
Service fn=send-cgi
</OBJECT>

Adding the NSAPI URI token to your loco.conf File

Now that you've gotten the web server set up for NSAPI, all that's left to do is to tell the Locomotive what token you've used to identify it in the obj.conf file. You do this by setting the NSAPI_TUNNEL_URI_TOKEN_PREFIX to whatever you set in the obj.conf file. For instance, if we used the following line to identify the Locomotive in the obj.conf file:

ObjectType fn=type-by-exp exp=*/locolink* type=magnus-internal/locolink

Then, we have to change our loco.conf file to the following:

NSAPI_TUNNEL_URI_TOKEN_PREFIX  /locolink

Once you do that, you're ready to use the Locomotive with the NSAPI tunnel! Note that you could have set the web server to use the locolink as its default service. If you do that, then there is no URI token that's used to identify the Locolink, and you can set the config above to anything you like.

Setting up Tunnel Destinations

In order for the tunnel to know how to connect to the Locomotive(s), you must tell it where they are located. Each Locomotive should be configured to listen on a TCP port on its host machine. If you are running multiple Locomotives on the same machine, you'll want to set each one up on a different port. This is done via the loco.conf file (see Installation Guide for details).

The "cfgpath" parameter in the Service line of Fasttracks obj.conf file is used to specify the location of the Tunnel config file for that specific Service. We call this the "locations file". In the example file above the path looks like "/var/tmp/locolink_pool.conf", or "/var/tmp/locolink1.conf".

The locations file has a very simple structure. It is not forgiving in syntax, nor does it support comments. The parser for it is very simple and very fast to reduce overhead in the web server. In a later version of the Locomotive, creation and maintainance of the file will probably be accomplished with a management tool, rather than be done by hand.

The first line in a locations file must contain only a single word, which can be either "balance" or "linear". This determines the type of scan pattern that the Tunnel will use when there are multiple destinations specified in the file. The scan pattern is describe further below.

The second line is the name of the virtual session cookie, and must be properly configured in order for the application server to achieve maximal efficiency. This is also explained below.

The remaining lines all have the same format, consisting of four tokens, separated by commas and no spaces:

ip_address,port,weight,instance_id

The ip_address is an Internet IP address, in Internet standard dot notation.

Examples: 172.16.0.9 or 206.79.139.8

Examples: 5421 or 5422

Let's look at the contents of a pool file first. Most often, you'll want requests to your site to be handled by a pool of Locomotives, for both performance and reliability. A pool is essentially a number of Locomotives, all running simultaneously and handling requests of the same nature. The file below specifies three Locomotive locations, and that the Tunnel should use the "balance" scan pattern.

balance
LOCO_INST_ID_SYSTAG=
172.16.0.9,5421,2,1
172.16.0.33,5422,3,2
172.16.0.33,5423,3,3

The file below specifies two locations, one a primary and one a backup:

linear
LOCO_INST_ID_SYSTAG=
172.16.0.9,5421,1,1
172.16.0.10,5421,1,2

Ok, so what do "balance" and "linear" mean? Linear is just that, linear. A request to a Locomotive will always be sent to the first address and port combination (or "location") listed in the file. Only if the tunnel fails to connect to the first location will the second (or nth) location be tried. This allows a simple fail over. Essentially the Tunnel will scan down the list starting at the beginning, until it gets a successful connect.

The balance scan type takes the weight into consideration and allows simple load balancing. The weights are summed, then a random position in the list is picked to start. The probability of an entry being picked is proportional to the individual weight of the entry divided by the sum of all the weights. For example, if we use the weights shown in the above example of a "balance" config file, we have weights of 2, 3, and 3. The sum is 8. Therefore the first location, with the weight of 2 will receive on average 2 out of every 8 Locomotive request (a chance or probability of .25).

Clearly, the load balancing described here is static - adjusting the load to a particular location accomplished by changing the weights. A future version of the Locomotive will allow dynamic load balancing, based on either response time or direct feedback from a "mission control" type system monitor.

The second line is the virtual session cookie name. This string consists of three smaller strings concatenated: the cookie prefix "LOCO_INST_ID_", the SYSTEM_TAG as configured in loco.conf, and an equals sign "=". For example, if the SYSTEM_TAG of your pool of servers is "Shipping_Dept", then the second line must be set to "LOCO_INST_ID_Shipping_Dept=" in order for the application servers to function efficiently. The tunnel needs this information to read cookies that have been set by the servers for virtual session management.

Notes on Location files:

The location files are scanned a maximum time of once per second. The file system timestamp on a file is checked first, and if the timestamp differs from a previous version the Tunnel will reload the file and update its internal list of destinations, weights, and scan types. If multiple locations files are specified in an obj.conf file, a file system timestamp is kept for each. Thus, it is not necessary to restart FastTrack to change destinations. A simple 'touch' of a file will trigger a reload. You can simply update the file, or replace it with a "rename" or "move", and the Tunnel will detect the change and reload it on the fly. Note that in most, perhaps all flavors of Unix, a mv is an atomic operation - you won't miss a hit.

Adding the Locolink module to Apache

The Apache tunnel is an Apache module. Traditionally, to add an module to the Apache Web Server, you must recompile the server. While that may sound daunting, it's actually not very difficult, as long as you have a C development environment set up.

Unix Installation

If you don't have a C development environment already set up, you'll need to do that first. The things you'll need are a version of 'make', and a version of 'cc', which is the UNIX C compiler. The GNU foundation make these readily available, in the form of GNU make and GCC.

Once you have your compiler set up, you should probably get the latest version of Apache. You can find that here.

Once you get the latest version of Apache, you can unpack by using the following command:

gzip -dc apache.tar.gz | tar xvf -

And you'll see the apache distribution being unpacked into a new apache directory. enter the directory and type:

./configure --prefix=/usr/local/apache \
                    --enable-module=so \
                    --enable-rule=SHARED_CORE \
                    --add-module=/usr/local/loco/src/c/locolink/apache/mod_locolink.c \
		    --enable-shared=locolink

Where --prefix is set to wherever you want to install Apache, and --add-module is set to wherever you have the mod_locolink.c source file located. Normally it's in the src/c/locolink/apache/ directory of the Locomotive distribution. The --enable-shared directive, along with the --anble-module=so and anble-rule=SHARED_CORE, allows you to swap in different versions of the locolink shared object after you've compiled and installed the server. This should create all the makefiles you need for your system.

Once the configure program is finished, type 'make'. That should compile the server, as well as the locolink tunnel. If you run into any problems, check out the Apache documentation for help.

After you've compiled Apache, you need to install it. To do that, type 'make install'. That should install everything to the directory location you specified in the --prefix directive above. If you are installing Apache in a protected directory, you may may have to run the 'make install' program as root or another privileged user.

Now that you've got Apache compiled and installed, all you have to do is edit the configuration files, and you're finished. To use the Tunnel, you'll have to specify a url token the Web server will use to direct requests to the tunnel. To do that, edit the 'etc/access.conf' file in your Apache installation and add the following:

	<Location "/locolink">
	  LocoLocationsFile /usr/local/apache/etc/locolink.conf
	  SetHandler locolink-tunnel
	</Location>

This tells the web server that all requests whose url paths begin with '/locolink' will be forwarded to the locolink-tunnel handler. The LocoLocationsFile is a config file that specifies which Locomotive to send the request to. See the information above on how to configure the Loco Locations File.

And that's it!

Configuring and Compiling the Locolink CGI

The locolink CGI is a fairly simple application, so it has been written in C to improve performance. The source code for the locolink is in a file called locolink.c in the bin directory. You can find the locolink executable program in your architecture-specific directory, such as loco/bin/solaris_sparc, loco/bin/linux, or loco/bin/freebsd. The default behavior of the locolink program is to route all requests from the web server to a Locomotive running on port 5421, on the same machine as the web server. If you wish to add more Locomotives, or to route to Locomotives running on other machines, you'll need to modify locolink.c.

Here's how you would modify locolink.c. Using your favorite text editor, go to the following lines in locolink.c:

/* Declare the number of server destinations and enter them:
 * portnum, expanded address, weight, instance ID. 
 * Note: 2130706433 is 127.0.0.1 expanded 
 */
#define DESTINATION_COUNT 1
static unsigned int destinations[DESTINATION_COUNT][DEST_COLS] = 
{ 
  {5421, 2130706433, 1, 1}
};
/* PORT, ADDRESS,   WT,ID */

First, let's talk about how the IP address is encoded. The four parts of the IP address have been encoded into one int value using the following formula (where the '^' symbol means 'to the power'):

part1*(2^24) + part2*(2^16) + part3*(2^8) + part4

127*2^24 + 0*2^16 + 0*2^8 + 1

Next, let's explain the meaning of the third number, which is the "weight" of a Locomotive. The weight of a Locomotive is an integer value. To compute how often the locolink will route to a particular Locomotive, add up the weights of all the Locomotives, and divide the weight of one Locomotive by the total weight to get a fraction which represents how often that Locomotive will be called. In this case, since there is only one Locomotive, it doesn't matter what its weight is; it will always be called.

The final number is the instance ID of the Locomotive. This value should match the value of LOCO_INST_ID as it is set in the Locomotive's configuration file. This number is used to route requests with the virtual sessions facility.

Now, let's modify our example. Let's say that you wanted to run two Locomotives, one on the same machine as the web server, on port 9876, and another on another machine on port 1543. The other machine's IP address happens to be 225.16.0.9. You want to make the locolink call the Locomotive on the other machine twice as often as the one on the current machine. First, let's calculate what the integer encoding for the other machine's IP address is:

225*2^24 + 16*2^16 + 0*2^8 + 9 = 3775922185

/* Declare the number of server destinations and enter them:
 * portnum, expanded address, weight, instance ID. 
 * Note: 2130706433 is 127.0.0.1 expanded 
 */
#define DESTINATION_COUNT 2
static unsigned int destinations[DESTINATION_COUNT][DEST_COLS] = 
{ 
  {9876, 2130706433, 1, 1},
  {1543, 3775922185, 2, 2}
};
/* PORT, ADDRESS,   WT,ID */

Notice that the weight of the first Locomotive is 1, whereas the second one has a weight of 2. Okay, now you modify locolink.c to work with this new setup. You would compile locolink.c into a program named locolink, and copy that executable program into your web server's CGI directory. For example, on UNIX systems, if you use gcc, you might do this:

% gcc locolink.c -lsocket -lnsl -o locolink
% chmod 755 locolink
% cp locolink /usr/local/web/cgi-bin/

Also note that the two Locomotives have different values (1 and 2) for their instance ID's. This is necessary in order for the virtual sessions facility to efficiently route user requests.