Howto-usosweb-with-openvz

What is USOSWeb?

Usosweb is web application written by Polish Universities for Polish education institutions. Its usage is simple, installation is a breeze, so this is why this document emerged.

The USOSWeb consist of two parts: the web application and migrator. Another important dependency is mysql. The role of migrator is to take data from external Oracle server and put it in local mysql (and back, opposite way), and web app displays the content to the client.

Well, the network construction can be somewhat unclear, so I'm explaining: one can install all three apps needed on one machine, but this is not safe, as migrator needs direct (network socket) access to Oracle which is usually behind firewall in more protected network. And by separating mysql one can achieve fairly clear installation.

So, MY way to install USOSWeb is:

• install webapp and migrator on 1st VE,
• install mysql on 2nd VE,
• copy 1'st mentioned VE to internal network, tweak it, and disable migrator in 1st VE.

Why in that order? because it is easy, you install two machines but later you got three... In this howto I'll omit mysql installation as its easy and not interesting.

What you can get from this howto?

You can have USOSWeb 3.09 with migrator installed in one day in OpenVZ VEs (Virtual Environments). And you can also skip A LOT of stress because you don't have to install X, kde, full blown Oracle etc. on Fedora or other crap ;-)

What is needed?

Nothing but some Gentoo and OpenVZ knowledge. Mysql and other administrative skills recommended. Equipment I use is two machines with OpenVZ properly set as well as three OpenVZ VEs.

Let's get dirty

Prerequisites (all VEs):

• mysql server is on separate VE in internet network.
• USOSWeb server is separate VE in internet network.
• internal server is separate VE in internal network.

Installing Apache, PHP and other basic stuff

Login to the USOSWeb machine and configure basic settings. Make the make.conf look like:

CFLAGS="-O2 -march=nocona -fomit-frame-pointer -pipe"
CHOST="x86_64-pc-linux-gnu"
CXXFLAGS="${CFLAGS}"
ACCEPT_KEYWORDS="amd64"
USE="-gtk -X -kde -gnome nptl nptlonly mysql ssl pam php apache2 usb syslog cdrom ldap"
CONFIG_PROTECT=/etc
LINGUAS="en pl"
MAKEOPTS="-j5"
FEATURES="parallel-fetch"
CLEAN_DELAY="0"

Then configure USE flags in /etc/portage/package.use:

dev-lang/php-5* cli apache2 expat fastbuild ftp gd hash iconv memlimit mysql nls pcre pic pdo reflection session simplexml sockets spl ssl tokenizer truetype unicode xml xsl zlib -xpm posix cgi force-cgi-redirect xmlrpc curl zip bzip2 imap gdbm xpm snmp dev-db/mysql berkdb perl ssl big-tables cluster -debug -embedded -extraengine -latin1 max-idx-128 -minimal -static ctype filter gmp mysqli oc8-instant-client soap spell xmlreader xmlwriter oci8-instant-client pcntl

..and then unmask ebuilds in /etc/portage/package.keywords:

dev-db/oracle-instantclient-sqlplus
app-admin/eselect-oracle

Ok, and then update VE and install apache and php with dependencies (fulfill fetch requirements for dev-db/oracle-instantclient-basic-10.2.0.3 when asked). Install eselect-oracle:

emerge -Du world 

Cool :-) stay tuned until it finishes. In the meantime you can check Portage buildpkg feature so the next try of USOSWeb installation goes faster... Then emerge some more stuff and if you just start with the system - make its starting default:

emerge -Du apache php vixie-cron syslog-ng app-admin/eselect-oracle
rc-update add vixie-cron default
rc-update add syslog-ng default

Installing other packages

Now install packages the maintainers ask in official doc. Please check the line, as for me there were already some of them installed as dependencies:

emerge mail-mta/ssmtp dev-java/sun-jre-bin dev-python/mysql-python dev-perl/XML-DOM app-text/sablotron dev-perl/Curses dev-perl/DBI dev-perl/DBD-mysql dev-perl/Text-Iconv dev-perl/XML-LibXML

Ok, usual server system have no tetex, am I right? There is nothing in official docs about, but I already know that we need to install it here. So please do another emerge:

emerge tetex

Installing cx_oracle

cx_Oracle needs to FIND oracle installation. Let's make it visible (I remember this step took me hours last time ;-):

eselect oracle set 1
env-update && source /etc/profile

And more yet: cx_Oracle 4.3.3 still doesn't find my Oracle, so I had to use not the very last version. Your mileage may vary and, as always, you should use the newest, not like me:

cd /home && wget http://dfn.dl.sourceforge.net/sourceforge/cx-oracle/cx_Oracle-4.2.tar.gz
tar -xvpf cx_Oracle-4.2.tar.gz && cd cx_Oracle-4.2
python setup.py build
python setup.py install

Cool. We've just finished system preparation. Now go to real fun.

USOSWeb application installation

Now we need to get usosweb sources from our boss (he has them on his mail or wherever). Copy them to /home, unpack and configure:

cd /home && tar -xvpf usosweb-3.09.tar.gz && cd usosweb-3.09 && ./configure

Now you need to confess with your settings.

You should change here at least:

• APACHE_CONF (/etc/apache2 instead of /etc/httpd)
• APACHE_GROUP (apache instead of http)
• SBINDIR (other than /usr/sbin)
• INITDIR (to /etc/init.d/ instead of /etc/rc.d/
• USOSROOT (to anything you wish)

These settings are written to /etc/usosweb/usosweb.conf and /etc/usosweb/migrator.xml (/etc/usosweb is default location for those files, you may choose other).

Now issue make commands:

make
make install
make setperms

Then go to your usosweb.conf and tweak it - if you don't use CAS comment corresponding lines (P_AUTH, CAS_HOST, CAS_PORT 443, CAS_URI, CAS)

Configuring the whole mess

Configure Apache:

echo "Include /etc/apache2/apache-usosweb.conf" >> /etc/apache2/httpd.conf

And put sth like this is your /etc/apache2/vhosts.d/default_vhost.include:

ServerAdmin postmaster@yourdomain
ServerName usosweb.yourdomain
DocumentRoot "/your/server/root"
<Directory "/your/server/root/">
   Options Indexes FollowSymLinks
   AllowOverride All
   Order allow,deny
Allow from All
#    AuthName "usosweb"
#    AuthType Basic
#    AuthUserFile /etc/apache2/htpasswd
#    Require valid-user
</Directory>

Commented lines are suitable for basic authentication, so you may want to uncomment them as well as create correct htpasswd file. Then configure php.ini with defaults included in official docs.

Then setup the services:

rc-update add apache2 default
rc-update add vixie-cron default
/etc/init.d/apache2 start
/etc/init.d/vixie-cron start

Congrats, you have fully functional USOSweb (without mysql). Just take a look at your site ;-)

Installing databases on mysql server

Default mysql databases are included in the usosweb package. You need to modify user, password and then you can source it easily:

nano  usos_bazy.sql.in <= here change user, passwd, change typo in here: "\. urp/std_wzrp.sql" to "\. urp/std_wzrp.sql.in" etc.

Then source the file from mysql console:

\. usos_bazy.sql.in

Now check if all records went correctly. Double this check, seriously.

Mysql databases

Hey, when changing views in mysql database SUPER privilege is needed. So the above mentioned user won't do...

If you're not sure what to do, create the user that way (you should change it for localhost if you connect locally):

CREATE USER 'stefan'@'remotehost' IDENTIFIED BY '****************';
GRANT SELECT ,
INSERT ,
UPDATE ,
DELETE ,
CREATE ,
DROP ,
FILE ,
INDEX ,
ALTER ,
SUPER ,
CREATE TEMPORARY TABLES ,
CREATE VIEW ,
SHOW VIEW ,
CREATE ROUTINE,
ALTER ROUTINE,
EXECUTE ON * . * TO 'stefan'@'remotehost' IDENTIFIED BY '****************' ;

Ok, now go to USOSweb web interface, click "Nowa Administracja" and follow instructions. Create admin user. Then go to "Stara administracja" and click "Aktualizuj struktur?". This should go fine.

Moving further

Ok, now we need to copy some data with migrator. We can do this from the server we just setup, but MY way is to copy it to more secure internal network.

So now stop the VE, pack it, and send closer to your Oracle:

vzctl stop 110
cd /vz/private/110 && tar -jcvf 110.tar.bz2 *
scp 110.tar.bz2 stefan@whatever:~

On the second machine unpack and install correctly the server (TO DO: isn't any simpler way of OpenVZ VE migration?)

Configuing the SECOND VE

OK, the true is that you have almost all done. Now enter the second VE and change hostname of the host to your wish:

echo "HOSTNAME=\"migrator\"" > /etc/conf.d/hostname

Now configure oracle listener. Thats easy (if you know what to do):

mkdir -p /usr/lib/oracle/10.2.0.3/client/network/admin

Nano the sqlnet.ora to include:

NAMES.DEFAULT_DOMAIN = here.is.your.oracle.domain
SQLNET.AUTHENTICATION_SERVICES= (NTS, HOSTS)

Then nano the tnsnames.ora to include:

HERE-YOUR-DATABASE-NAME.here.is.your.oracle.domain =
 (DESCRIPTION =
   (ADDRESS_LIST =
     (ADDRESS = (PROTOCOL = TCP)(HOST = ip-address-of-oracle-host)(PORT = 1521))
   )
   (CONNECT_DATA =
     (SERVICE_NAME = here-is-your-service-name)
   )
 )

Now just make the /etc/cron.hourly/migrator.cron file and mark it executable:

#!/bin/bash
C_INCLUDE_PATH="/usr/lib/oracle/10.2.0.3/client/include" ORACLE_HOME="/usr/lib/oracle/10.2.0.3/client" \ 
NLS_LANG=AMERICAN_AMERICA.EE8ISO8859P2 /opt/usosweb/lib/migrator/migrator.sh >> /var/log/migrator-cron.log

Kill the apache (or b0rk it if you wish, its not needed on migrator host) :

/etc/init.d/apache2 stop
rc-update del apache2 default

Now read Fix me up, dude!, comment and launch the cron file or just wait for full hour to come. You are done!

Fix me up, dude!

There is an issue with MV_STYP table. Before using migrator you should comment MV_STYP table def in /etc/usosweb/migrator-mappings.conf, or you finish with unreadable python die.

Postsriptum

Are you not done yet? having problems with migrator? with php settings? with CAS? connecting to Oracle? mail them: usosphp (at) mimuw.edu.pl .

Revision date

Thu Jan 31 13:25:00 CET 2008