HOWTO_create_a_logserver_with_syslog-ng
| Installation • Kernel & Hardware • Networks • Portage • Software • System • X Server • Gaming • Non-x86 • Emulators • Misc |
Contents |
Overview
Much of this content was originally posted by Apprentice in 2004 on the gentoo forums and has now been wikified and embellished for your convenience. Logging to remote server is always a good idea. You can keep an eye on what's happening on your apache/mysql/something-else server for peace of mind or to simply help you to debug some services your remote box.
This HowTO will assume you have a "server" (your log server) to which all the logs will be sent, and client(s) from which all logs will originate. These logs can optionally be encrypted using SSL and stunnel.
Required Software
We will need syslog-ng, openssl and optionally stunnel. Setting up tunneling using openssh can be headache inducing, so we will stick with stunnel instead.
Emerge the packages on both the server and clients.
| Code: emerge packages |
emerge -avn syslog-ng openssl stunnel |
Creating SSL-Certificates
Server Certificate
Now we need to create certificates for the logserver and clients. Go to some not-public directory (like /root) and do the following:
| Code: Generate certificate and private key |
openssl req -new -x509 -out cacert.pem -days 1095 -nodes |
If you want you can change the -days value to something else (eg 365 for a year). Openssl will ask you some questions, so answer them as best you can. The answers you give don't really matter. If you want, you can check man openssl and create default config file. This is a good idea if you have a lot of clients.
Now you have two files: cacert.pem and privkey.pem. These will be the server's certificate and private key.
Syslog-ng clients only require the certificate files, so copy the cacert.pem to the client(s), rename it something like syslog-ng-server.pem and put it in the client's /etc/stunnel/ directory.
| Code: Example code |
scp cacert.pem root@YOURCLIENT:/etc/stunnel/syslog-ng-server.pem |
Now concatenate cacert.pem and privkey.pem to create a new syslog-ng-server.pem for our server:
| Code: Concatenate the certificate and private key |
cat privkey.pem cacert.pem > /etc/stunnel/syslog-ng-server.pem |
Client Certificate
Move the files we created (cacert.pem and privkey.pem) somewhere for backup and generate another key-cert pair:
| Code: Generate new certificate and private key |
openssl req -new -x509 -out cacert.pem -days 1095 -nodes |
This time, we do the opposite. Copy the new certificate to the server's /etc/tunnel directory and name it something like syslog-ng-client.pem:
| Code: Copy certificate to server /etc/tunnel directory |
cat privkey.pem cacert.pem > /etc/stunnel/syslog-ng-client.pem |
| Code: Example Code |
cat cacert.pem >> /etc/stunnel/syslog-ng-client.pem |
Now concatenate the new certificate and private key for the client side and copy it over to the client in its /etc/stunnel directory:
| Code: Cat and scp client certificate and private key |
cat privkey.pem cacert.pem > ./syslog-ng-client.pem scp syslog-ng-client.pem root@YOURCLIENT:/etc/stunnel |
Repeat this for each client, or write a script to do the same thing.
Change File Permissions
Change permissions on certificates and private keys to keep it secure on the server and clients.
| Code: Change Permissions on both server and client |
chmod 600 /etc/stunnel/*.pem |
Necessary Configurations
Without SSL
Client Configuration
Add something like this to /etc/sylog-ng/sylog-ng.conf:
| File: /etc/syslog-ng/syslog-ng.conf |
# where to send the logs.
destination remote {tcp("SERVER_IP_ADDRESS" port(1999));};
# connect your system log sources to the remote server
log {source(src);destination(remote);};
|
Change SERVER_IP_ADDRESS to your server's ip address.
Server Configuration
Now edit your syslog-ng configuration on your server and add something like this:
| File: /etc/syslog-ng/syslog-ng.conf |
options {
...
create_dirs(yes); #will recursively create log files/directories if necessary
};
# The port where the logs will be sent to by the clients
source remote_log {tcp(ip("SERVERIPADDRESS")
port(1999));};
# where to log the files on the server. $HOST is a macro and will be replaced by the hostname sending the log
destination remote {file("/var/log/remote.d/$HOST.log");};
# link the source to the destination
log {source(remote_log); destination(remote);};
|
With SSL
Client Configuration
On the clients open /etc/stunnel/stunnel.conf and add:
| File: /etc/stunnel/stunnel.conf |
client = yes cert = /etc/stunnel/syslog-ng-client.pem CAfile = /etc/stunnel/syslog-ng-server.pem verify = 3 [5101] accept = 127.0.0.1:1999 connect = SERVER_IP_ADDRESS:5101 |
SERVER_IP_ADDRESS is again your syslog-ng server ip address.
Add something like this to /etc/sylog-ng/sylog-ng.conf:
| File: /etc/syslog-ng/syslog-ng.conf |
# where to send the logs. stunnel will forward these to the server.
destination remote {tcp("127.0.0.1" port(1999));};
# connect your system log sources to the remote server
log {source(src);destination(remote);};
|
Server Configuration
On the server side open /etc/stunnel/stunnel.conf and add this lines.
| File: /etc/stunnel/stunnel.conf |
... cert = /etc/stunnel/syslog-ng-server.pem CAfile = /etc/stunnel/syslog-ng-client.pem verify = 3 [5101] accept = SERVER_IP_ADDRESS:5101 connect = 127.0.0.1:1999 |
5101 is the tunneling port and SERVER_IP_ADDRESS is replaced by your syslog-ng server ip address.
Now edit your syslog-ng configuration and add something like this:
| File: /etc/syslog-ng/syslog-ng.conf |
options {
...
create_dirs(yes); #will recursively create log files/directories if necessary
keep_hostname(yes); # will use the hostname provided in the log messages and not the resolved logging client
# this prevents your remote logs from showing up as 'localhost' or '127.0.0.1' when using stunnel
};
# where to find the logs that stunnel will send from the clients.
source remote_log {tcp(ip("127.0.0.1")
port(1999)
max-connections(1));};
# where to log the files on the server. $HOST is a macro and will be replaced by the hostname sending the log
destination remote {file("/var/log/remote.d/$HOST.log");};
# link the source to the destination
log {source(remote_log); destination(remote);};
|
(Re)Starting Services
Restart syslog-ng and stunnel on both the server and clients.
| Code: Restart syslog-ng and stunnel |
/etc/init.d/stunnel restart /etc/init.d/syslog-ng restart |
Now check if logging works:
| Code: The moment of truth |
tail -f /var/log/remote.d/* |
You may have to log into your remote client to generate log-message:
| Code: The moment of truth |
logger "testlog" |
If it doesn't work, check your /var/log/messages. stunnel and syslog-ng are both verbose enough to track any configuration error.
Adding Services to Start at Boot
Now we can add stunnel to the default level:
| Code: Add stunnel to default runlevel |
rc-update add stunnel default |
Additional Syslog-ng Configuration
Up until now, this HOWTO has configured all the logs on the server to be kept in /var/log/remote.d/$HOST.log. This is only a single file and is not the most efficient and organized way to keep logs. The following will show you how to use filters to direct logs from specific processes to specific files.
Instead of a single file, we now want to keep logs in a directory. One directory for each client. We will put the logs in /var/log/remote.d/$HOST/, which is now a directory. But you can put them anywhere you wish. Check out the syslog-ng documentation (in References), specifically the appendix on "Macros" for more ideas on how to organize your logs.
Setting Up Filters
Here is a configuration with some basic filters you may wish to use. Simply add any of these to your syslog-ng configuration.
| File: /etc/syslog-ng/syslog-ng.conf |
...
#Some filters for your convenience.
filter f_auth { facility(auth); };
filter f_authpriv { facility(auth, authpriv); };
filter f_syslog { not facility(authpriv, mail); };
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail); };
filter f_user { facility(user); };
filter f_debug { not facility(auth, authpriv, news, mail); };
filter f_messages { level(info..warn)
and not facility(auth, authpriv, mail, news); };
filter f_emergency { level(emerg); };
filter f_info { level(info); };
filter f_notice { level(notice); };
filter f_warn { level(warn); };
filter f_crit { level(crit); };
filter f_err { level(err); };
filter f_failed { match("failed"); };
filter f_denied { match("denied"); };
|
Setting up Destinations
As previously mentioned, we want all log files in the /var/log/remote.d/$HOST/ directory, where $HOST will be replaced by the hostname of your client. So we need to specify the filenames that the filters will filter to.
Add something like this to your syslog-ng configuration:
| File: /etc/syslog-ng/syslog-ng.conf |
...
#define destinations.
destination remote_authlog { file("/var/log/remote.d/$HOST/auth.log"); };
destination remote_syslog { file("/var/log/remote.d/$HOST/syslog"); };
destination remote_cron { file("/var/log/remote.d/$HOST/cron.log"); };
destination remote_daemon { file("/var/log/remote.d/$HOST/daemon.log");};
destination remote_kern { file("/var/log/remote.d/$HOST/kern.log"); };
destination remote_lpr { file("/var/log/remote.d/$HOST/lpr.log"); };
destination remote_user { file("/var/log/remote.d/$HOST/user.log"); };
# Should be remote_maillog (Without dot) as it was the default on logwatch
destination remote_mail { file("/var/log/remote.d/$HOST/maillog"); };
destination remote_mailinfo { file("/var/log/remote.d/$HOST/mail.info");};
destination remote_mailwarn { file("/var/log/remote.d/$HOST/mail.warn");};
destination remote_mailerr { file("/var/log/remote.d/$HOST/mail.err");};
destination remote_newscrit { file("/var/log/remote.d/$HOST/news/news.crit");};
destination remote_newserr { file("/var/log/remote.d/$HOST/news/news.err");};
destination remote_newsnotice { file("/var/log/remote.d/$HOST/news/news.notice");};
destination remote_debug { file("/var/log/remote.d/$HOST/debug");};
destination remote_messages { file("/var/log/remote.d/$HOST/messages"); };
|
Connect the Source, Filter, and Destinations
Now we just tell syslog-ng which destinations each filter will put its contents. Add something like this to your syslog-ng configuration:
| File: /etc/syslog-ng/syslog-ng.conf |
...
#connect filter and destination
log { source(remote_log); filter(f_authpriv); destination(remote_authlog); };
log { source(remote_log); filter(f_syslog); destination(remote_syslog); };
log { source(remote_log); filter(f_cron); destination(remote_cron); };
log { source(remote_log); filter(f_daemon); destination(remote_daemon); };
log { source(remote_log); filter(f_kern); destination(remote_kern); };
log { source(remote_log); filter(f_lpr); destination(remote_lpr); };
log { source(remote_log); filter(f_mail); destination(remote_mail); };
log { source(remote_log); filter(f_user); destination(remote_user); };
log { source(remote_log); filter(f_mail); filter(f_info); destination(remote_mailinfo); };
log { source(remote_log); filter(f_mail); filter(f_warn); destination(remote_mailwarn); };
log { source(remote_log); filter(f_mail); filter(f_err); destination(remote_mailerr); };
log { source(remote_log); filter(f_debug); destination(remote_debug); };
log { source(remote_log); filter(f_messages); destination(remote_messages);};
|
Make Sure Syslog-ng Will Create Directories and Files
It would be a pain to create all of these directories and files manually, so make sure that in the options part of /etc/syslog-ng/syslog-ng.conf has the following option set: create_dirs(yes).
What To Do Now
You may want to look into logrotate, which will automatically move, compress, and eventually delete old logs depending on your configuration. This will make sure you don't eat up your hard disk space on your server.
References
Created by NickStallman.net, Luxury Homes Australia
Real estate agents should be using interactive floor plans and list their apartments, townhouses and units.