Openfire an easy to use IM Server

Openfire (formerly wildfire) from "Ignite Realtime" is a free and easy to install IM server based on XMPP protocol which is available for Windows, Linux and Mac platforms.
I have tried the windows version and it worked nice. Its client is named "spark" and there is a web-based client (Sparkweb) available as well.
I got to this simple and easy IM server while I was searching for a Linux client for Microsoft Live Communication Server. I have crawled a lot a of forums and after all it seems that there is no solution to that in the Linux world yet. Some posts on using Wine was available but the people who tested it didn't come out with a positive result.

Secure Your Apache

Once you start searching for a topic like "Securing Apache" or "Hardening Apache" you will get hundreds of results and everyone tries to set out his own security concerns. The fact is that not every recommendation applies to our environment but we need to study and take into consideration all possible approaches to secure our web server. Some of these guides are too much complicated and strict and some too mild!
The following is a very basic and reasonable list of things we have to do to bring minimum security to our Apache server. Of course, Server Hardening comes first!

1. Hide the Apache Version number, and other sensitive information.
2. Make sure Apache is running under its own user account and group.
3. Ensure that files outside the web root are not served.
4. Turn off directory browsing.
5. Turn off server side includes.
6. Turn off CGI execution.
7. Don't allow apache to follow symbolic links.
8. Turn off support for .htaccess files.
9. Run mod_security.
10. Disable any unnecessary modules.
11. Make sure only root has read access to apache's config and binaries.

For technical details on these and more steps follow the bellow link:
20 ways to Secure your Apache Configuration

JPGraph Error

I asked a colleague of mine to set up a PHP-Syslog-NG (http://code.google.com/p/php-syslog-ng/) as the central logging system of all our devices and servers. He did this on a FreeBSD 7.0 and everything started working fine but the "Graph" section which works with JPGraph couldn't draw any Graphs and it came up with this message: "JpGraph Error Font file "/usr/share/fonts/truetype/msttcorefonts/verdana.ttf" is not readable or does not exist..
After some digging into the codes and configs I got over the problem following the bellow steps:

1. Install TrueType font on FreeBSD (XfStt).
2. Fetch the "verdana.ttf" font and place it in the directory that XfStt created for TrueType fonts.
3. Change the default "TTF_DIR" parameter.

Step 1:
An easy way to use TrueType fonts in BSD is to install XfStt which is available through ports collection at "/usr/ports/x11-servers/Xfstt/".
After the installation a directory will be created for TrueType fonts at "/usr/local/lib/X11/fonts/TrueType/".
Step 2:
downloading verdana.ttf and placing it in "/usr/local/lib/X11/fonts/TrueType/".
I fetched my copy from "http://www.afosteo.org/Download/Fonts/"
Step 3:
The final step is to point JPgraph TTF_DIR parameter inside the jpg-config.inc configuration file to to proper location.
jpg-config.inc is located at "/usr/svr/php-syslog-ng/html/includes/jpgraph"

Make Putty Tab Based!

Everyone knows Putty well but I always prefer to use SecureCRT because I can open different connections in a tab based manner, I can easily clone my sessions in case I need more than one session to the same device mainly for debugging or diagnostic reasons and I can save my connections into a database which easily can be backed up (Putty stores the sessions into registery!).
The good part about Putty is free, it is handy and requires no installation!
Well there are always great people out there to make things work better and thanks to Ramesh I have leaned that there is a free add-on called "Putty Connection Manager" which does all the job I pointed out earlier. You have your Tabbed based interface, You can clone your sessions and everything is stored in a database and above all it makes putty look more modern ;)
There are many other useful add-ons for Putty and you can learn about them through the following link:
The Geek Stuff » Turbocharge PuTTY with 12 Powerful Add-Ons - Software for Geeks #3:

TCPDUMP Tutorial

TCPDUMP is a wonderful command line tool which helps analyzing and troubleshooting network traffic on a Linux host.

The tcpdump options I use the most are:

• -n : Don't resolve hostnames.

• -nn : Don't resolve hostnames or port names.

• -v, -vv, -vvv : Increase the amount of packet information you get back.

The followings are the most tcpdump expressions I use:

Display any traffic souring and destining a specific host:

• tcpdump host "Host Address"

Display any traffic sourcing a specific host:

• tcpdump src "Host-address"

Display any traffic destining a specific host:

• tcpdump dst "Host-address"

Display any ICMP traffic:

• tcpdump icmp

Display traffic sourced or destined a specific network:

• tcpdump net "net-address"

Display any traffic sourcing or destining a specific port:

• tcpdump port "port-number"

Display any traffic sourcing a specific port:

• tcpdump src port "port-number"

Display any traffic destining a specific port:

• tcpdump dst port "port-number"

It is also possible to use "AND", "OR", and "Excpet":

You can learn more about tcpdump options and expressions with great examples at this location: http://dmiessler.com/study/tcpdump/

Dig DNS Lookup in Windows!

Every System Administrator dealing with DNS administration knows the power of "dig" command-line tool in Linux/Unix environment. But there are times when an administrator needs to monitor and troubleshoot DNS from a Windows station and she would then realizes the deprivation!
The good news is that many Linux/Unix tools have been ported to windows (Check my post about Windows IPFW) and one of them is the "dig" utility.

The windows version of "dig" can be downloaded and installed from here: http://members.shaw.ca/nicholas.fong/dig/


For those who are new to "dig" the following link helps:
Dig Howto: http://www.madboa.com/geek/dig/

Fighting Spam with Barracuda Spam Firewall

It's been a couple of weeks since I started working on a dedicated solution to fight incoming spam and I did some study on the history and mechanisms available to block spam and which methods are efficient for an Internet Data Center. I was thinking of implementing Spamassassin using Qmail as MTA, but our company policy has changed and now I am considering an Anti-spam Appliance. There are many appliances available and most of them have other security features like Firewalling, Virus scanning and DoS protection.
Spam Firewall from Barracuda Networks seems a great option to me with different mechanisms including Bayesian and great control over what should be considered spam and non-spam (ham) and provides a good visibility to what is happening on the box by its statistics. Its control panel has got a lot of screens which might bring complexity but it also gives great control and make any policy enforcement possible!
I am really impressed with the administration options and I like to get my hands on it!
Spam Firewall is actually a Linux platform running spamassassin in its core.

How to Implement Source Routing With Linux

As mentioned in my previous post I got an Internet gateway which is a Linux box and I have two Internet connections connected to that server. One is a 2Mbps Leased-Line and the other a 1Mbps wireless connection. I want hosts from specific subnets have their Internet traffic directed to the wireless Internet connection while others go through the Leased-Line link.

This is easily done with Linux and iproute2 suit which is installed by default on Fedora.

By default all routes are stored in a table called "main" and by issuing the following command the routes stored inside this table can be displayed:

• ip route list table main
  

The results are exactly that same as just running the "route" command.

Any queries coming to this server for routing decisions will be looked up in the "main" table unless mentioned otherwise. But how is this possible?

It is also possible to define a new routing tables and have different routing entries inside the new defined table and apply rules so that traffic from specific sources are directed to this new table for route look up!

First:

we need to create a new table which easily handled by adding the name at the end of /etc/iproute2/rt_table. It may look like this:

10 wireless-link

Second:

New routes should be added to this table:

• ip route add 192.168.120.0/24 via 192.168.10.1 table wireless-link
• ip route add default via 80.120.99.12 table wireless-link (This defines the default route for "wireless-link" routing table)
• ip route list table wireless-link (This will display routes added to wireless-link)

Third:

Route rules must define when requests must be looked up in the "wireless-link" table!

• ip rule add from 192.168.120.0/24 table wireless
• ip rule list (display ip rules)

From now on, every traffic coming from 192.168.120.0/24 will be leaded to wireless-link table so its default route will be 80.120.99.12 while traffic from other sources will be still lookup routes in the "main" table which has its own default route (Leased-Line).

To undo ip rules and routes the following syntax must be followed:

• ip rule del from 192.168.120.0/24 table wireless
• ip route del default via 80.120.99.12 table wireless-link

Route Policy With Linux

In one of our premises I have two firewalls; A Linux iptable and a Microsoft ISA Server.
I got two Internet connections, each connected to one firewall and the plan is to remove the ISA server and add the Internet connection (currently servicing ISA server users) to the Linux box which makes two internet connections on the same server. I got around 20 VLANs and I want to split Internet traffic between these two connections based on the source address. This can easily be done by iproute2 suite. I have not done this before but I am studying it and it seems easy and straightforward. I will post more on this later.

Hardening Linux: Service Shutdown!

I am developing a security guide to provide recommendation to harden a Linux box after a base installation. One of the basic steps to harden a Linux box is to evaluate all Services and shutdown anything that is not necessary for the operation of the operating system and services it offers to its clients. Shutting down Services provides the following benefits:

• Decreases boot up process resulting in higher uptime
• Less system resources are consumed which increase the overall performance of server
• Removes / reduces the risk of any Service vulnerability or abuse

To accomplish this you should know about the functions of each and every Service. There are different documents available to explain this. Here I have found something that helped me a lot because my document is based on Redhat I have found this matching my requirements the best:
Understanding Your Redhat Enterprise Linux Daemons

How To Set Up A Linux Syslog Server

Any network administrator in charge of a few network devices would like to keep a record of the events on these devices like:

• who logged into devices (or tried to connect to network devices)
• when does he/she try to login
• what did she/he do after login
• what changes and events have been announced on the device (like interface status change)

And lots of other information that might give a clue about a policy violation or tracking a series of events that led to a network disruption incident.
What I am talking about is to have a simple SYSLOG server in place to collect all log messages to a central location.
If you need to setup a quick and easy syslog server just follow the link bellow. It is meant for Debian but will work on almost all Linux distributions :
Linux Syslog Server - How To Set Up A Debian Linux Syslog Server

Linux Traffic Control, DNS ALG issue , and Service Monitoring

I am busy with a couple of interesting stuff that keeps me away from posting here.
First,
I am working on a linux box which is an internet gateway and controls traffic using iptable and at same time it acts as the inter VLAN router for around 10 VLANs using 802.1Q. I am trying to run some sort of QoS to put traffic control on every VLAN for their internet usage. And with Linux this will be easily done with a tool called TC.
I have expressed that before but it is worth to mention again that with linux we will get tons of outstanding networking features that gives us full control on our network and what is running behind the scene on the wires! I am really impressed!!!
Second,
My firewall is not handling DNS ALG as expected so it has brought us some disturbance. Everything is fine with outside to inside regular DNS queries but when a DNS query for PTR record comes in, DNS ALG does not translate the IP address in the response payload while it does so when it comes to Forward queries. According to RFC 2694 this should work unless our Firewall is not RFC compliant. I am documenting the issue regarding RFC to send it over to our Firewall Vendor.
Third,
I was studying available service monitoring tools to monitor our IDC services and servers and nearly reached to the conclusion to implement "ManageEngine Applications Monitror" and I already have a pilot server in place. I will post on it later.

How to Add Persistant Static Routes in Linux

At times, when I work on my linux box I forget about the configuration file game and expect some tasks to be completed just by putting some commands in the line!
This is the second time that I forget adding my static routes to the config file and wake up after my server needs a reboot and things start going wrong afterwards!
The easy way, which works in any distribution is to simply add routes in /etc/rc.local but this is not welcomed by many professionals:
route add -net 192.168.125.0 netmask 255.255.255.0 gw 192.168.110.1
route add -net 192.168.145.0 netmask 255.255.255.0 gw 192.168.110.1

But to do it properly in Redhat and Fedora distributions we have to create a configuration file for each interface. For example, for all routes that need to go out from "eth1" a config file named "route-eth1" must be created in "/etc/sysconfig/network-scripts/route-eth1" containing the following:
(I will take the above route as example)

GATEWAY0=192.168.110.1
NETMASK0=255.255.255.0
ADDRESS0=192.168.125.0

GATEWAY1=192.168.110.1
NETMASK1=255.255.255.0
ADDRESS1=192.168.145.0

So if there are different interfaces that correspond to different routes we should expect config files like "route-eth0", "route-eth1","route-eth2" in "/etc/sysconfig/network-scripts/"

MySQL table crashed again on Syslog server!

A couple of days ago we were having a downtime so I had to shutdown all my servers including Syslog server which is a php-syslog-ng server, as mentioned in my previous posts. But after it came back online no logs were collected anymore and by closer examination I found out that the "logs" table in MySQL has been marked as crashed and automatic repair didn't help either(Through mysqld log files and "check table [tablename]" through mysql client).
I used "myisamchk" tool which is used to check and repair MyISAM tables as follow:
[root@logcollector]# myisamchk -rf logs.MYI (tried the quick mode first but not helpful)
It took eight hours to check and fix the indexes!
This was the second time that my syslog-ng server went down for about 2 days because of a failure in MySQL table. The first time a sudden and unexpected shutdown brought about the table crash but this time was different. I am doubting that mysqld couldn't shutdown its databases safely and that might be because there are more than 150 devices sending their messages to this server. Checking my past rotated syslog databases I see that each month I have a database of 15 to 20 GB in size. I am thinking of some way to split my database into different parts to make it easier to handle.

Firewalling Windows Servers with IPFW

For some time I spent thinking of an IPTable like firewall for host protection on windows servers and I didn't know whether there is one available until I gave it a try and surprisingly found WIPFW. I find it very cool to have a unix based firewall on my windows boxes!
WIPFW is the Windows version of FreeBSD IPFW firewal and It can be used on any version of windows, starting with windows 2000.
It gives a lot of flexibility in the way rules can be applied to different sort of traffic and it can also keep track of the states of packets as well. There are a lot of great features that comes with it and it can be checked through its online documentation. Any IPTable administrator can figure it out quickly.

Some missing features in its current release(0.2.8) are as follow:

• Unable to change packet contents
• No traffic shaping capabilities
• Does not support SNAT and DNAT

Check here for documentation and product download:
WIPFW: Windows Operable Version of BSD IPFW

Check also here for more info on WIPFW:
Jameser's Tech Tips: Stateful Packet Filter for Windows

Learn more about the original IPFW:
ONLamp - BSD Firewalls: IPFW
IPFW How-To

Got problem with Linux memory usage?

If you are coming from the windows world you might wonder about a lot of things while taking your walk into Linux world, as I did. One of the odd things is the way that Linux handles memory. Once a Linux server is up for a while you can notice that the system has eaten up all the available memory, and doesn't let a bit of it go free and you might wonder why you have left less than 10MB of free memory out of 1GB using the "Top" command!
First time I noticed this, it was on a Fedora core 5 with Apache and MySQL so I thought it is a database server thing that also happens on MS-SQL servers but got surprised when I came across the same issue on a different server with the same OS acting as an Internet Gateway, also handling the Inter-VLAN routing for 6 VLANs with small number of clients in each one.
The point is that unless your server is not using its swap space, it is OK. To find out how much memory is actually available, use the "free" command and look at the "-/+ buffers/cache" row.
To figure out how Linux handles memory and relieve the confusion, check the following link:
Linux Memory Management

Syslog Message Header Format

Finally I fixed my problem with huawei syslog messages! Actually I found a work around to it on my Syslog-NG server. There is a bad_hostname("regexp") directive in syslog-ng configuration file which can be used to ignore wrong hostnames extracted from syslog header messages if specified, and the cool thing is that you can use regular expresions to specify a range of hostnames matching your criteria. Once a bad_hostname matched syslog-ng will look into the IP header instead for the source address and places this address as the hostname.

But to realize what exactly the problem was I decided to take a close look into RFC 3164. Section 4.1.2 says:

• The HEADER contains two fields called the TIMESTAMP and the HOSTNAME.
• The TIMESTAMP field is the local time and is in the format of "Mmm dd hh:mm:ss" (without the quote marks)
• A single space character MUST follow the TIMESTAMP field
• The HOSTNAME field will contain only the hostname, the IPv4 address, or the IPv6 address of the originator of the message. The preferred value is the hostname
• Example: <34>Oct 11 22:14:15 mymachine su: 'su root' failed for lonvick on /dev/pts/8

It has firmly defined how a syslog message format should be and to see how Huawei is complying with this format I have captured traffic from huawei routers and switches with tcpdump:
>Oct 28 16:15:03 2006 SHZ-C2H-Router IFNET/5/UPDOWN:S[|syslog]

OK! instead of the HOSTNAME stgraight after the TIMESTAMP I see "2006" which is the YYYY part of the current date on huawei devices!
Anyway, as soon as the bad_hostname directive is there no need to bother for this kind of problem!

Huawei Devices and Strange Syslog Message Format!

My php-syslog-ng is up and running and I am in the process of configuring all my nodes to send their syslog messages to syslog-ng server but there seems to be some problems on some devices. No problem with my Linux servers and my Huawei firewalls but Huawei switches and routers are apparently using the wrong syslog message format. I can see all my switches and routers as a host named "2006" which is the YYYY part of the date in my php-syslog-ng web interface! and the hostname appears in the message string which is different from what RFC 3164 says about the syslog packet header. It seems that Huawei managed to have its own syslog message format on some of their devices. I am looking for some way to change this behavior.

Does Windows Support SNMP v3?

Moving towards SNMP v3 needs some thinking before the final decision to make sure all parties included in the migration are capable of that! This includes the Managers and Agents.
Managers can be a major concern since many of them do not support SNMP v3 nodes which can cause the whole move to collapse in the planning phase!!!
Agents includes all Devices like Switches, Routers, Firewalls or operating systems (Windows and unix/linux family). Many devices come with SNMP v3.
For Unix and Linux servers the Net-SNMP provides a complete solution for the move but for Microsoft Windows servers it should be mentioned that the SNMP service which is built into the operating system does not support SNMP v3 but luckily there is an alternative which is installing the Windows version of Net-SNMP:
Installing Net-SNMP on Windows

Setting up Syslog-ng in Fedora Core 5

I am busy with setting up a php-syslog-ng server on Fedora Core 5 these days but things are really getting harder than I imagined! Having the syslog-ng service itself up and running is simple and straight forward but adding the opportunity to monitor the logs through a web interface; which is what I really need has made my life difficult!
The difficulty is where different details from Apache, MySQL and syslog-ng configuration file must be adjusted.
I am documenting the process as I go on and will put it here once completed.

Nessus a handy tool for attackers!!!

The most significant incidents on my IDS/IPS sensor in the past two weeks were the logs which showed someone is trying to find out about any vulnerability on our corporate website and portal and more interestingly, this guy didn't care about any customer websites and was seriously working on our corporate webserver which has many meanings to me.
He/She was using NESSUS to scan our server and find out about any vulnerability on our operarting system, web server and Email System.
When fronting yourself with these cases the worst part is that you can not take any legal action against these guys! You may block their source IP address temporarily but at the end you can only carefully watch their activity and pray!!!