Tests of Sygate

Key criteria in choosing a personnal firewall are :

• Effectiveness of security protection : penetration, Trojans, controlling leaks, denial of service.

• Effectiveness of intrusion detection: few false positives, alerting of dangerous attacks.

• User interface: ease of use, instructiveness, simplicity, quality of online help. Does the interface suit the way you use your PC ?

• Price.

How did we test firewall/intrusion detection effectiveness?

1. Ping and accessing shares to and from the test host.

2. A powerful, well known "remote control" Trojan (Netbus Pro v2.1 [2]) was installed on the system on a nonstandard port (to make detection more difficult), the Netbus server started and attempts made to connect from a remote system.

3. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sT -P0 -O IP_ADDR).

4. An nmap [1] scan was run, to check that incoming ports were effectively blocked. With another local PC launching nmaps againts the test PC and the following options (nmap -v -sP -P0 -O IP_ADDR).

5. A test using Leaktest [4] was done.

6. We checked the system ressource usage of the firewall during the tests (just in case).

7. We tried to launch a modified (by us) release of IEXPLORE.EXE (C:\Program Files\Internet Explorer\IEXPLORE.EXE ) to check if the firewall detects the problem.

NB : These tests do not pretend to be exhaustives. By the way the aim is to be sure that the tested software offers at least expected security (or not) for a personnal use (do not compare this to professional use).

Jump to the test results.

The sygate firewall 2.475 [3] is full of interesting features :

• Possibility to allow/disallow some applications to connect to the net.

• Possibility to allow/disallow some services (ports) .

• Possibility to allow/disallow some protocols.

• Possibility to define trusted IP addresses.

• Possibility to send email notifications (when attacked).

• Possibility to define time range for blocking all traffic (when asleep for example).

• Possibility to define a password protection (such as in Conseal)

• Download size : 2.6 MB

1. Ping: Impossible if you unchecked the Allow ICMP message type and/or the Allow Echo Reply in the Advanced ICMP Settings. This is a good result.

2. The Netbus server: sygate 2 does not detect the Netbus server when started. But connexions to the netbus server are impossible. The result of this test is good.

3. An nmap scan without sygate 2 (on Win 2000 OS SP1 with a "standard" installation, it means NetBios active and so on) :
   
   $ nmap -v -sT -P0 -O IP_ADDR
   
   Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
   Initiating TCP connect() scan against (IP_ADDR)
   Adding TCP port 445 (state open).
   Adding TCP port 135 (state open).
   Adding TCP port 1025 (state open).
   Adding TCP port 913 (state open).
   Adding TCP port 139 (state open).
   
   The TCP connect scan took 0 seconds to scan 1523 ports.
   
   For OSScan assuming that port 135 is open and port 1 is closed and neither are firewalled
   
   Interesting ports on (IP_ADDR):
   (The 1518 ports scanned but not shown below are in state: closed)
   Port State Service
   135/tcp open loc-srv
   139/tcp open netbios-ssn
   445/tcp open microsoft-ds
   913/tcp open unknown
   1025/tcp open listen
   
   TCP Sequence Prediction: Class=random positive increments
   Difficulty=6634 (Worthy challenge)
   
   Sequence numbers: 747E9CE8 747F63FC 74800BF5 7480E3FE 7481BC4F 7482B3B2
   
   Remote operating system guess: Windows 2000 RC1 through final release
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 10 seconds
   
   Gloups : you'd better have a firewall installed :+) !!!
   
   An nmap TCP scan with sygate 2.475 (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) with options security level Medium activated gives events registered in the log which is a good result for detection (Medium because at upper level you are unable to run properly IE or Outlook or whatever) :
   
   $ nmap -v -sT -P0 -O IP_ADDR
   
   Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
   Initiating TCP connect() scan against (IP_ADDR)
   Adding TCP port 1025 (state open).
   The TCP connect scan took 122 seconds to scan 1523 ports.
   For OSScan assuming that port 1025 is open and port 1000 is closed and neither are firewalled
   Interesting ports on (IP_ADDR):
   (The 999 ports scanned but not shown below are in state: filtered)
   Port State Service
   1000/tcp closed cadlock
   1001/tcp closed unknown
   1002/tcp closed unknown
   1003/tcp closed unknown
   [...] hundred of remote closed port detected
   32787/tcp closed sometimes-rpc27
   43188/tcp closed reachout
   47557/tcp closed dbbrowse
   65301/tcp closed pcanywhere
   
   TCP Sequence Prediction: Class=random positive increments
   Difficulty=8711 (Worthy challenge)
   
   Sequence numbers: B5119B54 B5122941 B512EEDE B51399E4 B51479E0 B5156457
   
   Remote operating system guess: Windows 2000 RC1 through final release
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 123 seconds
   
   This means that with these options too many ports remains visibles even if access attempts are logged, and that it remains possible to guess wich OS !! This is a bad result.
   

4. An nmap UDP scan with sygate 2 (on Win 2000 SP1 OS with a "standard" installation, it means NetBios active and so on) gives events registered in the log which is a good result for detection :
   
   $ nmap -v -sU -P0 IP_ADDR
   
   Starting nmap V. 2.53 by fyodor@insecure.org ( www.insecure.org/nmap/ )
   Initiating FIN,NULL, UDP, or Xmas stealth scan against (IP_ADDR)
   The UDP or stealth FIN/NULL/XMAS scan took 90 seconds to scan 1448 ports.
   Interesting ports on (IP_ADDR):
   (The 455 ports scanned but not shown below are in state: closed)
   Port State Service
   1/udp open tcpmux
   2/udp open compressnet
   3/udp open compressnet
   4/udp open unknown
   [...] thousand of remote ports detected
   996/udp open vsinet
   997/udp open maitrd
   998/udp open puparp
   999/udp open applix
   
   Nmap run completed -- 1 IP address (1 host up) scanned in 90 seconds
   

   This means that the security seems totally un-efficient for UDP at security level High and Medium. This is a bad result.

5. The Leaktest : The result is good at High security level (where it's impossible for most application to connect), leaktest go through when at Medium (or lower) . The result of this test is bad.

6. sygate 2.475 goes up to 70% of CPU during the heavy portscans :-). In normal operations it uses up to 4% max. Memory usage is 9.1 MB, up to 11,8 MB peek (both service and application cumulated).

7. The substitution test : (you can do it by yourself for example : you replace Iexplorer.exe with leaktest.exe - yes this one - by renaming the last one and launch it). The result is sygate allow the trojan horse to connect... the result of this test is bad.

1. sygate can be configured to ignore ping (from any sources).

2. Allows to specify some port/ protocol specific rules

3. Allows to warn through email.

1. sygate cannot makes any difference between local network connexions and Internet connexions. In fact it seems impossible to specify which interface to protect.

2. The security seems either difficult to configure (set) either too low.

• Improve all the security efficiency !

• Product internationalization.

A good start but needs much more security and work to be really usefull.

Evaluation :

• Installation process (2) : 15/20

• Configuration , GUI (3) : 5/20

• Filtering security (5) : 0/20

• Additionnal security (3) : 0/20

• Software load and memory usage (2) : 12/20

• Import/Export configuration (2) : 0/20

• Help , FAQ (2) : 10/20

• Product internationalization (1) : 0/20

Total : 4.45 / 20

Note : the result may be modified with the release , and when adding new criteria or re-evaluating their weight or their content.

1. Nmap - Network mapper, a really efficient tool to check networks
   http://www.insecure.org/nmap

2. Netbus Pro - Remote control program often used as an attack tool to control remote PCs.
   http://www.netbus.org/
   download

3. Sygate 2 firewall
   http://www.sygate.com

4. Leaktest - Small testing software written by Steve Gibson to check firewalls. It makes a simple TCP (ftp) connexion that simulate sennding of personnal content, which can also be used to take remote controle in reverse mode (arg).
   http://grc.com/
   download