*Last Updated:* Monday 22nd of July 2002 10:56:45 AM GMT
SourceForge Logo <http://sourceforge.net>
URL: http://qmail-scanner.sourceforge.net/
<http://qmail-scanner.sourceforge.net/>


  Qmail-Scanner: Content Scanner for Qmail
  <http://qmail-scanner.sourceforge.net/>

/Copyright 2000/2002 Jason Haar. This software is distributed under the
terms of the GNU General Public License. See COPYING for additional
information. /


    Description

*Qmail-Scanner*, (also known as scan4virus) is an addon that enables a
Qmail Email server to scan all gatewayed Email for certain
characteristics (i.e. a content scanner). It is typically used for its
anti-virus protection functions, in which case it is used in conjunction
with commercial virus scanners. but also enables a site (at a
server/site level) to react to Email that contains specific strings in
particular headers, or particular attachment filenames or types (e.g.
*.VBS attachments). It also can be used as an archiving tool for
auditing or backup purposes. Qmail-Scanner is integrated into the mail
server at a lower level than some other Unix-based virus scanners,
resulting in better performance. It is capable of scanning not only
locally sent/received Email, but also Email that crosses the server in a
relay capacity.


    Features

    * Uses any commercial Unix command-line virus scanner.
    * Can call more than one virus scanner for each mail message
    * Has its own internal scanner that can be used to pick up viruses
      for which commercial scanner updates are not yet available
    * The internal scanner can also be used to block Email based on
      attachment types, or Email with certain Email headers... Need to
      stop *.mp3 files or "*Subject:* ILOVEYOU" Email getting onto and
      off your LAN - can do! :-)
    * Can integrate with SpamAssassin to provide comprehensive anti-spam
      tagging for an entire site
    * Auto-detects Email from "postmaster"-style and mailing-list
      addresses - and doesn't send virus reports to them (i.e. attempts
      to act more like a responsible net citizen)
    * Each message is tagged via a new *Received:* header with a virus
      report showing whether it is clean or not and virus scanner
      version numbers/etc
    * Messages with viruses are moved into a "maildir" mail folder for
      later perusal by the appropriate staff
    * Can optionally add a descriptive header: *X-Qmail-Scanner* to
      every Email that passes through the system to allow users to see
      that a scanner has run over their messages
    * Messages caught by Qmail-Scanner generate an Email message
      (currently supports English, Italian, German, Spanish, Turkish,
      Lithuanian, French, Portuguese and Chinese messages) to a
      configurable combination of the sender, recipients and a
      "quarantine-admin" address explaining why their message was rejected
    * Can archive all processed Email into an archive maildir. Useful
      when debugging Email-based apps, for backup purposes and for audit
      policy reasons. Currently the mail envelope headers (the "rcpt
      to:" and "mail from:" headers) are appended to the bottom of each
      message.
    * Can report via syslog or to a file, a one-line description of each
      processed message, giving information such as subject line,
      attachment filenames, sizes, etc.
    * Redundant scanning. Not only does it unpack each message before
      running the scanners over it, it can also scan the original Email
      message as well as the unpacked messages (if you think a
      particular scanner can do a better job than /Qmail-scanner/'s
      internal systems allow.)


    Download

The latest release is 1.13 (via http)
<http://prdownloads.sourceforge.net/qmail-scanner/qmail-scanner-1.13.tgz>,
and is kindly housed by SourceForge <http://sourceforge.net/>. GnuPG
signature of qmail-scanner-1.13.tgz
<http://prdownloads.sourceforge.net/qmail-scanner/qmail-scanner-1.13.tgz.asc>
is also available. Of course, you'll be needing my GPG Public Key
<http://qmail-scanner.sourceforge.net/jhaar@users.sourceforge.net.gpg>
to verify that.


    Requirements

    * Qmail 1.03 (there's a patched src RPM
      <http://untroubled.org/qmail+patches/> for Linux users available
      that contains the QMAILQUEUE patch amongst other things - just
      "rpm --rebuild" as root to build your own i386.rpm. NOTE: I cannot
      vouch for it - I do not use it. Please ensure you know how it
      works before installing Qmail-Scanner.)
    * *reformime* from Maildrop 1.3.8+
      <http://download.sourceforge.net/courier/>
    * Perl 5.005_03+
    * Perl module Time::HiRes
      <http://search.cpan.org/search?module=Time::HiRes>
    * Perl module DB_File <http://search.cpan.org/search?module=DB_File>
      (most distributions come with it pre-installed, although the
      latest Perl doesn't)
    * Perl module Sys::Syslog
      <http://search.cpan.org/search?module=Sys::Syslog> (most
      distributions come with it pre-installed)
    * *Barely Optional:* Mark Simpson's TNEF unpacker
      <http://world.std.com/~damned/software.html>. Can decode those
      annoying MS-TNEF MIME attachments that Microsoft mail servers just
      /love/ to use. If you don't have this, there are several classes
      of Email that you basically won't be able to detect viruses in.


      Patches

Bruce Guenter's QMAILQUEUE <http://www.qmail.org/qmailqueue-patch> patch
is required to enable Qmail to call a different qmail-queue program than
the one compiled in by default. /Qmail-scanner/'s
*qmail-scanner-queue.pl* perl script is used instead of Qmail's
*qmail-queue* binary. After *qmail-scanner-queue.pl* has run, it calls
the original *qmail-queue* binary to resubmit the message back into the
system.

    * *Note 1:* Bruce Guenter also provides a patched qmail-1.03 RPM
      <http://untroubled.org/qmail+patches/> for Linux systems that
      contains the above patch plus a bunch of other bits and pieces.
      Linux users may find it easier to use that. As mentioned above,
      please ensure you have Qmail working before installing
      Qmail-Scanner. On the Qmail-Scanner mailing-lists, we are seeing
      too many cases of users having "problems" - which end up being
      Qmail configuration/understanding issues...


      Supported Virus Scanners

The following virus scanners are known to work with qmail-scanner. Other
Unix-based scanners should be simple to add support for.

    * Trend's Virus scanner <http://www.antivirus.com/>
    * Sophos's "sweep" virus scanner <http://www.sophos.com/>
    * H+BEDV's antivir scanner <http://www.hbedv.com/>
    * Kaspersky's AVPLinux scanner <http://www.kaspersky.com/>
    * MacAfee's (NAI's) virus scanner <http://www.nai.com/>
    * Command's virus scanner <http://www.commandcom.com>
    * F-Secure Anti-Virus scanner <http://f-secure.com/>
    * F-Prot Anti-Virus scanner
      <http://www.f-prot.com/f-prot/products/fplin.html>
    * InocuLAN Anti-Virus scanner <http://www.cai.com/>
    * Clam Antivius <http://clamav.elektrapro.com/> - an Open Source
      antivirus scanner
    * RAV Antivirus <http://www.rav.ro/>
    * Sophie: Daemon front-end to Sophos Sweep
      <http://www.vanja.com/tools/> (see FAQ <FAQ.php> for details)
    * Trophie: Daemon front-end to Trend iscan
      <http://www.vanja.com/tools/> (see FAQ <FAQ.php> for details)
    * Spam Assassin Daemon <http://spamassassin.org/> (see FAQ <FAQ.php>
      for details)


    CHANGES

There is a separate page <CHANGES> listing changes that have been made
between releases


    TODO

There is a separate TODO <TODO.php> page.


    FAQ

There is a separate FAQ <FAQ.php> page.


    Performance/Resource Usage

Adding content/virus scanning to an Email server /will/ considerably add
to the resource usage of that server. As this "wrapper" is written in
perl instead of low-level C, quite a lot of memory and file opens/stats
occurs just to get it going. Adding to this the actual scanners memory
and CPU usage and it becomes quite complicated (certainly the debugging
info shows that the scanner harness spends more time running the
commercial scanners than it does doing things itself [that is to be
expected as they do quite a lot of thinking...]).

As a "rule of thumb" I'd suggest you look at how many simultaneous SMTP
sessions you are willing your box to have going at any one point in
time. Each SMTP session can invoke up to 'n' different virus scanners
(although they run one after the other - not simultaneously) and I'd
estimate that leads to around 5-6Mb of memory usage per SMTP session.
Thus if your dedicated SMTP host has 256Mb RAM + 256Mb swap - that
should mean you can handle - well heaps ;-) The scanners cause the CPU
to be thrashed while they're running, so I'm making sure for our site
that our Qmail server will only accept up to 30 incoming SMTP sessions
at any one time - that way I know the box will handle it. As this leads
to an increased memory usage, don't forget Qmail's memory limits will
need to be increased to deal with it (set via ulimit or softlimit calls
with Qmail system startup scripts).

One thing you should test for is what happens if connectivity between
this server and another local SMTP server is down for any length of time
(due to failure/power outage). When the link is restored, can your
server handle the other trying to dump 1,000's of Email msgs onto it at
once? You need to use softlimit and tcpserver's limit options to ensure
your box doesn't get killed. Note that this resource issue isn't caused
by Qmail-Scanner. The same thing will happen with a pure, untouched
Qmail (or any other) system - it will just happen sooner...

After that scare-mongering I should say that I have tested Qmail-Scanner
under ridiculously low resource conditions - and it reacts as it should
- so at worst your system should start deferring Email. Thankfully DJB's
layering of programs is such that this is easy to accomplish :-)


    Installation

    * Unpack /Qmail-Scanner/ and run *./configure --help*. This will
      show you what options <configure-options.php> are available to you.
    * Run *./configure ...* [with your options], it will autodetect what
      software is installed on your system, and will generate a script
      specific to your system. If you don't see any errors reported,
      then the build is (probably) successful.
    * Run *./configure* again, this time include "/--install/" along
      with the options you chose, this will do the same as the previous
      line, but will also create the directory structure required, and
      install *qmail-scanner-queue.pl*
    * If you want to manually install it, see the Manual Installation
      <manual-install.php> page.
    * Before going any further, you can test the installation by running
      *./contrib/test_installation.sh*. This will send three Emails: one
      normal and two "infected" with the EICAR test virus - to the
      administrator Email address. Obviously /Qmail-Scanner/ should
      catch them, quarantine them and send messages to the administrator
      stating that such messages had been caught. The EICAR is a test
      virus - not a real one - so don't get too concerned! :-) If you
      receive the original messages instead of "Virus found" messages,
      then something is wrong with your install.

At this stage qmail-smtpd will need to be "told" that Qmail knows to use
*qmail-scanner-queue.pl* instead of *qmail-queue*. This is done via the
tcpserver control files for smtp. Look to see where tcpserver for
qmail-smtpd gets its rules from - it's the file after the "-x" option
(well, that's the CDB version actually - find the text file yourself!
;-). Edit that file and tell qmail-smtpd which IP address ranges
(corresponds to SMTP client IP addresses) you want Qmail-Scanner to be
invoked on - typically all of them.

#/etc/tcpserver/smtp.rules
#
# No Qmail-Scanner at all for mail from 127.0.0.1
127.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-queue"
# Use Qmail-Scanner without SpamAssassin on any mail from the local network
# [it triggers SpamAssassin via the presence of the RELAYCLIENT var]
10.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"
#
# Use Qmail-Scanner with SpamAssassin on any mail from the rest of the world
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

The above example means from now on all SMTP mail will be scanned, but
with different characteristics. Mail from the LAN (10. network) will be
scanned by the supported virus scanners, whereas mail from the Internet
will be scanned for viruses AND tagged by SpamAssassin. This finer
control allows you a lot of versatility, e.g. virus scanning only
performed on mail coming from your Exchange server, and not from your
Unix servers.

You /must/ increase the amount of memory your system allows qmail-smtpd
to run with, as it it now running the entire perl interpreter PLUS virus
scanners. Typical installs of Qmail have system rc/startup scripts (e.g.
*/etc/rc.d/init.d/qmail* or */service/smtp/run*) that limit the amount
of RAM qmail-smtpd can use via ulimit or softlimit. You must increase
that to around 5-6Mb. If you don't qmail-smtpd will crash with a "qq"
error on the receipt of the very first message... The actual amount is
dependent on the OS in question as well as the virus scanners being
used, so be prepared to experiment a little. Whatever you do, /don't/
just set it to something stupid like 100M "just to be sure". The whole
point about limiting RAM usage is so that "unusal" mail messages (e.g.
from spammers or hackers) can't cause your system to become unusable by
making it run out of RAM.

To scan all mail sent by local shell users, the *QMAILQUEUE* will also
need to be defined within */etc/profile* or the like so that when they
send mail, it will be affected as well.
Although as they are obviously Unix users, you may want to save your
system the effort and explicitly NOT do that! :-)

Also, think twice before running Qmail-Scanner in front of any
mailing-list servers. Do you really think it's a good idea to have
10,000 messages banging away at your antivirus system at the same time?
Either put your mailing-list servers beyond the reach of your
Qmail-Scanner servers, or put the mailing-list on the Qmail-Scanner
servers themselves - that way each message is only scanned once and the
load issues disappear.

If "/$DEBUG=1/" (the default) is set within *qmail-queue-scanner.pl*,
then every transaction will be logged to
*/var/spool/qmailscan/qmail-queue.log* - so you'll see how it goes.
Regardless of debugging, errors (and attachment info if enabled) should
also be recorded in the qmail logs (probably via /syslog/) - just look
for entries containing the string "*X-Qmail-Scanner*".

Any SMTP sessions that are dropped (due to network outages/etc) may lead
to files lying around in */var/spool/qmailscan *. Running
*/var/qmail/bin/qmail-scanner-queue.pl -z* at least once daily will
ensure such files are deleted when they're over 30 hours old - make a
cronjob to do that. Also realise that
*/var/spool/qmailscan/qmail-queue.log* will grow without bounds. At some
stage turn debugging off (/$DEBUG=0/) and delete the logfile.
Personally, I like the logfile, so I run a cronjob that just does "mv -f
qmail-queue.log qmail-queue.log.1" at 3am every morning. That way logs
don't grow without bound, but you still end up with the logs from the
past two days. The file can be safely deleted at any time if it becomes
a disk-hog, but unless "/$DEBUG=0/" is set, it'll just get re-created
the next time a message comes through.

Qmail-Scanner contains an internal scanner which allows you to reject
Email based on attachment filenames and/or Email headers. Read the
minimal document <perlscanner.php> on it for details.


    Philosphy behind Quarantining...

When Qmail-Scanner decided to quarantine a message, it moves it into a
local mail folder (maildir format) - by default
*/var/spool/qmailscan/quarantine/*. This means the message can be read
in its pure "adulterated" state (e.g. still containing viruses) by
maildir clients like mutt <http://www.mutt.org/> - or via IMAP (if
maildir format supported - you'll have to work that out for yourself).
At worse you can just read it with an editor - it's just a MIME file...

If you want a good IMAP server that supports maildir natively - try
Courier-IMAP <http://www.inter7.com/courierimap/>.

I made the decision to write it into maildir format for performance and
reliability reasons - and it expressly makes it difficult for any
Windows admin to click on it with their vulnerable Windows mailer and
read it :-) Qmail actually comes with a program called
/var/qmail/bin/maildir2mbox which can do just that... (you could run it
from cron to automatically suck all the new mail messages from
*/var/spool/qmailscan/quarantine/new/* into a mbox.)

Also note that Qmail-Scanner /only/ quarantines. It doesn't drop or
"clean" messages. Cleaning a message is complicated and is becoming less
of an issue. Personally I feel that if someone sends a virus-laden
message through Qmail-Scanner, then it should be blocked - not cleaned.
Why should Qmail-Scanner "fix" someone elses problem? People need to
take responsibility for their own problems. They get an alert telling
them they're infected, so they will need to disinfect their system
before attempting to send mail through your site again. Virus scanners
that "clean" just make these people think they don't have to worry about
their problem - someone else is fixing it.

Qmail-Scanner also doesn't distinguish between "blocking" attachments
(via the perlscanner module) and viruses. i.e. if a zip file contains a
MP3 file, and you are blocking MP3s - then the message will be
quarantined. Some people think that Qmail-Scanner should only block
files if they are "direct" attachments - not contained within archive
attachments. Again, if you have a policy saying "no MP3" - then it
should apply no matter how it's sent...

Finally, I am finding the majority of viruses are now trojans - which
means that there is no actual human-generated message to allow through
anyway. There is nothing more annoying than being behind some other
vendors antivirus scanner when 1,000 copies of the latest virus comes
through. Receiving 1,000 copies of a "cleaned" trojan is only maginally
better than receiving the trojan! (in fact it's worse for Unix users
like me! ;-) In fact, but the 10'th copy, all the users receiving mail
are now paranoid to open /any/ mail as they don't feel secure about
what's going on. The better response is that the ORIGINATOR receive
1,000 alerts telling them they're infected - maybe they'll do something
about it.

Also this event is logged in */var/spool/qmailscan/quarantine.log* in a
tab-delimited format (for post-processing). A good script is needed to
convert this file into some nice graphs for management :-). See QSS
<http://www.mandranet.com/open-source/qss> for an example of one way of
generating stats.

If Qmail-Scanner was configured with the "--log-details" option, then a
one-line summary of every message processed is recorded either in
mailstats.csv or via syslog. e.g:

Jan 16 10:42:51 mailhost perlscanner: BadTrans 0.967762 xuy@213.3 remote@addr Franky_says_RELAX <> 1646 Wed,_16_Jan_2002_10:42:42_+1300 1011130969.31369-0.mailhost:117 1011130969.31369-1.mailhost:139  
Jan 16 10:42:52 mailhost perlscanner: Clear: 0.2322 jhaar@users.sourceforge.net jhaar@users.sourceforge.net Testing_testing <08008383883@mailhost> 2345 Wed,_16_Jan_2002_10:42:42_+1300 1011130969.31369-0.mailhost:117 1011130969.31369-1.mailhost:139 


    Support

This software is released under the GPL as found in the COPYING file
enclosed.

This package is housed on SourceForge
<http://sourceforge.net/project/?group_id=6116>.

Any questions, suggestions, etc to the mailing-list set up to discuss
this, subscribe via
http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-general , or
subscribe to the announcements-only list via
http://lists.sourceforge.net/mailman/listinfo/qmail-scanner-announce.

*Last Updated:* Monday 22nd of July 2002 10:56:45 AM GMT

