Note for Users Upgrading to SpamAssassin 3.4.1 ---------------------------------------------- - The TxRep plugin is now included and disabled by default for new installs. To replace an existing AWL configuration with TxRep, follow the steps below: - Disable AWL - Enable TxRep - Set txrep_factor equal to your previous AWL factor - Set use_txrep to 1 For more detailed information and more configuration options, consult the documentation in Mail::SpamAssassin::Plugin::TxRep. - The $VALID_TLDS_RE global in registrar boundaries is deprecated but kept for third-party plugin compatibility. It will become increasingly out of date and may be removed in a future release. See lib/Mail/SpamAssassin/Plugin/FreeMail.pm for an example of the new way to obtain a valid list of TLDs, i.e. $self->{main}->{registryboundaries}->{valid_tlds_re} - Mail::SpamAssassin::Util::RegistrarBoundaries is being replaced by Mail::SpamAssassin::RegistryBoundaries so that new TLDs can be updated via 20_aux_tlds.cf delivered via sa-update. ***3rd Party Plugin Authors, Please Note*** The following functions will be removed in the next release after 3.4.1 excepting any emergency break/fix releases immediately after 3.4.1: Mail::SpamAssassin::Util::RegistrarBoundaries::is_domain_valid Mail::SpamAssassin::Util::RegistrarBoundaries::trim_domain Mail::SpamAssassin::Util::RegistrarBoundaries::split_domain Mail::SpamAssassin::Util::uri_to_domain And the following variables will also be removed: Mail::SpamAssassin::Util::RegistrarBoundaries::US_STATES Mail::SpamAssassin::Util::RegistrarBoundaries::THREE_LEVEL_DOMAINS Mail::SpamAssassin::Util::RegistrarBoundaries::TWO_LEVEL_DOMAINS Mail::SpamAssassin::Util::RegistrarBoundaries::VALID_TLDS_RE Mail::SpamAssassin::Util::RegistrarBoundaries::VALID_TLDS This change should only affect 3rd party plugin authors who will need to update their code to utilize Mail::SpamAssassin::RegistryBoundaries instead of the functions and variables in Mail::SpamAssassin::Util::RegistrarBoundaries and the function Mail::SpamAssassin::Util::uri_to_domain which are deprecated and will be removed. For example, the $VALID_TLDS_RE global in registrar boundaries is deprecated but kept for third-party plugin compatibility. It will become increasingly out of date and may be removed in a future release. See lib/Mail/SpamAssassin/Plugin/FreeMail.pm for an example of the new way to obtain a valid list of TLDs, i.e. $self->{main}->{registryboundaries}->{valid_tlds_re} - It is now recommended that users uncomment "normalize_charset 1" in local.cf. It will break rules that depend on messages being in non-UTF8 encodings, but going forward this will enable more robust unicode rules and will eventually become the default. Note for Users Upgrading to SpamAssassin 3.4.0 ---------------------------------------------- - When Bayes classification is in use and messages are 'learned' as spam or ham and stored in a database, the Bayes plugin generates internal message IDs of learned messages and stores them in a 'seen' database to avoid re-learning duplicates and accidental un-learning messages that were not previously learned. With changes in bug 5185, the calculation of message IDs in a bayes 'seen' database has changed, so new code can no longer associate new messages with those learned before the change. - Note that this change does not affect recognition of old tokens and the classification algorithm, only duplicate detection and unlearning of old messages is affected. - Because of this change, if you use Bayes and you are upgrading from a version prior to 3.4.0, you may consider wiping your Bayes database and starting fresh. - There is a new optional dependency on Net::Patricia to speed up lookups on internal_networks, trusted_networks or msa_networks when these lists contain a larger number of entries. Consider installing this module to speed up classification. - The minimal required version of NetAddr::IP was bumped to 4.010 - In addition to existing backends, the 3.4.0 introduces support for keeping a Bayes database on a Redis server, either running locally, or accessed over network. Similar to SQL backends, the database may be concurrently used by several hosts running SpamAssassin. - For more detail on these and other changes, please see the Announcement file at: http://svn.apache.org/repos/asf/spamassassin/branches/3.4/build/announcements/3.4.0.txt Note for Users Upgrading to SpamAssassin 3.3.0 ----------------------------------------------- - Rules are no longer included with SpamAssassin "out of the box". You will need to immediately run "sa-update", or download the additional rules .tgz package and run "sa-update --install" with it, to get a ruleset. - The BETA label has been taken off of the SpamAssassin SQL support. Please be aware that occasional changes may still be made to this area of the code. You should be sure to read this upgrade document each time you install a new version to determine if any SQL updates need to be made to your local installation. - The DKIM plugin is now enabled by default for new installs, if the perl module Mail::DKIM is installed. However, installation of SpamAssassin will not overwrite existing .pre configuration files, so to use DKIM when upgrading from a previous release that did not use DKIM, a directive: loadplugin Mail::SpamAssassin::Plugin::DKIM will need to be uncommented in file "v312.pre", or added to some other .pre file, such as local.pre. Note for Users Upgrading to SpamAssassin 3.2.0 ----------------------------------------------- - The "127/8" network, including 127.0.0.1, is now always implicitly part of "trusted_networks" and "internal_networks". It's impossible to remove these from the trusted/internal sets, since if you cannot trust the host where SpamAssassin is running, you cannot trust SpamAssassin itself. If you previously had "trusted_networks" and "internal_networks" lines listing those hosts, you should now remove them, otherwise a minor (non-lint-error) warning will be output. - For ISPs -- version 3.2.0 now includes a new way to specify Mail Submission Agents, relay hosts which accept mail from your own users and authenticates them appropriately. See the Mail::SpamAssassin::Conf manual page for the "msa_networks" setting. Note for Users Upgrading to SpamAssassin 3.1.0 ----------------------------------------------- - A significant amount of core functionality has been moved into plugins. These include, AWL (auto-whitelist), DCC, Pyzor, Razor2, SpamCop reporting and TextCat. For information on configuring these plugins please refer to their individual documentation: perldoc Mail::SpamAssassin::Plugin::* (ie AWL, DCC, etc) - There are now multiple files read to enable plugins in the /usr/pkg/etc/spamassassin directory; previously only one, "init.pre" was read. Now both "init.pre", "v310.pre", and any other files ending in ".pre" will be read. As future releases are made, new plugins will be added to new files named according to the release they're added in. - Due to license restrictions the DCC and Razor2 plugins are disabled by default. We encourage you to read the appropriate license yourself and decide if you are able to re-enable the plugins for your site. - The use_auto_whitelist config option has been moved to a user config option, this allows individual users to turn on or off whitelisting regardless of what is set in the system config. If you would like to disable AWL (auto-whitelist) on a site-wide basis, then you can comment out the plugin in "v310.pre". - The bayes_auto_learn_threshold_* config options for bayes have moved to a plugin. In general the default should work just fine however if you are interested in changing these values you should see: perldoc Mail::SpamAssassin::Plugin::AutoLearnThreshold - The AWL support for NDBM_File databases has been dropped, due to a bug in that package which renders it useless (bug 4353). It appears that SDBM_File, the package which will be used instead, is fully compatible with NDBM however, so this should be unnoticeable. - The prefork algorithm for spamd has been changed. In this version spamd will attempt to keep a small number of "hot" child processes as busy as possible, and keep any others as idle as possible, using something similar to the Apache httpd server scaling algorithm. This reduces memory usage and swapping. You can use the --round-robin switch for spamd to disable this scaling algorithm, and the behaviour seen in the 3.0.x versions will be used instead, where all processes receive an equal load and no scaling takes place. - As of 3.1.0, in addition to the generic BayesSQL support (via Mail::SpamAssassin::BayesStore::SQL) usable by multiple database drivers there is now specific support for MySQL 4.1+ and PostgreSQL. This support is based on non-standard features present in both database servers that allow for various performance boosts. If you were using the previous BayesSQL support with MySQL, and already have MySQL 4.1+ installed you can begin using the new module immediately by replacing the bayes_store_module line in your configuration with: Mail::SpamAssassin::BayesStore::MySQL We do however recommend that you switch your MySQL tables over to InnoDB for better data integrity and multi user support. You can most often do this via a simple ALTER TABLE command, refer to the MySQL documentation for more information. If you were previously using PostgreSQL for your bayes database then we STRONGLY recommend switching to the PostgreSQL specific backend: Mail::SpamAssassin::BayesStore::PgSQL To switch to this backend you should first run sa-learn --backup to make a backup of your existing data and then drop and recreate the database following the instructions in sql/README.bayes. Then you can restore the database with sa-learn --restore. If you have multiple users then you will have to run --backup and --restore for each user to fully restore the database. - See http://wiki.apache.org/spamassassin/UpgradeTo310 for a supplementary list of upgrade notes. It will be updated with any upgrade notes not captured in this document. - Finally, this document is likely not complete. Other configuration options/arguments may have changed from older versions, etc. It would be good to double-check any custom configuration options to make sure they're still valid. This could be as simple as running "spamassassin --lint", or more complex, as required by the environment. Note for Users Upgrading to SpamAssassin 3.0.x ---------------------------------------------- - The SpamAssassin 2.6x release series was the last set of releases to officially support perl versions earlier than perl 5.6.1. If you are using an earlier version of perl, you will need to upgrade before you can use the 3.0.0 version of SpamAssassin. You will also want to make sure that you have the appropriate versions of required and optional modules as they may have changed from old versions. The INSTALL document has the modules and version requirements listed. - See http://wiki.apache.org/spamassassin/UpgradeTo300 for a supplementary list of upgrade notes. It will be updated with any upgrade notes not captured in this document. - SpamAssassin 3.0.0 has a significantly different API (Application Program Interface) from the 2.x series of code. This means that if you use SpamAssassin through a third-party utility (milter, etc,) you need to make sure you have an updated version which supports 3.0.0. See http://wiki.apache.org/spamassassin/UpgradeTo300 for information about third-party software. - The --auto-whitelist, --whitelist and -a options for "spamd" and "spamassassin" to turn on the auto-whitelist have been removed and replaced by the "use_auto_whitelist" configuration option which is also now turned on by default. - The --virtual-config switch for spamd had to be dropped, due to licensing issues. It is replaced by the --virtual-config-dir switch. - The "rewrite_subject" and "subject_tag" configuration options were deprecated and are now removed. Instead, using "rewrite_header Subject [your desired setting]". e.g. rewrite_subject 1 subject_tag ****SPAM(_SCORE_)**** becomes rewrite_header Subject ****SPAM(_SCORE_)**** - The "sa-learn --rebuild" command has been deprecated; please use "sa-learn --sync" instead. The --rebuild option will remain temporarily for backward compatability. - The Bayesian storage modules have been completely re-written and now include Berkeley DB (DBM) storage as well as SQL based storage (see sql/README.bayes for more information). In addition, a new format has been introduced for the bayes database that stores tokens in fixed length hashes (Bayes v3). All DBM databases should be automatically converted to this new format the first time they are opened for write. You can manually perform the upgrade by running "sa-learn --sync" from the command line. Due to the database format change, you will want to do something like this when upgrading: - stop running spamassassin/spamd (ie: you don't want it to be running during the upgrade) - run "sa-learn --rebuild", this will sync your journal. if you skip this step, any data from the journal will be lost when the DB is upgraded. - upgrade SA to 3.0.0 - run "sa-learn --sync", which will cause the db format to be upgraded. if you want to see what is going on, you can add the "-D" option. - test the new database by running some sample mails through SpamAssassin, and/or at least running "sa-learn --dump" to make sure the data looks valid. - start running spamassassin/spamd again If, instead of uprading your Bayes database, you want to wipe it and start fresh, you can run "sa-learn --clear" to safely remove your Bayes database files. If the --clear command issues an error then you can simply delete the Bayes database files ("bayes_*") while SA is not running; SpamAssassin will recreate them in the current format when it runs. - "spamd" now has a default max-children setting of 5; no more than 5 child scanner processes will be run in parallel. Previously, there was no default limit unless you specified the "-m" switch when starting spamd. - If you are using a UNIX machine with all database files on local disks, and no sharing of those databases across NFS filesystems, you can use a more efficient, but non-NFS-safe, locking mechanism. Do this by adding the line "lock_method flock" to the /usr/pkg/etc/spamassassin/local.cf file. This is strongly recommended if you're not using NFS, as it is much faster than the NFS-safe locker. - Please note that the use of the following commandline parameters for spamassassin and spamd have been deprecated and may be removed in upcoming versions of SpamAssassin. Please discontinue usage of these options: in the 2.6x series: --add-from, --pipe, -F, --stop-at-threshold, -S, -P (spamassassin only) in the 3.0.x series: --auto-whitelist, -a, --whitelist-factory, -M, --warning-from, -w, --log-to-mbox, -l - user_scores_sql_table is no longer supported. If you need to use a table name, other than the default, create a custom query using the user_scores_sql_custom_query config option. - SpamAssassin runs in "taint mode" by default for improved security. Certain third-party modules may be incompatible with taint mode. - 2.6x deprecated the use of the "check_bayes_db" script, and it has been removed in 3.0.0. Please see the sa-learn man/pod documentation for more info. - Finally, this document is likely not complete. Other configuration options/arguments may have changed from older versions, etc. It would be good to double-check any custom configuration options to make sure they're still valid. This could be as simple as running "spamassassin --lint", or more complex, as required by the environment. An example: "require_version " hasn't changed itself, but the internal version representation is now "x.yyyzzz" instead of "x.yz" which could cause issues if "require_version 3.00" is expected to work (it won't, it needs to be "require_version 3.000000"). Note for Users Upgrading from SpamAssassin 2.5x ----------------------------------------------- - Due to major reliability shortcomings in the database support libraries other than DB_File, we now require that the DB_File module be installed to use SpamAssassin's Bayes rules. SpamAssassin will still work without DB_File installed, but the Bayes support will be disabled. If you install DB_File and wish to import old Bayes database data, each user with a Bayes db should run "sa-learn --import" to copy old entries from the other formats into a new DB_File file. Due to the database library change, and the change to the database format itself, you will want to do something like this when upgrading: - stop running spamassassin/spamd (ie: you don't want it to be running during the upgrade) - run "sa-learn --rebuild", this will sync your journal. if you skip this step, any data from the journal will be lost when the DB is upgraded. - install DB_File module if necessary - upgrade SA to 3.0.0 - if you were using another database module previously, run "sa-learn --import" to migrate the data into new DB_File files - run "sa-learn --sync", which will cause the db format to be upgraded. if you want to see what is going on, you can add the "-D" option. - test the new database by running some sample mails through SpamAssassin, and/or at least running "sa-learn --dump" to make sure the data looks valid. - start running spamassassin/spamd again Obviously the steps will be different depending on your environment, but you get the idea. :) Note For Users Upgrading From SpamAssassin 2.3x or 2.4x ------------------------------------------------------- - SpamAssassin no longer includes code to handle local mail delivery, as it was not reliable enough, compared to procmail. So now, if you relied on spamassassin to write the mail into your mail folder, you'll have to change your setup to use procmail as detailed below. If you used spamassassin to filter your mail and then something else wrote it into a folder for you, then you should be fine. - Support for versions of the optional Mail::Audit module is no longer included. - The default mode of tagging (which used to be ***SPAM*** in the subject line) no longer takes place. Instead the message is rewritten. If an incoming message is tagged as spam, instead of modifying the original message, SpamAssassin will create a new report message and attach the original message as a message/rfc822 MIME part (ensuring the original message is completely preserved and easier to recover). If you do not want to modify the body of incoming spam, use the "report_safe" option. The "report_header" and "defang_mime" options have been removed as a result. (end of UPGRADE) //vim:tw=74: