Rspamd (3.10.2)

Recent Releases

3.10.221 Oct 2024 07:33 minor bugfix: Fix for ARC signing contributed by `@jscissr`; Added EOF to Prometheus metrics by `@henry-spanka`

3.10.116 Oct 2024 17:01 minor bugfix: Fixed Ragel state machine on ARM; Fixed OpenSSL-related crashes on RedHat systems; Fixed build with PUC-Rio Lua by @arkamar; Avoid null-bytes in Log-Tag header by @smarsching; Fixed incorrectly reported ASAN flag; Fixed counting of ham/spam learns; Updated to upstream hiredis removing all hacks

3.10.001 Oct 2024 12:21 minor feature: MIME UTF8 support; Support for negative group score limits; Per key fuzzy ratelimit rules; Detect CPU using __builtin_cpu_supports where it s possible; Sender Reputation Improvements; Better OpenSSL 3.0 Support; DMARC Enhancements; various fixes; rules updates. https://rspamd.com/announce/2024/09/30/rspamd-3.10.0.html

3.9.108 Aug 2024 16:25 minor feature: 23 Jul 2024 Conf Spf: Add R_SPF_PLUSALL symbol with some score. Feature Spf: Treat SPF +all in a special way. Minor Ensure some safety when checking weights. Minor several with flag propagation. Minor Gpt: Improve prompt and add some conversion heursitics. Minor Gpt: Remove top_p reduce temperature to 0. Minor Gpt: Set response_format. Minor Gpt: Use gpt-4o-mini by default.

3.9.013 Jul 2024 03:16 major bugfix: 12 Jul 2024 Crit Protect regexp matcher from regexps with empty patterns. Feature Allow adding X-CMAE-Score header. Feature Allow custom milter quarantine and tempfail messages. Feature Allow to specify minimum weight in GPT plugin. Feature Cloudmark: Add scores_symbols setting. Feature Further optimization to the hot path. Feature Make min_resolve_interval configurable. Feature Report slow synchronous rules. Feature Rspamc: add `files-list` option. Feature Support metrics command in normal/proxy workers. Feature Support reply in message pack format. Feature Use msgpack in the client. Feature Verify sanity of l= tag in DKIM. Feature milter_headers: support not removing headers. Allow autolearn for the controller worker. Allow spaces in DMARC records. Allow to set 0 as number of rows to disable roll history. Always set the unlearn flag when relearning. Apply detection phase if fasttext could not detect language. Backport from libucl. Backport multiple from libucl. Change expiration logic for redis_cache. Check nconns when firing the final termination event. Do not apply tableshape for known senders. Do not crash if symbol is missing in the metric. Do not read out-of-boundary when doing base64 encoding. Do not save multipatterns to FS in certain cases. another corner case that allows candidates to be freed without init. documentation nesting. dynamic_symbols in the multimap plugin. inconsistent nesting in mime parts. parsing of maps definitions. shared memory proxying when compression is set. Fuzzy add/delete handlers are badly broken for compression. Honor dynamic thresholds for greylisting module. If we have one statfile disabled we need to disable all. Increase/decrease hash_key value for tokens depending on is_unlearn. Libucl: messagepack autodetection. Resgore the collaterally removed line. Unset autolearn f

3.8.426 Feb 2024 10:24 minor bugfix: - Fix regression in DMARC reporting

3.8.322 Feb 2024 16:58 minor bugfix: - Fix regression in redis_history

3.8.220 Feb 2024 18:17 minor bugfix: - Feature Add extraction type for from maps by @vstakhov in #4794 - Rework Breaking: Do not report module as action by @moisseev in #4795 - Minor Move server selection logic to common.js by @moisseev in #4796 - WebUI Show pass-through module in History by @moisseev in #4798 - Feature Allow to add templates to redis history prefix by @vstakhov in #4801 - Deal with Connection and Host headers on proxying by @vstakhov in #4802 - Minor configwizard: really use LOCAL_CONFDIR by @fatalbanana in #4804 - Fix Resolve issue with bayes stat in rspamadm mode by @vstakhov in #4805 - Minor rspamd_task API docs: fix broken link by @fatalbanana in #4807 - Minor Add more returnbits to surbl configuration by @fatalbanana in #4803 - Feature Support JSON logging when in syslog mode by @vstakhov in #4813 - Fix issues with URL fragment in HTTP requests by @vstakhov in #4814 - Fix Encode headers in metadata exporter by @vstakhov in #4816 - Improve Kaspersky AS integration by @vstakhov in #4806 - Allow dynamic keypairs loading in fuzzy storage by @vstakhov in #4818 - Build tests with CTest in Cmake by @vstakhov in #4822

3.8.125 Jan 2024 19:33 minor bugfix: * statistics: Fix `per_user` * statistics: Fix learn error propagation * statistics: Fix storing of tokens * Fixed potential double-free in regex cache * Fixed header insertion * Fix compatibility with Lua 5.1 (by @arakmar) * Fixed RBL plugin warning text

3.8.020 Jan 2024 07:05 major bugfix: 19 Jan 2024 Project Rspamadm fuzzyping command. Project Use Redis scripts for Bayes statistics and cache. Project Support JSON logging. Crit - reported length of logging structure. Feature Allow to set `max_users` for Redis bayes backend. Feature Escape JSON when needed. Feature Proxy: Allow `encrypted_only` option. Feature Reiterate on gtube patterns. Feature Support ping command in fuzzy storage. Feature Support suppressing DMARC reporting for particular recipients. Feature rbl: support disabling or replacing url_whitelist per RBL. Another try to setproctitle. Cleanup session on exit. Do not cleanup hyperscan files unless new ones are loaded. bad access when printing pending events on failure path. with the raw header on header insertion/modification. overflow in fuzzy_stats command. some corner cases when parsing maps with no newlines. various with canonicalisation of the paths. wrong raw_len usage. Make words selection random deterministic upon content. Properly set config field when creating tasks from Lua. Really the language detector statistical heuristic. Set loaded variable explicitly. dkim_signing: siging_table: lowercase before lookup. known_senders: config handling. rbl: `exclude_local`.

3.7.515 Dec 2023 16:11 minor bugfix: * Fixed logging in `rspamadm` (Reported by `@cfasnacht`) * Fixed logging when using `systemd = true` (Reported by `@AdamMajer`) * `dkim_signing`: `signing_table`: lowercase inputs before lookup (Reported by `@EurenikZ`) * Fixed build without hyperscan (Reported by `@a16bitsysop`) * Fix build on loongarch64 (Contributed by `@wuruilong01`)

3.7.414 Nov 2023 15:28 minor bugfix: * Fixed `setproctitle` on CentOS-likes * Fixes for hyperscan cache file handling * Fixed `exclude_local` in RBL module * Properly set config field when loading tasks from Lua * Set loaded variable explicitly * Enable fasttext on RPM based Linux * Improve `FREEMAIL_AFF` capture rates * Add rule for messages missing both X-Mailer and User-Agent header * Add composite rule for suspicious URLs in suspicious messages

3.7.327 Oct 2023 07:45 minor bugfix: 27 Oct 2023 (Fix) Emergency fix for the hyperscan path error

3.7.226 Oct 2023 15:39 minor bugfix: 26 Oct 2023 (Fix) Deal with fmtlib exceptions properly (Fix) DMARC reporting: fix reporting for subdomains (Fix) Fix DMARC munging (by @dzjaivnt) (Fix) ICAP: restore old content-type behaviour; add `use_specific_content_type` setting (Feature) rbl: support use of different matchers for return codes (WebUI) Fix history table vanishing (WebUI) Add control to invert action filter (Rules) Blank spam detection (Rules) Tighten rspamd's attachment policy (by @twesterhever)

3.7.108 Oct 2023 03:15 minor bugfix: 07 Oct 2023 Crit leak in `gzip` function. Feature Add ICAP Content-Type and Filename. Feature Add `logging`- `task_max_elts` option. Feature Add utility to split string like stuff for C++ code. Feature Allow to set HTTP auth parameters for the maps. Feature Check for plugin configuration errors on `configtest`. Feature `known_senders` plugin. Feature Use backward-cpp instead of manual libunwind stuff. Feature rbl: support checking numeric URLs in isolation. CMakeLists.txt remove whitespace added by linter as it makes tests fail. Change Date: header location to conform with RFC. Correct format pattern for RE tree tempfile name. Correct format string for unw_word_t. Do not accept invalid ucl object types. Do not pollute public headers with libev internals. Do not set output type if list application failed. `url:set_redirected` method. format string and some length grammar definition for content-disposition attributes. lua schema enrichment logic for Redis params. lua stack corruption when logging large tables. merge table utility. output of non-RSA DKIM keys. some corner cases of single-host urls parsing. various in the `url_redirector` plugin. MISSING_MIMEOLE: avoid matching messages from Android GMail app. Prevent DNSWL sabotage. Try to unzip function. rbl: really dependency registration when symbols_preis used. rspamadm mime: arguments beginning with letter `t`. Rework Breaking: return back to semver. Rework Move rcl logic to C++.

3.604 Aug 2023 07:25 minor bugfix: 03 Aug 2023 Conf Add `one_shot` to some specific multimap rules. Conf Add language detection configuration. Conf Add missing attributes for the language detection configuration. Conf Remove outdated composite rules. Feature Add `sentinel_password` option. Feature Add ability to deny specific fuzzy flags by default. Feature Add controller endpoint to get fuzzy hashes from messages. Feature Add extra symbol when URL redirector reaches nested limit. Feature Add function to transliterate utf8 to ascii with some normalisation. Feature Add html parsing limit. Feature Add order to urls structure. Feature Add some missing functions to `lua_rsa` library. Feature Allow fuzzy workers to exchange blocked information. Feature Allow to have weak flags in fuzzy storage. Feature Allow to read options from maps in the multimap plugin. Feature Allow to use other methods when fasttext detection is enabled. Feature Count stats per key per flag. Feature Finish all features of dkim_keygen in Lua. Feature Khash: Allow static initialisation. Feature Maps: Add on_load support. Feature Preliminary implementation of dynamic composites. Feature Process HTML parts before text ones. Feature Reorganise struct rspamd_url to be 64 bytes size. Feature Save fuzzy ratelimit buckets. Feature Use in-place deflation for strings. Feature external_relay: add ip_map strategy. Avoid race between config new/free by using a counter. Do not use `rspamadm.dkim_keygen`. Feed fasttext language model with the pre-tokenized words. `rspamd_has_only_html_part`. an old with order of destruction race between redis pool and lua. format string usage. parsing due to old revealed. parsing of the mask values that are invalid. Ignore non-unique stop words. Include the last character when parsing the last header with no value. More to fuzzystat. Set proper counter. Try harder to clean pending bucket. Try harder to remove bad hyperscan files. Upda

3.518 Mar 2023 03:16 minor bugfix: 17 Mar 2023 Conf Clarify `timeout` in rspamd_proxy. Conf hashbl. Conf Add SURBL hashbl support (WIP). Conf RBL: selector. Conf Reduce the default timeout. Crit Deserialise hyperscan to the page-aligned space to prevent alignment Crit Fill path field in hyperscan notice command. Feature Add `thresholds` field to the scan result. Feature Add possibility to execute lua scripts for blocked fuzzy clients. Feature Add preliminary support of the external maps in the multimap plugin. Feature Allow to build a map by combining tuples of selectors. Feature Allow to query external maps for settings. Feature Allow to specify `selector_alias` in the maps definition. Feature Enable Mime part filters on antivirus module. Feature Improve ratelimit redis scripts. Feature Selectors: Add specific_urls_filter_map extractor. Feature Some rework of the selectors framework. Add O_CREAT flag when creating a file. Add a database check function unless we have anything from Hyperscan. Add hyperscan databases sanity check. Add workaround for ENOBUFS error on sending. Do not do `lstat` when we are creating file. Finally get rid of RSPAMD_USE_47BIT_LIGHTUSERDATA_HACK. boundaries that contain only dashes. off-by-one error in css tokenizer. url reputation plugin. usage of the Redis config schema as `extra_fields`. Further checks for the hs_scratch_alloc. Honor group flag for one shot. Normalize glob paths to avoid hash table misses. Ratelimit: Use unpack for `HMGET` return value. Rbl: helo check pipeline. Replace broken strict_domains with phishing_exceptions. Restore `strict_domains` support. Return true from has_urls(true) if only emails are present. Rework lists applications. Set symcache item in coroutine calls. Treat `hs_allocate_scratch` errors as non-fatal. Treat hostnames with no dots as eSLD of their own. received: filtering of artif

3.402 Nov 2022 03:16 minor bugfix: 01 Nov 2022 Crit Restore compatibility with the integrations and headers alterations. Feature Milter_headers: Add `x-rspamd-action` routine. Feature Share hyperscan database among processes. Another corner case in url parsing. Another for the enable password. Another try to method in lua_tcp. additional fields in the Redis schema. emoji joiner FP. favicon.ico Content-Type header. hang when is used. Lua_tcp: Sigh, another try to `` invocation. Mx_check: Cache the fact of a missing MX record. Try to parsing of the unencoded ` ` characters in html attributes. Try to the case where password == enable_password. Project (Re)implement hyperscan caching. Project Rework cleanup. Project Synchronize hyperscan caches via the main process. Rework Convert multipattern to use hyperscan tools. Rework Make http normalize path function a generic function. Rework Split locked and unlocked files, as mmap does not need flock normally. Rework Start movement of the hyperscan related routines into a single unit. Rework Store the current worker, so other libraries could use this information. Rework Use blocking socket for IPC between main and workers. Rework Use more predictable size for commands buffers. Rules Do not insert ONCE_RECEIVED_STRICT on RDNS missing. Rules Reduce score of HTTP_TO_HTTPS - subject to remove completely.

3.302 Oct 2022 03:15 minor bugfix: 01 Oct 2022 Conf Add missing groups for whitelist module symbols. Crit Neural: keys regression after #3968. Feature Accept upstream in lua_tcp. Feature Add ability to statically maintain disabled/enabled patterns. Feature Add function to store upstreams for HTTP urls. Feature Allow augmentations set in Lua API. Feature Allow lua_http module to accept upstreams. Feature Allow to limit write access to fuzzy storage by key. Feature Allow to sort symbols output. Feature Check content for binary stuff before dumping it to Lua. Feature Implement symbols augmentations. Add missing flags. Add more sanity checks for rua in dmarc_report. Adjust length of the fuzzy checks for short text parts. Another try to add headers compatibility logic. Another try to race condition in the runtime destruction. Avoid cyclic references in symcache and memory leaks. Avoid overriding IP with Sender IP. BAD_REP_POLICIES did not trigger when message was classified as spam by Bayes. Bind AF_UNIX DGRAM client connection to annonymous address. Disable IPv6 lookups for Blocklist.de RBL. Distinguish dynamic and static items. Dkim: Ignore unknown DKIM kv pairs as stated in RFC. Dmarc report: Use local timezone instead of GMT. Do not exclude authenticated users from URIBL lookups. Empty envelopes should not be emitted as arrays (json+messagepack) when populated envelopes are objects. This greatly complicates decoding in strictly typed languages. External_relay: Restore the originating hostname check. DKIM keys with spaces still allowing errors on invalid base64. copying of sockaddr_un addresses. crash with cname replies. dependencies propagation. iteration over milter headers. ordering when sorting symcache. reading of the cached maps. several with the HTTP keepalive parsing. stack smashing. synchronous auth/select in lua_redis. various symcache Igno

3.228 Mar 2022 03:15 minor bugfix: 26 Mar 2022 Conf Score MIME_OBFUSCATED_ARCHIVE to 8 points. Conf Set one_shot for URIBL rules by default. Crit upstreams name resolution when there is also a port. Feature Add ROC feature to neural network plugin. Feature Add public suffic compilation utility. Feature Add support of Cloudmark. Feature Allow hyperscan for ppc64, as vectorscan now suports it. Feature Allow to skip DNS resolution for keep-alive connections. Feature Aws_s3: Allow to store large parts separately. Feature BIMI: Add preliminary version of the BIMI plugin. Feature JSON endpoint for querying maps. Feature Lua_magic: Add a sane CSV heuristic. Feature Lua_mime: Add schema for message transfer. Feature Output average scan time in /stat endpoint. Feature Show average scan time in `rspamc stat` output. Add guards to avoid race condition on TCP connection. Allow spaces in DKIM key records. Apply the similar to the url_reputation. Avoid overwriting whitelisted_signers_map. Backport PR from libucl. Clear SSL errors. ClickHouse cleanup of old partitions. Do not double call error handler on ssl errors in the timeout path. Do not forget to clear pointers on IOC reset. External_relay: Remove useless check of the map value. Find suspicious url encodings that could break url extraction. HTTP(s) client timeout. exclude flags setting. expanding of the variables. host header usage in lua_http. http maps shared memory cache cleanup. logic in HTML processing FSM. parsing of the compound mailto urls. processing captures from pcre2. removing from khash. stuctured headers pushing. Further for i386 compilation. Improve duplicate settings error reporting. Lua: task:remove_result didn't work in some cases. Output service parts as well. Phishing: Deal with phishing + redirected URL. Phishing: finding domains in the phishing map. Plug memory leak by u

3.102 Nov 2021 03:16 minor bugfix: 01 Nov 2021 Feature Add junk_threshold for autolearn. Feature Add neural test command. Feature Antivirus: Allow to set fake eicar patterns for testing AV engines. Feature Lua_cdb: Add cdb building interface. Feature Ratelimit: Add per bucket configurations. Feature S3: Allow to store structured data in messagepack. Add concept of uncancellable events to prevent use-after-free. Add temporary guard to prevent linked list exploitation. Another rework of the ucl hashing. Another try to references safety. Another try to rspamd_text passing in the selectors. Avoid copy for received structure as it has raw C pointers. Avoid dangling reference. Correctly check numeric URLs in URL DNS lists. Delete the correct pointer type. Dmarc: Always lowercase domain. compilation of the hyperscan databases with errors. hash table lookup. http message flag shift. parsing of the from_hostname when it is an IP address. parsing of the unquoted attributes in HTML. passing of rspamd_text in selectors pipelines. rubbish QP sequences decoding. some complicated case with the closing tags parsing. the case when l tag is too small. Html: the case where only bgcolor is explicitly set. Libucl: deletion from ucl objects. Namespace and add metadata for OpenMetrics, interleaving. Plug memory leak in http settings reload. Preserve SPF top record in the mempool variable. Remove aarch64 GC64 workaround. Remove bogus G_LIKELY. Spf: Do not parse non TXT DNS replies as TXT replies. Try to use on_connect/on_disconnect callbacks to handle internal Redis failures. buffer overflow in rspamc counters. static building. lua_scanners - message_min_words logic. src/lua/lua_mimepart.c: null dereference. Project Add constant iterators. Project Add helper library to handle mime strings in a more safe matter. Project Add preliminary support of CDB bayes dump.

3.019 Aug 2021 12:04 major feature: - HTML parser rework - CSS parser - Amazon S3 support - DMARC reporting rework - DMARC munging support - External relay plugin - Bayes export tool - Pyzor support - Monitoring rework - Other fixes and improvements https://rspamd.com/announce/2021/08/19/rspamd-3.0.html

2.708 Jan 2021 15:34 minor feature: - Neural plugin offline learning - Fixed issues with DKIM and ARC verification - Added support for S/MIME containers - Several important rules rework - Support of caching for regexp multimaps https://rspamd.com/announce/2021/01/08/rspamd-2.7.html

2.630 Sep 2020 20:11 minor feature: - Neural network plugin rework - Reworked bitcoin detection library - IDNA bugs are fixed - Fuzzy module telemetry - Other major improvements - Critical/important fixes https://rspamd.com/announce/2020/09/30/rspamd-2.6.html

2.501 Apr 2020 14:25 minor feature: - URL extraction rework - URL structure update - Hyperscan early load - Several major fixes - Added whitelisted_signers_map in ARC module - Implemented /etc/hosts files processing https://rspamd.com/announce/2020/04/01/rspamd-2.5.html

2.426 Feb 2020 19:49 minor feature: - Logger system rework - URL composition library - Implemented SSL client caching - Custom additional columns in Clickhouse plugin - Support of CDB maps everywhere to share huge maps across workers with no extra cost https://rspamd.com/announce/2020/02/26/rspamd-2.4.html

2.304 Feb 2020 16:22 minor feature: - Lua content library - SPF plugin revamped - Slow rules protection - Other features - Important bug fixes https://rspamd.com/announce/2020/02/04/rspamd-2.3.html

2.219 Nov 2019 18:37 minor feature: - Added virustotal support - Clickhouse collection rework - ASAN builds - Faster base64 decoding - Fast unicode validation library - Upstreams fixes - Build system rework https://rspamd.com/announce/2019/11/19/rspamd-2.2.html

2.128 Oct 2019 17:26 minor feature: - Add uuencode support - Critical issue found in dkim verification - Improved neural training - Maps fixes - Event loop fixes https://rspamd.com/announce/2019/10/28/rspamd-2.1.html

2.011 Oct 2019 18:40 major feature: * Libevent has been replaced with bundled libev * Torch has been dropped from Rspamd * RBL module improvements and replacement of the SURBL and Emails module * New Lua Magic library * Neural module rework * Clickhouse module improvement * Lua scanners improvements * Mime modifications * Users settings improvements * Upstreams library improvements * Performance improvements * Rules and other improvements https://rspamd.com/announce/2019/10/11/rspamd-2.0.html

1.9.423 May 2019 14:49 minor bugfix: Fix case sensitivity when parsing Content-Type. Fixes for ARC DKIM signing. Fixes to printing of FP numbers. Ratelimit: Fix dynamic score.

1.9.313 May 2019 16:59 minor feature: Hashicorp Vault support. Added least passthrough result. Improved oversigning logic. Important bugs fixes. https://rspamd.com/announce/2019/05/13/rspamd-1.9.3.html

1.9.216 Apr 2019 17:00 minor feature: Improvements in Clickhouse plugin. OpenDKIM compatible DKIM signing setup. Better encrypted archives support. Calendar files parser. New `rspamadm dns_tool` utility. Better bitcoin addresses detection. https://rspamd.com/announce/2019/04/16/rspamd-1.9.2.html

1.9.105 Apr 2019 11:25 minor feature: Jinja templates in the configuration. New template subcommand in Rspamadm. Changes in URLs extraction for HTML parts. Per user settings for mime_types plugin. Better greylisting conditioning. Bitcoin addresses validation. Replies plugin validation. Bug fixes. https://rspamd.com/announce/2019/04/05/rspamd-1.9.1.html

1.9.012 Mar 2019 16:50 major feature: External services module. New mime modify tool in Rspamadm. Offline DKIM signing tool in Rspamadm. HTTP Keep-Alive support. New Lua UDP client. Better unicode normalisation. Configuration graph utility. Flexible actions support. New Received headers parser. Telephone URLs support. Support for ED25519 DKIM signatures. Custom functions in Regexp module. Added support of gzip archives. Additional detection of types for attachments. Lots of major fixes. https://rspamd.com/announce/2019/03/12/rspamd-1.9.0.html

1.8.303 Dec 2018 16:57 minor bugfix: Large Unicode handling rework - Rspamd now provides better access to normalized/unnormalized unicode. We have added some fail-safety checks against certain types of messages that could cause Rspamd crashes due to invalid unicode in headers. Redis sentinel support. Kaspersky antivirus support. Improvements of the Authentication-Results header. Better statistical tokens. Better and RFC conformant support of pct in DMARC module. Other fixes/features, https://rspamd.com/announce/2018/12/03/rspamd-1.8.3.html

1.8.220 Nov 2018 07:05 minor bugfix: 19 Nov 2018 Conf Add DWL support in the default configuration. Conf Disable rspamd_update by default (again). Conf configuration sample for ratelimit. Crit Disable broken url tags by default. Crit 0 processing when doing RSA sign. Crit adding symbols to their primary groups. Feature Add `rspamadm cookie` utility. Feature Add specialised functions for generating encrypted cookies. Feature Add support of cookies in replies module. Feature Add support of words regexps. Feature Allow to add 3rd party clang plugins. Feature Allow to create lua regexps from glob or plain patterns. Feature Allow to set custom limits for upstream lists. Feature Detect orphaned parts and attach them to message. Feature Filter tokens in bayes. Feature Fold b= value when doing arc sealing. Feature Ignore cookies in the future and too old in the past. Feature Skip stop words in statistics. Feature Store stop words and allow to query them. Feature Support query arguments in controller's custom commands. Feature Tune upstream limits in Rspamd proxy. Feature Use different callback symbols for different uribls. Feature Write DKIM selector in dkim allow/reject symbols. Add obs_fws state support to eoh state machine. Add sanity check when applying mime boundaries heuristic. Antivirus - virus names with 0 were recognized as tables. Disable headernames in bayes temporarily. Do not allow syntax errors in include files. Do not allow to merge an object with an array (or vice versa). Don't perform forged recipients check for missing recipients. DKIM based RBLs. actrie implementation, OOB read. explicit methods call in selectors. extraction of additional parts. finalization for internal plugins. override_defaults function. squeezed symbols when using settings. urls insertion in Clickhouse module. Furhter to ratelimits logic. Ignore signatures when looking for boundaries. Properly set learned count. Really ratelimits configu

1.8.116 Oct 2018 17:40 minor bugfix: Fix options insertion. Fix words decay one more time (affects long messages). Increase default words_decay. Plug memory leak in redis pool. Add `check_violation` feature to DKIM/ARC signing. Add only unique elements to Clickhouse url arrays. Allow `g+:` and `g-:` composite atoms. Allow dkim domains check in surbl. Allow maps with HTTP auth. Allow to disable actions by users settings. Extend whitelisting options. Store url object in images. Use verdict instead of the plain action in plugins. Allow to call fstring append with NULL string. DCC - luacheck. Do not load torch on each rspamadm invocation. Fix boundaries detection and rework stop words algorithm. Fix dependencies for DNS_SIGNED symbol. Fix errors when dealing with dynamic rates/bursts in Ratelimit. Fix groups mess. Fix groups mess. Fix parsing address with comments. Fix resolving in DMARC reports. Fix various issues with parsing of the received headers. Fix watchers issue in lua_tcp when doing no resolving. Plug memory leak in language detector (affects reloads). Remove one letter stop words. Slashing: backport chunk logic from libucl. Stop libevent from using cached time in rspamadm. Try to fix watchers chaining. Various fixes in redis sync interface. ip_score - respect check_authed and check_local settings from config. Rework passthrough actions. Clustering module. Always create result for a task. Completely rewrite DMARC checks logic. Rework and fix whitelist plugin. Add symbols sorting buttons. Change symbols order without updating history. Colorize symbols. Do not display password form when secure_ip is set. Fix symbol description tooltips display. History: add sorting by symbol score value.

1.8.024 Sep 2018 14:13 major feature: New selectors framework. Coroutines API support in Lua. Clickhouse optimization. Unicode processing improvements. Language detection improvements. https://rspamd.com/announce/2018/09/24/rspamd-1.8.0.html

1.7.902 Aug 2018 03:15 minor bugfix: 01 Aug 2018 Crit caseless comparison of equal length strings. Feature Add HTTP basic auth support to elastic and clickhouse plugins. Feature Add SPF selector to reputation. Feature Add support of the fallback backends for HTTP maps. Feature Allow to print full mime structure when extracting mime data. Feature Allow to split symbols in reputation plugin. Feature Check attachments only on AV scanners in attachments_only mode. Feature Disable all SSL checks if ssl_no_verify flag is set. Feature Implement parsing of scoped IPv6 addresses. Feature Improve `rspamc counters` output. Add sanity checks when expanding SPF macros. Allow to parse SA rules with no spaces around = (dirty hack). Avoid one extra byte writing. Deal with direct hash table. Detect empty text part as text, not HTML. Do not reduce map watch timeout for mixed http/file maps. HTML part detection heuristic. double free in redirectors cleanup. legacy history handling in the controller. messages insertion. sending string method. statconver command line arguments. argument checking for being null. reported by luacheck. Freeze updates queue when do actual storage update. HTTP map hash is per-backend and not per-map. Plug memory leak in fuzzy updates. Prefer 'MTA-Name' when producing authentication results. Replace bad unicode sequences instead of stopping on them. Set classifier version on learning. Project Reworked ratelimits. Project Apply topological sorting for symbols in Rspamd. Project Remove global contexts from C modules. Project Move performance critical hash tables to khash. WebUI Avoid unused indexes. WebUI Do not execute `on_success` callback. WebUI history reset for "All SERVERS". WebUI query URL for selected server. WebUI symbols display in legacy history. WebUI Hide symbols order selector for legacy history. WebUI Refactor query functi

1.7.813 Jul 2018 03:15 minor bugfix: 12 Jul 2018 Feature Add more extended statistics about fuzzy updates. Feature Add more non-conformant Received headers support. Feature Add preliminary function to get fuzzy hashes from text in Lua. Feature Allow to configure AV module rejection message. Feature Implement fuzzy hashes extraction in mime tool. Feature Improve WHITE_ON_WHITE rule. Feature Improve integer - string conversion. Feature Reuse maps in multimap module more aggressively. Avoid race condition in skip map as pool lifetime is not enough. Eliminate all specific C plugins pools. DKIM check rule if DNS is unavailable. build where ucontext is defined in ucontext.h. crash in base url handling. descriptors leak in sqlite3 locking code. messages quarantine. padded numbers printing. race condition on maps reinit. regexp functions when no data is passed. specific urls extraction. styles propagation. Improve resetting of the limit buckets. Initialize sqlite3 properly. Work with broken resolvers in resolv.conf. Project Implement HTTP maps caching. Project Refresh fuzzy hashes when matched. Project Add logic to deduplicate fuzzy updates queue. WebUI Add missed declarations. WebUI Avoid using "undefined" property. WebUI Do not accept passwords containing control characters. WebUI Do not redeclare variables. WebUI Enable strict mode. WebUI variable assignment. WebUI Initialize variables at declaration. WebUI Remove duplicated path from RequireJS config. WebUI Remove unused block. WebUI Remove unused variable. WebUI Remove unused variables. WebUI Use self-explanatory notation. WebUI Use type-safe equality operators.

1.7.703 Jul 2018 03:15 minor bugfix: 02 Jul 2018 Crit Check NM part of pubkey to match it with rotating keypairs. Crit Do not overwrite PID of the main process. Crit maps after reload. Crit maps race conditions on reload. Crit shmem leak in encrypting proxy mode. Feature Add a concept of ignored symbols to avoid race conditions. Feature Add ability to print bayes tokens in rspamadm mime. Feature Add method to get statistical tokens in Lua API. Feature Add preliminary mime stat command. Feature Add rspamadm mime tool. Feature Add urls extraction tool. Feature Address ZeroFont exploit. Feature Allow rspamadm mime to process multiple files. Feature Allow to extract words in `rspamadm mime`. Feature Allow to print mime part data. Feature Allow to show HTML structure on extraction. Feature Distinguish IP failures from connection failures. Feature Improve output for mime command. Feature Improve styles propagation. Feature Main process crash will now cleanup all children. Feature Preload file and static maps in main process. Feature Print stack trace on crash. Feature Process font size in HTML parser. Feature Propagate content length of invisible tags. Feature Read ordinary file maps in chunks to be more safe on rewrites. Feature Support base tag in HTML. Feature Support more size sufwhen parsing HTML styles. Feature Support opacity style. Another for nested composites. Fill nm id in keypairs cache code. colors alpha channel handling. destruction logic. double free. maps preload logic. nested composites process. proxying of Exim connections. reload crash. rspamadm -l command. Update ed25519 signing schema. WebUI Stop using "const" declaration. WebUI Update RequireJS to 2.3.5.

1.7.616 Jun 2018 03:15 minor bugfix: 15 Jun 2018 Crit multiple neural networks support. Feature Add decryption function to keypair command. Feature Add gzip compression for HTTP requests in elastic module. Feature Add gzip methods to lua util. Feature Add maps based on Top Level Domains. Feature Add pubkey checks for dkim_signing. Feature Add support of fake DNS records. Feature Add tool to encrypt files. Feature Allow to add symbols using settings directly. Feature Allow to match private and public keys for DKIM signatures. Feature Allow to set task flags via settings. Feature Allow to specify fake DNS address from the config. Feature Implement signatures verification using rspamadm keypair. Feature Implement signing using `rspamadm keypair`. Feature Improve error reporting for DKIM key access Feature Provide HOSTNAME variable in UCL. Feature Rework levenshtein distance computation. Feature Split message parsing and processing. Feature Support ED25519 DKIM signatures. Feature Support encrypted configs in UCL. Feature Suppress duplicate warning on very large radix tries. Feature Use OSB to combine header names. Cleanup maps data on shutdown. ' ' behaviour in composites. HTTP maps updates. NIST signatures. RFC822 comments when processing a mime address. double free. dynamic settings application. for CommuniGate Pro maillist. keypair creation method to actually create keypair. matching patterns with no paths. memory leak in parsing comments. parsing of urls with numeric password. plugins intialisation in configwizard. potential crash on reload. potential race condition for a finished HTTP connections. race-condition leak on processes reload. signing in openssl mode. Free language detector structures. Relax alignment requirements. Send DMARC reports compressed. Try to leak in dmarc module. Try to plug memory leak in metric exporter. Project Convert rspamadm subcommands to

1.7.518 May 2018 15:54 minor bugfix: Add MSBL proposed return codes. Add additional groups for policies. Do not use volatile Lua strings as UCL keys. Add ability to add fuzzy hashes to headers. Add function to extract most meaningful urls. Add rule to block mixed text and encrypted parts. Allow multiple groups for symbols. Allow to disable lua squeezing logic. Allow to get multipart children in Lua. Allow to insert multiple headers from milter headers. Allow to print scores in subject and further extensions. Be more error-prone in squeezed rules. Support multiple return codes in emails module. Use EMA for calculating averages. Use common jit cache for all regexps. support for CommuniGate Pro self-generated messages. Allow to have multiple values for headers as arrays. Do not open sockets for disabled workers. Fix AuthservId. Fix base64 folding in Lua API. Fix build on non-x86 platforms. Fix cached maps logic. Fix compatibility with old maps query logic. Fix crash if skip_map is used. Fix importing static maps from UCL. Fix parsing of unix sockets. Fix raw_mime regexp on HTML part with no text content. Fix tables logging. Fix vertical tab handling in libucl. Try to fix frequency counters. Use better sharding for ip_score. Use multiple results from SURBL DNS reply. When doing AV scan select a different server for retransmit.

1.7.401 May 2018 17:46 minor bugfix: Major stock config updates. Make more sane fuzzy_check default settings. Fix ucl escape for bad symbols. Add failure symbol for AV module. Add lazy expiration mode for new classifier schema. Add preliminary version of maps stats plugin. Allow to block fuzzy requests from specific networks. Allow to change `expire` of live statistics. Distinguish AV failure from clean result. Further improvements of language detector by using khash. Further optimization of the lang_detection. Implement cluster-aware bayes expiry. Implement exclude patterns in rspamc. Implement glob maps in addition to regexp maps. Implement map statistics function for lua API. Implement stop symbols for Clickhouse collection. Support recipients separated by commas. Try harder to upload scripts to the Redis server. Upgrade t1ha distribution. use_domain_sign_inbound. Use scores from maps if `symbols_set` is not defined. Add resolving version of radix map helper. Check URL before adding implicit prefix. Do not check pid/state when using PRNG. Fix CentOS logrotate script for systemd. Fix slash + dot in urls. Fix systemd version of the logrotate script. Propagate key when import implicit array from Lua. Strip spaces from map keys and values. Try to fix a specific case when processing milter protocol. Try to fix crash when a tcp connection cannot be set. Typo use_domain_local -- use_domain_sign_local. Various fixes to once_received module. Store hits counters for map elements.

1.7.310 Apr 2018 17:30 minor bugfix: Plug bad memory leak in protocol reply. Add avx2 codec for base64. Add method to receive all URL flags from Lua API. Allow to fold headers on stop characters. Allow to set lua_cpath from options. Allow to specify custom rejection message in milter. Deal with unnormalised Unicode obfuscation. Do not detect language twice for relative parts. Implement oversigning feature. Implement silent logging level to minimize noise in logs. Improve URL_IN_SUBJECT rule. Use hashing to reduce redis attack surface. Add oversigning for the most important headers. add 'rewrite subject' to History dropdown. Another fix in folding algorithm. Do not call multimap addr for parts of addr if filter is presented. Do not clean hostname on generic reset. Do not create pid file in no-fork mode. Fix fold_after case to preserve multiple spaces. Fix folding and folding tests. Fix hostname usage in milter mode. Fix lua RSA verify and its tests. Fix metadata exporter send_mail backend (#2124). Fix processing of ' v' in libucl. Fix shemaless URLs detection. Fix support of multiple headers in sign_header. Fix usage of util.parse_mail_address. Fix weights of dynamic squeezed rules. Leak from bucket before checking the burst. Stop using own localtime as DST could be messy in many cases. Treat unnormalised URLs as obscured. Restore leaky bucket model in ratelimit plugin. Add messages total to throughput summary. Add symbols order selector to history. Config: Load list on demand. Fix modalBody for maps that appear more than once. History: Fix Tooltips on paging, filtering and sorting. Remove a previously-attached event handler. Update D3 to v5.0.0 and jQuery to v3.3.1.

1.7.223 Mar 2018 19:35 minor bugfix: Store emails in Clickhouse. Support single quotes in config. Use templates when publishing CH schema. Improve Docker image. Add rounding when printing a lot of FP variables. Allow to disable certain actions by assigning null to them. Disable results caching. Fix disabling of squeezed symbols. Fix scan time set. Rework logic of actions setting. Try to fix various Lua stack issues. Add link tag for favicon.ico. Display hostname:port/path in the page title.

1.7.120 Mar 2018 19:29 minor bugfix: Fix lowercase comparison. Timezone defines seconds WEST UTC not East. Add filename to log format. Add lua rules squeezing. Add related symbols analysis to rspamd_stats. Remove upstream `X-Spam: Yes` header by default. rspamd_stats: Output progress info on STDERR. Whitelist for emails module. Do not allow dependencies on self. Do not cache metric result. Do not trust all issuers as a client certificate. Fix dependencies in lua squeeze. Fix enabling/disabling squeezed rules. Fix enabling/disabling symbols. Fix external dependencies. Fix processing of a single compressed file. Fix some typos. Fix various modules in case of empty message. Handle callbacks that returns table of options. Improve cached action interaction. Make dynamic conf more NaN aware. Never hide actions from WebUI `configuration` tab. Implementation of Lua rules squeezing.

1.7.012 Mar 2018 21:50 major feature: Better machine learning support and embedding of Lua Torch. Language detection support based on ngramms and heuristics. New rspamadm configwizard command for a simple configuration setup. New statistics model for Redis backend allowing expiration and better analytics. Improved wizards for statistics conversion and management. Added automatic corpus test and rescoring utility based on Google Summer of Code 2017 project completed by @cpragadeesh. New Elasticsearch plugin. New experimental Reputation plugin. Various other important improvements and fixes.

1.6.616 Feb 2018 16:37 minor bugfix: Add sanity guards for badly broken HTML. Another errors path handling fix. Fix ARC chain verification. Fix crash in milter errors handler. Allow to insert headers into specific position. Allow to receive signing keys from mempool vars. Authentication-Results: support hiding usernames. Another try to deal with #1998. Another try to fix #1998. Better handling of the legacy protocol. Check decoded headers sanity (e.g. by excluding 0). Deal with nan and inf encoding in json/ucl. Deal with URLs wrapped in in text parts. DKIM signing: allow for auth_only to be false. Do not crash on empty subtype. Do not fail rbl plugin when there are no received or emails. Do not skip the last character. Do not try to dereference last character. Do not try to sign unknown domains. Exim Received header protocol parsing. First load selector_map and path_map. And only return false when domain not found if try_fallback is false. Fix bad archive characters stripping. Fix comparision. Fix connecting to a unix socket in rspamadm statconvert. Fix empty headers simple canonicalization. Fix extra hits in PCRE mode for regular expressions. Fix parsing of the per-user script. Fix processing of skip-hashes in fuzzy storage. Fix Redis timeout setup. Fix sanity checks on macro value. Fix text splitting: stack overflow (too many captures). Fix urls/emails distinguishing found in queries. F-PROT Antivirus: only check return code to determine infection. Metadata exporter: check IP sanity. Multimap: received: filtering of artificial header. Plan new event on HTTP errors. Plug another possible memory leak. Remove hop-by-hop headers in proxy. Sanitize IP in history redis. Setting check_local / check_authed in plugins (#1954). Settings: avoid checking invalid IP (#1981). Try harder in passing IPv6 addresses. WebUI: use relative path for savemap (#1943). Fix message count in throughput summary (#1724). Fix NaNs display on Throughput graph. Restore passwordless login support (#2003).

1.6.522 Oct 2017 18:00 minor bugfix: Another portion of tokenization fixes. Fix memory leak in spf caching logic. Fix milter commands pipelining. Fix newlines detection. Filter nan and inf when adding scores. Implement headers flags in mime parser. Support Expires header when using HTTP maps. Actively load skip hashes map in fuzzy storage. Add workaround for IPv6 in sendmail. Authentication Results: Fix SPF smtp.mail_from. Check for magic when checking for an archive. Deal with another case when processing exceptions. Deal with URLs with no slashes after protocol. Do not allow garbadge when checking url domain. Do not ignore short words. Do not strip last character in the last word. Do not treat script content as text. Erase unknown HTML entities. Fix another tokenization issue. Fix DKIM forgeries via multiple headers. Fix emails detection. Fix empty threshold check in greylisting module. Fix enormous scores for R_WHITE_ON_WHITE. Fix loading of per-user redis backend for statistics. Fix multiple headers in DKIM headers list. Fix obscured url in format user@@example.com. Further tokenization fixes. Load skip map from all processes as shared cache is unavailable. Lowercase words. Milter headers: skip_local / skip_authenticated settings. Milter headers: X-Spamd-Result header if X-Virus ran first. Ratelimit: fix whitelisted_rcpts matching. Some more fixes towards emails detection. SpamAssassin: Fail check_freemail_header if regexp didn't match. Use greylisting threshold in greylisting module.

1.6.410 Sep 2017 11:36 minor bugfix: Add method to get all content-type attributes in Lua. Add some sanity checks for actions and controller. Allow randomly select User-Agent from a list. Deal with obscured URLs with @ symbols. Milter headers: support adding/removing arbitrary headers from config. Add another workaround to display history properly. Add missing rspamadm control options to help. Auth-Results: Multiple DKIM signatures. Crash in URL processing. Default monitoring domain for surbl plugin. Detection of maillist optimized and fixed. Do not cache SPF records with PTR elements. Fix blacklists and DMARC in whitelist. Fix exceptions list in surbl. Fix processing of closed tags. Fix PTR processing in SPF. Lowercase HTTP headers to make them searchable from Lua. Fix options.local_networks setting. Ratelimit: lowercase email addresses. Rebalance and slightly rework MX check plugin. Fix Redis script loading in DMARC; URL tags; URL reputation. Reject invalid bh for DKIM signatures earlier. Remove incorrect method `task:set_metric_subject`. Fix rewriting subjects via force actions module. Fix RPM postinstall. Treat 'rewrite subject' as spam action. Try harder to find urls. Use full URL when making an HTTP request. Use raw urls when sending requests to redirector. Use weight from map for fuzzy scoring. Penalise R_BAD_CTE_7BIT for utf8 messages.

1.6.327 Jul 2017 08:13 minor bugfix: Fix semicolons parsing in the content type. Add EBL to the default config. Allow to configure monitored. Allow to skip specific hashes in fuzzy storage. Multimap: checking of symbol options. Redis settings: support checking multiple keys. ARC: Fix Lua 5.3 compatibility; timestamp should be integer. Avoid changing content-transfer-encoding header's value. Don't use whitelist/greylist maps as regexp, but as map. Fix get_content method. Header checks: Fix get_raw_header method. Header checks: REPLYTO_UNPARSEABLE rule. Lua_http: freeing. Milter headers: custom headers: removing headers. Parse HREF urls without explicit prefix. WHITE_ON_WHITE: Ensure score is matched to part that fired the rule. Escape strings inside HTML in history.

1.6.208 Jul 2017 09:46 major bugfix: Remove Rambler email bl for now. Switch RAMBLER_URIBL to a locally managed source. Switch from ragel to C for Content-Type parsing. Add `-e` option for lua_repl. Add per-domain emails normalisation rules. Add sessions cache to debug dangling sessions. Add short_text_direct_hash for fuzzy check module. Add text_part:get_stats function. Allow to add custom processing script for surbl. Allow to check reply-to email. Allow to customize spam header, remove existing spam headers. Allow to disable specific workers in the config. Allow to discard messages instead of rejection. Allow to specify custom delimiter in emails plugin. Allow to specify custom User-Agent for rspamc. Allow to store symbols data in Clickhouse. Allow to use HTTPS when connecting to Clickhouse. Enable sessions cache tracking for milter connections. Implement per-line mode in lua_repl (like `perl -p`). Implement rdns-curve plugin based on rspamd cryptobox. Improve maps cached data lifetime. Improve maps checking frequency. Improve monitored timeouts logic. milter_headers: add `extended_headers_rcpt` option. Milter headers: Add X-Spam-Flag to rmilter-compatibility headers. Milter headers: remove-header routine. Multimap: received filters for extracting TLDs from hostnames. Normalize email aliases in emails module. Re-add rambler email bl (as hashed list). Reload file maps more frequently. Rework newlines strip parser one more time. Skip updates for messages scanned via controller. Split long DKIM public keys. Store more data when stripping newlines. Support SPF macros transformations. Support suppressing DMARC reports for some domains. Add missing `break` statement. Allow modifiers in SPF macros. DKIM sign tools: edge-cases around use_esld. Do not cache SPF records with macros. Do not overwrite score when setting pre-action. Fix comparision logic. Fix DKIM base64 folding for milter flagged messages. Fix emails module configuration. Fix folding for arc headers when milter interface is used. Fix gmail dots.

1.6.114 Jun 2017 14:53 minor bugfix: Allow to init resolver without rspamd_config. Do not crash when resolver failed to initialize. Fix abstract context layout. Fix CGP helper reply parsing. Fix crashes when socket write errors occur. Fix parsing IPv6 nameservers in resolv.conf. Milter: Don't defer on "greylist" action.

1.6.012 Jun 2017 13:58 major feature: Milter protocol support. ARC support (signing and validation). New statistics model for Redis storage. New expiration algorithm for internal caches. DMARC reporting support. Spamtrap plugin. URL redirector improvements. Compression support in proxy. Various performance improvements. Miscellaneous changes/fixes. https://rspamd.com/announce/2017/06/12/rspamd-1.6.0.html

1.5.901 Jun 2017 21:10 minor bugfix: Increase min_bytes to avoid FP. Remove ratelimits from default configuration. Fix accepting on IPv6 sockets. Zero fill sockaddr_un. Add `text_multiplier` param. Check min_bytes for images as well. Do not add exact hashes from different parts. Fix memory leak when accepting from unix sockets. Fix some more issues about duplicated fuzzy requests. Phishing: fixed strict_domains. Skip text parts when checking binary parts in fuzzy check. Add the same duplicates protection for all fuzzy hashes types. SPOOF_DISPLAY_NAME: Use all SMTP/MIME recipients; validate assumed spoofed display name domains to contain a dot.

1.5.823 May 2017 18:18 minor bugfix: Fix memory leak in fuzzy check. Fix memory leak in maps scheduling. Multimap: email:domain:tld filter. DKIM Signing: avoid nil index when From header is missing. Do not set bayes probability if we don't use it. Do not stop on illegal unicode points - replace them. Fix brain-damaged spamc protocol for now. Fix Redis timeout set. Fix spamc support in rspamd proxy. Fix Multimap Received IP filters with Redis. Parse rgb a (x,x,x ,x ) css colors. Reresolve upstreams even if there is a single server there. Treat empty address as valid. Try harder to detect CTE. Try to deal with v4 mapped to v6 addresses on accept. Add `wsf` and `hta` bad archive extensions. Fix configuration option. Fix result parsing for SAVAPI. Logging improvements. Prevent MID_CONTAINS_FROM from firing on empty address. Reduce digit- number transmission penalty. Relax CTYPE_MISSING_DISPOSITION rule.

1.5.702 May 2017 13:34 minor bugfix: Fix corruption when multiple fuzzy are defined. Fix learn condition in fuzzy check. Add rules to detect bad 8bit characters in From and To. DKIM signing: sign_networks/local address specific use_domain settings. Support numeric arguments for Redis requests. Deal with 8bit characters in email addresses. Fix couple of issues in FORWARDED rule. Fix passing data to log helper when many symbols defined. Fix R_BAD_CTE_7BIT rule. Fix REPLYTO_UNPARSEABLE rule. Fix setting of email address. Rspamadm grep: Disable Lua patterns in string search by default. Add Lua 5.3 workaround. Add lua methods to detect if a part has 8bit characters. Allow session-less lua dns requests. Allow to append greylist end time to message reported. Avoid `nil` table. Disable dkim_signing if redis is specified but not configured. Fix build with pcre2. Fix rule. Fix warnings. Format floating point number. Push email flags to the lua API. Silence some warnings. Silence warning. Try all hostname regexps to find the most significant one. Fix add header filter in history.

1.5.619 Apr 2017 17:29 minor bugfix: Add unigramms support in bayes. Allow configurable sign headers for DKIM. Allow to add unigramm metatokens from Lua. DKIM Signing: envelope match exception for local IPs. Always try to adjust filename. Do extra copy to ensure that original content is never touched. Fix SPOOF_REPLYTO rule. Ignore Rmilter added Received. More fixes for hashed email dnsbls. Plug memory leak in chartable module. Display multiple alerts at once in WebUI.

1.5.510 Apr 2017 10:32 minor bugfix: Fix classifier learning with Redis backend. Fix issue when parsing encoded rfc822/messages. Add escaped version of lua_ucl import. Add task:headers_foreach function. Allow to process filenames from content type. Allow to query hashed emails. Ignore bayes with mostly metatokens or with too few text. Probabilistically skip metatokens. Retrieve all virus names from SAVAPI. Rework classifiers lua metatokens. Store headers order. Store text tokens inside bayes tokens. Use cached shingles keys. Add missing score normalisation for HFILTER_URL_ONLY. Avoid lookup in absent hash. Check return values from Lua functions called from C. Do not count sending and loading time in rspamc. Escape json strings for controller rejplies from Lua. Fix archive scans for savapi. Fix domain_only emails RBL. Fix ip_score map configuration. Fix JSON output for history_redis. Fix one character length substrings search. Fix parsing of non-RFC compatible Exim received. Fix parsing of options for workers with the same type. Fix processing of small tokens vectors. Fix rfc2047 tokenization. Fix typo. More fixes for inplace decoding. Try to avoid modifications of the original data. URL redirector: Fix call to is_redirector. Set token data as uint64_t instead of chars array. Check if neighbours' history backend versions match. Disable phrase connectors replacement in history filtering. Disable phrase connectors replacement in symbols filtering. Do not hide messages with bad subject, just replace it with '???'. Fix error message. Fix history v2 display. Fix legacy history. history: break To address lists on commas. Increase default timeout to 20 seconds. Save some history table space.

1.5.428 Mar 2017 15:57 minor bugfix: Add history_redis default configuration. Add spoofed rules. Add URL_IN_SUBJECT rule. Allow to get task's subject. Allow to specify maximum number of shots for symbols. Distinguish URLs found in Subject. Memoize LPEG grammars. Parse else parts in SA rules. Process subject for mixed characters. Resolve url chains in url_redirector module. Stat greylisted messages as greylisted not soft-rejected. Support checking for redirector in Lua SURBL. Support tag_exists SA function. Work with broken rfc2047 tokens. Check all watcher's dependencies. Do not compile hyperscan with no SSSE3 support. Do not crash if cannot decode qp encoded part. Fix dependencies of DKIM when multiple signatures are found. Fix lists in whitelist plugin. Fix one-shot symbols weight calculations. Fix options and shots match. Fix order of symbol options. Fix parsing of dot at the end of the address. Fix parsing of lua table arguments. Fix processing of subject words. Fix string split memoization. Fix templates grammar usage. Fix various issues related to Lua stack manipulation. Force actions: Use postfilter if we have honor_action / require_action. Further fixes to avoid PHISHING FP. Preserve order of options in symbols. Rspamadm grep: deal with unusually-formatted logs. Use hostname suffix when dealing with history. Remove outdated SA rules. Add flexible columns. Add footable. Add sender, recipients and subject columns. Allow message-id break. Fix history clustering. Fix history display. Fix sorting. Humanize sizes. Initial move towards footable. Remove datatables. Replace `.values` method with `.map`. Rework v2 symbols display. Try to normalize frequencies. Unbreak WebUI. Use Footable to draw Throughput summary table.

1.5.317 Mar 2017 14:05 major bugfix: Add composite for hacked wordpress phishing. Fix base64 decoding when there are unparseable characters. Additional symbol metadata in metadata exporter. Add method to get protocol reply from Lua. Add symbols when tagged rcpt/sender are normalised. Add task:get_symbols_all() function. Allow multiple formats of DKIM signing key. Allow to cache and use flexible protocol reply. Allow to set one_shot flag from register_symbol. Allow to skip certain types of hashes when learning fuzzy. Cache and insert scan time into the protocol. Detect newlines in rspamc --mime. DKIM signing: support use of maps. Greylist: Support excluding low-scoring messages from greylisting. Implement lua history in controller. Implement redis history querying. Preliminary implementation of redis history plugin. Support using request headers in settings. Change default template to deal with non-ASCII characters. Deal with lists of maps in whitelist module. DKIM signing: use domain-specific signing key. Do not reallocate completed zstd buffer. Do not use local_addrs in proxy. Fix crash when resolver is undefined. Fix double free when closing lua_tcp connections. Fix for lua 5.3. Fix freeing of arrays iterators. Fix issue with task:get_symbol and symbols with no metric. Fix log line duplication in `rspamadm grep`. Fix memory corruption on termination. Fix out-of-bound access in base64 decode. Fix ratelimit + greylisting. Fix subject rewriting. Fix task:set_recipients function. Fix URI_COUNT_ODD rule. Follow the traditional symbols conventions in RCPT_COUNT rule. Greylist: Suppress greylist action for whitelisted hosts too. Metadata exporter: use rule-specific settings for emails. Properly set missing fields in exporter. Proxy: max_retries option. RCPT_COUNT fixes. Rework HAS_X_PRIO rule to match symbols conventions. Update issues in ac-trie. Use optimised base64 decoding in DKIM. Add preliminary v2 history parser. Allow different history parsers. Display symbols. Rework history v2 function.

1.5.207 Mar 2017 16:49 minor bugfix: Antivirus: use scanner-specific redis prefix. Couple of fixes for DKIM signing module. Distinguish missing and broken mandatory headers. Do more heuristical detection for missing CTE. Do not resort cache on each check. Fix CGP escaping. Fix MISSING_MIME_VERSION rule for plain messages. Fix parsing of cte in expressions. Fix partial matches in rspamadm grep. Fix setting class on style field. Add default config for spamassasssin plugin. Add default configuration for antivirus module. Add dkim signing docs. Add mx_check default config. Add replies config. Add trie default config. Add heuristic to find text parts in files. Add rule to detect broken content type. Allow to extract CTE in Lua API. Allow to set from address for a lua_task. Allow to set recipients of a task from Lua. Enchance text_part:get_content method. Remove + aliases from emails. Support rmilter block and dkim signature in CGP helper. Support running event loop from Lua. Auto-switch Throughput units to `msg/min` for very low rate. Update D3Evolution to 0.0.2.

1.5.102 Mar 2017 17:55 minor bugfix: Fixed processing order for postfilters. Fixed DKIM signing plugin. Fixed ClamAV support in antivirus module.

1.5.001 Mar 2017 15:33 major feature: New MIME parser. Switching from libiconv to libicu. WebUI rework. Lua TCP module rework. URL redirector module. Rmilter headers module. DKIM signing module. Force actions module. Reworked improved metadata exporter. URL tags plugin. URL reputation plugin. Multimap received maps. Changes in RBL module. Support for Avira SAVAPI in antivirus module. Neural net plugin improvements. Fuzzy matching for images. New rules. Rspamadm grep. Performance improvements. Stability improvements and bug fixes. https://rspamd.com/announce/2017/03/01/rspamd-1.5.0.html

1.4.416 Feb 2017 11:24 minor bugfix: Fix bad memory leak in TLS certificates validation. Fix memory leak in HTTP maps. Replies module: fix symbol weight. Add missing macro. Do not treat IDN urls as bad URLs. Fix build with gcc 4.2. Fix cast. One more definition missed on Linux.

1.4.318 Jan 2017 14:29 minor bugfix: Plugged memory descriptor leaks; fixed race conditions other issues.

1.4.206 Jan 2017 14:32 minor bugfix: Fixes for DKIM parsing signing, fuzzy redis backend, dependencies in SpamAssassin plugin, URL detection, trie, fann_redis a memory leak. Added toggle for disabling SURBLs. New rules. Other features/bugfixes.

1.4.130 Nov 2016 15:14 minor bugfix: Clickhouse plugin. Support for embedding maps in config. Lua modules debugging improvements. New rules. Multiple fixes to the ANN module. Other bugfixes. https://rspamd.com/announce/2016/11/30/rspamd-1.4.1.html

1.4.021 Nov 2016 12:55 major feature: Redis pool support New neural nets plugin. Neural nets plugin has been reworked to store both training vectors and neural nets in Redis. Bayes improvements. New Antivirus plugin. New MX check plugin. Compression support in the protocol. Reworked model for DNS failures in SPF, DKIM and DMARC. Adaptive user-defined ratelimits. Monitored objects. Redis backend for fuzzy storage. Delhash support for fuzzy storage. Metric exporter and metadata exporter. Dynamic configuration in redis. Users settings in Redis. Errors ring buffer. Messages rework. Multiple updates to Rspamd Lua API. Rules improvements. WebUI improvements. Bugfixes. https://rspamd.com/announce/2016/11/21/rspamd-1.4.0.html

1.3.501 Sep 2016 12:41 minor bugfix: Fix issue with finding of end of lines pointers. Add DMARC_NA / R_DKIM_NA / R_SPF_NA / AUTH_NA symbols. Add spamhaus DROP dnsbl. Add termination callbacks for workers. Add user-agent for rspamc. Allow parsing of mailbox messages from the commandline. Support excluding selected users from ratelimits Yield DMARC_DNSFAIL on lookup failure. Banish table.maxn from Lua parts. Do not use headers to calculate messages digests Fix absurd scores for HFILTER_URL_ONLY. Fix actrie patterns. Fix format string. Fix handling of ' 0' in lua_tcp. Fix issue with empty messages and dkim. Fix min_learns option. Fix options for SPF dnsfail symbol. Fix processing of symbols when reject limit is reached. Settings: fix `authenticated` parameter (#886). Try to avoid race condition when using rrd. Use the correct macro to get the size of controller. Fix build and tests on OpenBSD.

1.3.423 Aug 2016 12:41 minor bugfix: Rspamd reload command has been fixed. Better ASN/country support. Variable maps in the multimap module. DNNSEC stub resolver support. DMARC and DKIM module fixes. Statistics backend now uses global redis settings. Tasks checksums. DKIM signature header is now folded by Rspamd. MIME messages. Ratelimit module fixed. Fixed X-Forwarded-For header processing. https://rspamd.com/announce/2016/08/22/rspamd-1.3.4.html

1.3.315 Aug 2016 14:12 minor bugfix: Important bugfix for fuzzy hashes (please update). Support for redis maps in multimap plugin. Important fix for hyperscan cache. SARBL URIBL added to default configuration. Lua API fixes and improvements. https://rspamd.com/announce/2016/08/15/rspamd-1.3.3.html

1.3.208 Aug 2016 17:06 minor bugfix: Fixed multi value header processing. Attributes in HTML tags are now HTML decoded. Fixed processing of last element of DMARC record. Reworked Hfilter module. New SPF plugin features. Better HTML support. Improved settings matches for authorized users. Redis integration fix. DKIM signing fixes improvements. URL detection fixes.

1.3.101 Aug 2016 17:42 minor bugfix: Removed systemd socket activation. New features in multimap: conditional maps; support for multiple symbols and scores per map; hostname map type. Bugfixes for greylist module. Fixed Catena password scheme. Fixed writing of Hyperscan cache. Implemented message size limit. Fixed configuration files includes logic. Other bugfixes and improvements. https://rspamd.com/announce/2016/08/01/rspamd-1.3.1.html

1.3.025 Jul 2016 14:30 major feature: Proxy Support. Fuzzy storage mirroring. Redis replication support. Improved content filtering. Internal greylisting supporting. Replies module. DKIM signing support. WebUI improvements. Many other changes that improve stability, filtering quality and performance. Bug fixes to URL detection logic and phishing detection. The chartable module has been completely rewritten to provide more useful homograph detection. https://rspamd.com/announce/2016/07/25/rspamd-1.3.0.html https://github.com/vstakhov/rspamd/blob/master/ChangeLog

1.2.820 Jun 2016 11:10 major bugfix: Another fix for exim workaround (#637). Fix unencrypted passwords processing in the controller. Fix setting path for lua (#652). Fix usage of rdns reply structure (#654). Use file lock in logger to avoid deadlocks. Add `application/octet-stream` mime type for `pdf` extension (by @moisseev). Implement new automata to skip empty lines for dkim signing (#651). Fix parsing of missing classes. Clarify some rspamc arguments (by @fatalbanana). Correct suppress spelling.

1.2.725 May 2016 16:31 minor bugfix: Slightly reduce weights of rules with high FP rate. Add workaround for rspamd-1.3. Fix possible FP in TRACKER_ID rule. Simplify MISSING_MIMEOLE rule. Add workaround for gmime CTE stupidity. Fix mime headers processing. Fix false positive URL detections in text parts. Fix Exim shutdown patch. Enable workaround for exim mailbox format. Backport shingles static test. Fix levenshtein distance calculations. Fix max_train setup in ANN module. Fix redis structure by adding NULL, NULL member. Fix build with unmodified LibreSSL opensslv.h. Repair optional dependencies. Really skip filters in case of pre-result set. Restore the intended pre-filters behaviour. Fix ipv6 mask application.

1.2.629 Apr 2016 15:49 minor bugfix: Fix parsing of URLs in texts. Fix creating of URLs from LUA. Fix some more URL detector issues. Fix unit tests. Fix JIT compilation for PCRE2 expressions. Fix JIT usage for PCRE2. Fix UTF8 mode in PCRE2. Add workaround for pre-historic compilers (#605). Fix and rescore R_PARTS_DIFFER logic. Properly set lua paths for tests. Fix SA rawbody processing - exclude top part. Store text parts content with newlines stripped. Properly support SA body regexps. Fix body rules in SA plugin. Fix setting of score for parts differ. More fixes to parts distance calculations. Fix percents output in R_PARTS_DIFFER. Plug memory leak in dkim module. Plug minor memory leak in regexps creation.

1.2.522 Apr 2016 20:52 minor bugfix: Plug an important memory leak in headers getting code. Remove some bad domains from whitelists.

1.2.420 Apr 2016 14:41 minor bugfix: Implement new multipattern matcher that uses hyperscan if possible. Use mutlipattern for lua_trie code. Add utility methods for multipattern. Use multipattern in url matcher. Add escape functions for hyperscan. Allow to optimize lua - C transition by flattening table args. Optimize hot paths in SA plugin. Optimize rspamd_re_cache_type_from_string. Allow empty tries. Fix extraction of URLs from Subject. Allow to have different flags for different patterns in multipattern. Add common directory for hyperscan cache to config. Implement caching for hyperscan multipattern. Attach domain part to `R_SUSPICIOUS_URL` (by @moisseev). Allow multipattern scans to be nested for the case of hyperscan. Simplify SURBL redirector search code and avoid ac_trie. Add two way substring search algorithm. Avoid acism usage to find gtube pattern. Fix processing of empty headers. Allow to disable pthread mutexes on broken platforms. Make web interface not send password in query strings (#585) by @fatalbanana. Add maximum delay to ratelimit module. Backport fix for empty files inclusion from libucl. Fix settings id setup. Add min_learns option to classifiers. Use more clever to utf8 conversion strategy. Fix disabling of virtual symbols in the settings. Rework settings to work properly in metric-less configuration. Set the default limit for classifier. Fix ttl based expiration from LRU cache. Rework DKIM module to use OpenSSL for digests. Fix mailto urls parsing with hyperscan. Do not set obscured flag for urls starting with spaces. Fix crash on redis learn. Fix ratelimit ctime setting.

1.2.312 Apr 2016 12:13 minor bugfix: New DCC module (by @smfreegard). Rework whitelist module. Fix regexps results combination (*critical*). Fix issue with expressions processing (*critical*). Optimize strlcpy for aligned input. Add support of half-closed connection in lua_tcp. Allow to print compact json in client. Save required score in history (#581). Allow to attach file descriptors to control commands. Allow to send descriptors from workers to main. Allow to attach fd when broadcasting to workers. Implement log pipe feature for rspamd logs analysis. Add `log_helper` worker. Add `URIBL_SBL_CSS` (by @smfreegard). Add worker scripts functionality. Add on load hooks for rspamd_config. Add lua scripts for log_helper worker. Add generic maillist detector (#584). Implement FANN autolearn using log_helper worker. Rework metrics configuration to allow includes. Change default value of forced removal in composite rules. Allow to use assembly version of blake2b on x86 cpu. Use less precise (but faster) clock if possible. Insert redirected URL to the urls list. Allow to get and set callback data for rspamd symbols. Add binary heap implementation. Use binary heap for expire algorithms in the hash. Use `least frequent used` expiration strategy. Allow to get mime headers from a task. Add support for mime headers in `regexp` module. Update Exim patches (by @fatalbanana). Allow building rspamd with jemalloc. Save multipart boundaries. SA plugin changes. LUA API changes.

1.2.201 Apr 2016 14:23 minor bugfix: Use HTTP Content-Type on non mime input if possible. Save log level when compressing log messages. Further rework of composite rules (add ' ' prefix). Add tracking for rspamd expressions. Store actions limits in metric result. Fix parsing of include/redirect with many records in SPF. Add method to disable symbols execution in the cache. Allow to disable checks from settings. Allow to select settings by id in HTTP query. Find URLs with ' r' and ' n' inside href attribute. Implement vectored mode for hyperscan (experimental). Improve client connection errors diagnostics. Allow to edit new files with signtool. Improve hashes performance on 32 bit platforms. Fix sorting of limits. Remove slow and unused rules `INVALID_EXIM_RECEIVED*`. Add expression:process_traced lua method. Allow tables in task:insert_result. Save trace for SA metas. Do not parse broken TLD parts in URLs. Investigate many border cases in URLs parser.

1.2.125 Mar 2016 16:18 minor bugfix: Add list support to `mime types` module configuration (by @moisseev). Allow symbols params to be printed in logs. Fix `MIME_BAD_ATTACHMENT` false positives for MDN/DSN. Fix crashes on arm32. Do not classify message if some class is missing. Fix cryptobox cleanup. Remove multipart/report from bad mime types (#569). Improve logging for fuzzy hashes. Show map URLs in webui. Sort symbols in webui.

1.2.024 Mar 2016 02:05 major feature: New dynamic updates plugin. Regular expressions map support. Faster radix trie algorithm. Faster siphash for AVX2 supporing CPUs (used in fuzzy hashes). PCRE2 support. Allow quoted and slashed keys in map. Add proper support of DNS resolvers balancing. Rework includes and configuration system for better local changes support. New keypairs framework for signing and encryption. Added support for dynamic modules and workers. Allow to dump configuration with help comments. Rework once_received module. - priority for `good_hosts`. - If a good host has been found do not add once_received symbols. - priorities for strict once_received. - Add ability to whitelist IP addresses. Implement support of signed maps for HTTP and file maps. Add command to sync fuzzy storage. Rework system of symbols and actions registration It is possible now to use priorities when adding symbols to metrics and override scores for symbols with lower priority with the scores with high priority. Add auth support and db selection for redis stats. Improve composite rules application. Add ignore_received option. critical with inconsistent resorting. `all` in spf redirects. Add punycoded versions for IDN domains. Improve sorting order for symbols cache. Add lockless logging for processes management. Allow to specify flags for metric symbols. Load images height and width from style attribute. Override DNS requests limits for SPF and DKIM. resetting symbols to their default values in WebUI. Improve configuration agility for redis stats. Allow to set db and password for redis in stat_convert. Import the latest libucl. LUA API changes: - Add rspamd_version function to LUA API. - Add lua_cryptobox module. - Add lua_map module. - Add task:set_metric_action lua API method. - race condition in lua_tcp module. - a lot of in lua_redis module. - Rework and abstract lua maps API. - Add util.strlen_utf8 lua function. - Add lua functions for caseless comparison. - Allow optional symbols registration. - Add config:add_

1.1.422 Feb 2016 16:03 minor bugfix: Print traceback on lua errors in lua config. Fix leaks in lua error paths. Improve 'R_EMPTY_IMAGE' rule. Fix metas memoization in SA plugin. Properly set `flag` in fuzzy replies. Fix arguments order. Fix issue with out-of-boundary reading. Fix issues found by coverity. Fix error in printing hex. Reduce weights for some hfilter patterns. Add aliases for task:get_from_ip: task:get_addr, task:get_from_addr, task:get_ip. Rework once_received module. Fix `MISSING_MIMEOLE` rule for modern OE. Treat meta tags as embedded tags (#501).

1.1.305 Feb 2016 03:15 minor bugfix: DSN rules when SMTP from is unavailable. statconvert routine to avoid lua module usage. Set a sane quark for configtest to avoid NULL to be printed in logs. Support c11 if available. parsing of ip:port strings. Add more diagnostic for lua subr errors. task:set_from_ip lua method. Add basic routines for digital signatures. Add tool for digital signatures. Add plain open file API method for atomic open. parsing nested braces inside logger vars. Pre filters now actually skip processing. Add pre-filter mode for multimap. Switch to apache 2 license.

1.1.229 Jan 2016 16:24 minor bugfix: * Fix stat_cache closing * Add checkpoints to sqlite3 learn cache * Do not recompile lua generated headers all the time * Increase number of messages learned * Fix issues with dual stack and hfilter * Disable MID checks for hfilter by default * Fix cache definitions in multiple classifier and no type * Don't crash if learn cache failed to initialize * Fix googlegroups support in maillist plugin * Rework flags LUA API * Add `BROKEN_HEADERS` rule * Add support for forged confirmation headers (by @AdUser) * Allow `any`, `mime` and `smtp` for get_from/get_recipients * Add mime types checking plugin * Add rule to detect spammers attempts to cheat mime parsing * Rework parsing of IP addresses in configuration (better IPv6 support) * Add `util.parse_mail_address` function to LUA API * Add lua sqlite3 module * Implement synchronous redis call * Ratelimit: avoid possible indexing of nil value (Fixes #498) (by @fatalbanana) * Add stat_convert command to convert stats tokens from sqlite3 to redis * Implement redis advanced lua api with pipelining * Fix memory leak on redis stat (#500) * Fix user/language learn count in sqlite statistics (#496) (by @fatalbanana) * Fix build with custom pcre * Fix fuzzy relearning (#498) * Improve planning of asynchronous tasks * Show slow rules in log * Add warning for slow regexps * Add base32 decode/encode routines to lua util * Allow converting of learn cache from sqlite to redis * Add methods to check if a messages has from/rcpts * Improve and fix multimap plugin * Disable reload command in rc scripts * Improve runtime CPU dispatcher for libcryptobox * Add preliminary support of digital signatures via ed25519 * Add detection for RDRAND support * Print configuration of crypto on start * A in SPF presumes AAAA lookup as well

1.1.121 Jan 2016 16:14 minor bugfix: * Fix duplicated XBL symbol * Reduce log severity for ratelimit missing servers * Fix XBL composite to avoid duplicate symbols * Reduce weight of URL_ONLY rule due to FP rate * Disable fuzzy hashes from the metadata for now * Fix processing of empty messages (#486) * Always treat DNS timeouts as temporary fail for SPF * Fix issue with SPF double IP stack (#483) * Use X-Forwarded-For when checking secure_ip (#488) * Fix hash calculation for sqlite stats * Fix memory corruption on punycode * Fix strings allocation in punycode * Fix error message (#491)

1.1.018 Jan 2016 16:35 major feature: * Incompatible change: sqlite3 and per_user behaviour * Implement redis statistics backend and cache * Implement autolearning for statistics * Reworked statistics architecture from scratch * Add hyperscan (https://github.com/01org/hyperscan) engine for regular expressions * Implement flexible task logging * Rework fuzzy worker * Various fixes to SPF * Add clang plugin for static analysis * Add 'allow_raw_input' option for non-mime messages * Recognize types using libmagic * Fix parsing of IPv6 received headers * Add new interface of communication between workers in rspamd * Add support for named socketpairs * Don't write URLs by default as it is too verbose * Set status for HTTP replies * Try load `rspamd.conf.override` * Implement words decaying for text parts to limit many checks * Improve support of SA rules and plugins * Many fixes to the URL parser * Match any newline character in regexps * Fix resolving of upstreams and detection of poor IPv6 configurations * Parse upstreams selection algorithm from the configuration line * Add `reresolve` command to the control interface * Generate fuzzy hashes from task metadata (URLs and headers) * Add method to check if IP is local and `local_addrs` option * Implement forced timeout for delayed filters * Disable fast path of pcre-jit as it seems to be broken * Bayes fixes * Rules changes * Plugins changes * Rework ratelimit plugin * Add neural network **experimental** plugin * Add a sample script to learn neural network from rspamd logs * Add documentation strings support to rspamd * Documentation improvements, tutorials section, statistics description * Many other minor and major bugfixes not noted here

1.0.1121 Dec 2015 19:40 minor bugfix: * Fix spf redirects * Fix domains when parsing mx/ptr/a records in includes/redirects * Fix unfolded base64 encoding * Fix GError use-after-free * Do not rewrite the original url when using redirector * Fix parsing of fragment in urls * Fix processing of HTML tags * Improve empty image rule * Avoid long double type * Fix tokens weights in OSB algorithm * Improve debugging for bayes

1.0.1006 Nov 2015 14:01 minor bugfix: * Fix settings application (#416) * Fix another issue with fixed strings * Fix hash function invocation * Use the proper string for make_dns_request in lua_http * Fix scan time output * Update webui: - fix labels for greylisting - fix dimension of scan time

1.0.928 Oct 2015 05:28 major bugfix: * Emergency fix in keyed blake2 to fix fuzzy hashes and encrypted password * Support passwords longer than 64 symbols * Add function to traverse AST atoms * Allow dependencies on rspamd symbols for SA metas * Fix memory corruption when timeout is removed in fuzzy check * Fix encrypted fuzzy add processing * Avoid use-after-free in controller session destructor * Use session pool instead of task pool in fuzzy check * Fix assembly in i386 mode (#413, #412)

1.0.726 Oct 2015 19:13 major bugfix: * Plugged memory leaks in internet address object html parser * Fixed static build * Fixed multiple sigchld processing * Fixed deletion of signal events after event processing loop * Fixed build on ARM (#404 - reported by @Gottox) * Fixed setting the default mask for SPF. * Fixed sanitisation of HTTP query values * Fixed parsing of the last header in encrypted HTTP messages * Additions and fixes for test suite benchmarks * Added openssl aes-256-gcm support to libcryptobox HTTP server * Implemented support for starting multiple HTTP servers * Implemented batch accept in HTTP server * Added module to get data from HTTP headers (#285 - reported by @msimerson) * Added `rspamadm control` command * Added ability to sort counters output. * Added ability to specify custom headers for rspamc client * Fix architecture detection * Converted history storage to the UCL format * Allow flexible number of rows in history * Fix action badges in WebUI * Add universal cryptobox hash API * Migrated to the optimized blake2b implementation adopted from Andrew Moon * Allow explicit loading of specific modules * Always load settings module * Allow to add symbols from settings * Fix double free in the controller fuzzy learn command * Avoid endless loop when cannot open sqlite db * Updated libucl

1.0.616 Oct 2015 13:37 major bugfix: * Fix build on i386 * Update CentOS7 service file patch (by @fatalbanana) * Fix path to rspamadm in Debian init script (by @fatalbanana) * Fix broken '_SC_GETPW_R_SIZE_MAX' on FreeBSD * Fix portability issues * Use cryptobox chacha for libottery * Better support of 32 bit builds * Fix header name tokens setup * Fix levenstein distance method for words * Add workaround for old libevent (#400) * Fix microseconds in termination timer * Fix some more issues with fixed strings * Explicitly test CPU instructions even after CPUID call * Do not check out of boundary memory * Do not output broken emails * Fix unknown symbols registration * Handle SIGILL using longjmp * Block signals when exiting event loop * Fix incorrect allocation size * Slightly optimize alignment * Restore rspamd -t for compatibility * Add more sanity checks for emails

1.0.514 Oct 2015 17:24 minor bugfix: * Add rspamd control interface: - support `stat` command to get runtime stats of rspamd workers - support `reload` command to reload runtime elements (e.g. sqlite3 databases) * Rework curve25519 library for modular design: - add Sandy2x implementation by Tung Chou - fix CPU detection for variables loading assembly - add testing for curve25519 ECDH * New fixed strings library * Add `R_SUSPICIOUS_IMAGES` rule * Enable mmap in sqlite3 * Use new strings in the HTTP code * Improve google perftools invocation * Improve performance profiling in http test * Reorganize includes to reduce namespace pollution * Allow specific sections printing in configdump command * Rework workers signals handlers to be chained if needed * Update socketpair utility function * Add control_path option for rspamd control protocol * Fix ownership when listening on UNIX sockets * Rework signals processing in main * Remove extra tools from rspamd (they live in rspamadm now) * Remove global rspamd_main * Add global timeout for the overall task processing (8 seconds by default) * Sanitize NULL values for fuzzy backend * Store NM between encrypt/decrypt * Add textpart:get_words_count method * Fix generic DNS request in lua * Tune hfilter weights * Add support of IPv6 in hfilter * Fix parsing of HTTP headers with IP addresses * Sync with the recent libucl * Various minor bugfixes

1.0.405 Oct 2015 14:13 minor bugfix: * Add configdump routine to rspamadm * Implement retransmits for fuzzy_check plugin * Fix events processing for learning anf checking fuzzy hashes * Avoid dependency on unneeded and uncompatible glib include * Add `historyreset` command to the controller * Fix loading of tokenizer config from dump (#389) * Add sorting hints for the history * Allow custom lua scripts for users/languages extraction (#388) * Do not add FORGED_RECIPIENTS when 'To' is missing (#387) * Do not add R_UNDISC_RCPT when 'To' is missing (#387) * Add encryption to fuzzy check plugin * Add encryption for fuzzy storage * Add new epoch for encrypted fuzzy request * Add encryption for `rspamd.com` storage * Remove gmime processing for LDA mode as it is deadly broken * Add routine to find end of headers position in mime messages * Fix LDA headers folding * Init libraries in rspamc client as well to avoid locale issues * Avoid collision with locally installed includes * Allocate and free memory with the same allocator in rspamadm (#385) * Preserve expired fuzzy hashes counter * Improvements in webui: - Add favicon.ico - Rework history table - Fix sorting for the history - Migrate to bootstrap 3 and jquery 2 - Fix css bugs - Add glyphicons - Add reset history - Improve history buttons - Redraw graph to avoid display issues - Webui is now MIT licensed to match licensing policy of rspamd

1.0.328 Sep 2015 16:28 minor bugfix: * Fix piechart clean slice (#380) * Fix controller crashes when GString is reallocated (#381) * Correctly set locale before start * Set C locale for numeric values * Add rspamadm routine: - add `pw` command to manage passwords - add `help` command for displaying help - add `configtest` command to check configuration files - add `keypair` command for generating encryption keys - add `fuzzy_merge` routine to merge fuzzy sqlite databases - add a simple manual page for rspamadm * Allow metric registration for composite expressions * Add strict mode for configtest * Add logger counters * Save and show learned messages count (#383) * Add `no_stat` flag * Add `task:set_flag` and `task:get_flags` (#382) * Enable foreign keys in sqlite3 * Remove orphaned shingles from fuzzy storage * Optimize synchronization steps for fuzzy storage * Allow delayed conditions registration * Add lua API for conditions registering

1.0.223 Sep 2015 17:01 major bugfix: * Fix critical bug in webui that prevents password from being sent * Rework webui view: * Add descriptions for whitelist maps * Fix build on arm (#379) * Fix issue with the last element in the radix trie * Add more tests for radix trie algorithm * Allow to extract URLs from query strings of other URLs (#361) * Initialize rrd fields before writing to file * Fix double free if no password has been specified

1.0.122 Sep 2015 16:08 minor bugfix: * Add writing to rrd from the controller * Fixed lots of bugs in rrd code * Adopt new DNS API in hfilter plugin (by @AlexeySa) * Allow only one controller process to manage rrd file * Set event base for fuzzy calls * Improve fuzzy IO errors logging * Add rra extraction function to rrd library * Add graph handler to the controller * Cache correct passwords to avoid too high CPU usage when working with webui * Controller sockets are owned by router do not export them to task * Optimize logging by skipping hash table search if it's empty * Fix loading issue whith broken statfiles * Print assertions from glib to rspamd logger * Load legacy `lua/rspamd.local.lua` * Update webui with some fixes to learning and scanning

1.0.017 Sep 2015 15:51 major feature: * Reworked symbols processing * Fixed URL redirector * New sqlite3 backend * Store tokenizer configuration within statfiles * Improve bayes statistics: * Rewritten url parsing * Fix settings plugin to allow custom actions scores * Improve rbl plugin * Allow capturing patterns in rspamd lua regexp library * Add GTUBE support * Fix spamc legacy support * Add DKIM support to RBL module * Fix issues with multiple DKIM signatures * Fix issue if rspamd cannot create statfiles (#331) * Reworked parts and task structure * Add ip_score plugin support (not enabled by default) * Improve PARTS_DIFFER rule to count merely different words * New HTML parser * New version of LUA DNS API * Table versions of many functions in LUA API * Improved rspamc client * Allow scanning of local files using HTTP requests * Reworked configuration system: * Rework and enable DMARC plugin * Add whitelist plugin for SPF/DKIM/DMARC based whitelisting * Add some common domains to whitelists shipped with rspamd * Reworked logging * Improved spamassassin plugin. * Resurrect rrd support code * Save controller stats between restarts * Fixed tonns of bugs * Added tonns of minor improvements and features * Added more unit tests * Create functional tests framework * Added documentation for missing modules * Added rpm/deb repositories and scripts * Updated WebUI and libucl externals

0.9.1019 Aug 2015 19:02 minor bugfix: * Do not dereference null pointer on learning. * Fix some extreme cases in BAYES. * Add a workaround to avoid bad HTML messages breaking. * Build with -O2 flags by default. * Add constraints to limit DNS requests count per task. * Add workaround for SURBL DNS flood. * Set error if rspamd cannot learn anything.

0.9.903 Jul 2015 14:07 minor bugfix: * Don't use RWL_SPAMHAUS_WL (unknown result) for whitelisting (by @fatalbanana) * Import updated public suffix list (by @fatalbanana) * Remove debug message * Fix settings (by @fatalbanana) * Remove duplicated symbol registration * Use WAL for fuzzy storage * RBL fixes (by @fatalbanana): silence errors; yield unknown results from RBLs; fix scoring for DNSWL; fix use of RBL name as symbol; ignore RBL names that would not be yielded * Support captures in regular expressions * Add captures support to lua_regexp * Support dist on FreeBSD and Darwin * Add RCVD_IN_DNSWL_NONE as whitelisting exclusion (by @fatalbanana) * Multiple fixes to URL detection: support port definition; fix query and path recognition; fix parsing of multiple slashes in URL; fix parsing query just after port; fix path field in `url:to_table` method; improve support of IP based URLs. * Set ignore_whitelists = true for RECEIVED_SPAMHAUS_XBL (by @fatalbanana) * Add GTUBE support * Ignore User header in SA mode

0.9.825 Jun 2015 15:21 major bugfix: * Fix critical bug in bayes classifier (#305) * Fix critical bug in RBL module (by @fatalbanana) * Fix and rework settings plugin. * Fix get_all_opts for a case of non-iterable options. * Use tld for redirector's matching.

0.9.723 Jun 2015 16:18 minor bugfix: * Add whitelist_exception setting to RBL module (by @fatalbanana) * Don't use RWL_MAILSPIKE_POSSIBLE or DNSWL_BLOCKED for whitelisting (by @fatalbanana) * Fix extreme cases in bayes classifier. * Fix parsing of urls with '?' at the end of hostname. * Update interface. * Fix number of issues with webui interaction. * Fix saving maps. * Allow user@ and @domain matches in multimap. * Fix issues with bounces From processing. * Fix abs/fabs misuse. * Fix builds on suse and arch linux distributions.

0.9.618 Jun 2015 12:37 major bugfix: * Fix memory leak if mime cannot be parsed. * Fix dkim cache expiration. * Fix issues with redirector HTTP response. * Fix abnormal connection closing with certains messages with a high score (issue #296) * Fix redirector installation. * Use specific POE loop for some systems. * Fix number of issues in URL redirector. * Fix selecting URLs for sending to redirector.

0.9.529 May 2015 12:24 major bugfix: * Avoid double free when extending HTTP message. * Fix double free if multiple classifiers are defined. * Fix misprint in spamassassin plugin. * Fix cpuid invocation on i386. * Fix ownership issues for zero-copy decode. * Allow __len metamethod on rspamd text . * Add base64 decoding lua utility. * Fix build on FreeBSD * Skip spaces at the beginning of mime messages. * DBL_ABUSE_REDIR should not have significant weight. * Allow to split by lua_regexp rspamd text objects. * Allow to specify custom stop pattern for lua_tcp.

0.9.421 May 2015 13:52 major bugfix: * Fix critical bugs in tokenization algorithm * Write unit tests for tokenization * Add documentation for lua_tcp * Switch off legacy tokenization by default. * Fix critical bugs in words normalization * Add lua bindings to tokenizer. * Implement storing of HTTP headers inside task * Add lua API to accerss HTTP headers data * Implemented base64 encoding suitable for MIME * Use caseless hash and equal functions for HTTP request headers. * Improve debian architectures support (by @dottedmag)

0.9.319 May 2015 16:25 major bugfix: * Revert incorrect regexp change that broke the default rules * Fix lua_tcp module

0.9.219 May 2015 15:58 major bugfix: * Fix error on spawning unique workers. * Add preliminary version of generic LUA TCP requests API. * Use lua 5.1 if luajit is not available (Arm64, PowerPC, s390x etc) * Fix fuzzy mime strings with only type. * Improve thunderbird sanity checks. * Fix critical bug on matching regular expressions. * Make hiredis optional dependency. * Fix multiple bugs in daemon reloading

0.9.117 May 2015 16:07 minor bugfix: * Restore utf8 validation for regular expressions to avoid crashes * Fix symbols displaying in the interface * Add symbol groups to the interface * Fix maps ID parsing in the controller * Add multimap and regexp modules documentation * Backport fixes from libucl * Fix debian package (by @dottedmag) * Rework XXH32 invocations

0.9.013 May 2015 21:11 major feature: * Add support of the fast and secure protocol level encryption: * Rework expressions and create new expressions library: * Many improvements to the LUA API: * LuaJIT is now the default requirement for rspamd allowing to speed up lua execution by a large margin (however, plain lua is still supported) * New plugins: SpamAssassin, DMARC * New aho-corasic trie implementation from @mischasan that allows to load and use hundreds of thousands of patterns with no influence on load * Support of PCRE JIT and PCRE JIT fast path modes that significantly improves the performance of regular expressions if supported by PCRE * New URLs parser and extractor * New statistics infrastructure * Reworked utf8 tokenization that previously corrupted all UTF8 words (minor incompatibility with old fuzzy hashes with utf-8 symbols) * SPF module has been completely rewritten to support complex cases of `include` and `redirect` within SPF records * DKIM module now supports multiple signatures * Controller passwords can now be stored encrypted by `PBKDF2-HMAC` in the configuration file * Many hand-written HTTP clients has been replaced with the common rspamd http module * New test framework * Randomize hash seed to avoid certain hash tables vulnerabilities * Documentation improvements: * Fixed tonns of bugs and memory leaks * Added tonns of minor features

0.8.315 Mar 2015 13:20 major bugfix: * Various critical fixes in distribution (by @dottedmag and @fatalbanana) * Fixed bugs in url detector to parse certain patterns * Add default host and helo for a client * Some sanity checks for tokenizer and classifier * Reiterate on systemd support * Fix missing symbol registration * Add support of spamc compatible output * Filter double-dots in rbl.lua validate_dns (by @fatalbanana) * Update ucl submodule due to critical bugfix

Rspamd 3.10.2

Recent Releases

Links

Submitted by

Manage

Share project