Several new plugins/modules implemented

[sblinch]

I've made some additions to Maddy that were essential for my own deployment; I'm not sure if any of these would be useful to anyone else, or appropriate for the Maddy core, but I thought I'd offer them up for review just in case. I wouldn't consider them heavily battle-tested, but they've been in production for about 6 months on my own low-volume mail server.

Complete change list is here.

Some highlights:

• pattern check plugin, to whitelist/blacklist senders, recipients, hosts, and headers by pattern, regexp, or CIDR
• spamassassin check plugin, adding SA support
• geobl check plugin, to ban countries by connecting IP
• domainbl check plugin, to support SURBL/URIBL/etc
• safelist support to add "safelist" in addition to "reject", "ignore", etc., which overrides all other check plugins and ensures a message is delivered
• crypto storage module, a wrapper that transparently adds encryption support to any other storage blob module
• delivery attempt logging, to more easily grep for specific delivery incidents in production logs
• dnsbl improvements to log which dnsbl providers returned hits, and why and be more resiliant to dnsbl outages
• table storage blob module, to use any table module (such as sqlite) as blob storage
• require_matching_rdns_lax check plugin, handles a commonly-encountered corner case for require_matching_rdns (not sure if this is even relevant any more; I see there were some changes involving this code since I originally implemented it)

For documentation, for now, I've included a doc.go for each plugin/module that's formatted similarly to docs/reference/checks/*.md.

[foxcpp]

crypto storage module, a wrapper that transparently adds encryption support to any other storage blob module

Note that while it protects full message contents, storage.imapsql saves a lot of metadata in its database directly, therefore leaking information about the message, e.g. subject, sender, date, content-types.

[foxcpp]

I will go over these in detail this weekend and see how (if) these can integrated into core.

[sblinch]

storage.imapsql saves a lot of metadata in its database directly

Right! I should probably better document this. My use-case (which since changed, as I ended up needing to switch back to Dovecot for IMAP) was to host Maddy on a trusted server with full-disk encryption, but to use less-trusted S3 buckets for blob storage. This was just intended to add a crypto layer to Maddy's existing S3 support, but I built it as a module wrapper as it could be useful for other future remote blob storage modules.

I will go over these in detail this weekend and see how (if) these can integrated into core.

Sounds great.

[stellarpower]

What would be thoughts on architecting a plugin system (i.e. dynamic libraries) within Maddy? So that a recompile would not necessarily be required. I myself already have three usecases, and expect further over time:

• Detecting and logging bounce/rejection messages using Sisimai - I can do this with scripts, but the core now seems to be written in go, so would make sense to integrate it closely
• Server-side signature support
• Automatic out-of-office replies

[foxcpp]

Dynamic plugins are possible via https://pkg.go.dev/plugin. maddy's module system is well-separated enough to support that. However there are many considerations and drawbacks outlined in Go documentation - notably, requiring exact build environment match.

Statically linking additional modules during main executable compilation solves most of technical issues.
Also GPL licensing should be considered - any third-party plugin for maddy must use the same license.

In general, I prefer creating integrations that would allow for custom code to exist outside of server executable. For example, this is solved by Milter protocol for message checks (supported by maddy) and LDAP for custom delivery targets (also supported by maddy).

[foxcpp]

Converted into discussion, this is currently not an actionable issue. maddy module interfaces diverged significantly that it would be impossible to simply add these plugins into current source code.