Dovecot Solr – Solving ‘FTS_Solr for Dovecot – Can’t Find Plugin File’

dovecotsolr

I'm trying to follow this to get Solr search setup with my Dovecot install:

https://doc.dovecot.org/configuration_manual/fts/solr/#fts-backend-solr

It all seems to work fine – and I can see the core in Solr admin. I'm using Solr 8.9.0, but it seems happy with the config files.

However, one of the commands it says to run to actually index:

doveadm fts rescan -u [email protected]

I get:

Fatal: Plugin 'fts_solr' not found from directory /usr/lib/dovecot/modules

Sure enough, I don't see anything in anything by that name:

root@east:/usr/lib/dovecot/modules# ls -lh *fts*
-rw-r--r-- 1 root root 263K Jul  7 17:17 lib20_fts_plugin.so
-rw-r--r-- 1 root root  71K Jul  7 17:17 lib21_fts_squat_plugin.so

What am I missing? Do I need to manually install the plugin as well?

dovecot --version
2.3.7.2 (3c910f64b)

UPDATE: Looking around, I can see dovecot-fts-solr a plugin (https://alpine.pkgs.org/3.15/alpine-main-aarch64/dovecot-fts-solr-2.3.17.1-r0.apk.html ) . I can't seem to get it to install through. Not too sure how to proceed? (I could justr download the .so file and put it into /usr/lib/dovecot/modules , but I'm pretty sure that won't work :))

UPDATE 2: So I managed to find:

apt-get install dovecot-solr

This creates the lib21_fts_solr_plugin.so file. However, when I try and re-index the messages I don't get anything:

doveadm -D fts rescan -u [email protected]
Debug: Loading modules from directory: /usr/lib/dovecot/modules
Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so
Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message)
Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message)
Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/libdoveadm_mail_crypt_plugin.so: undefined symbol: mail_crypt_box_get_pvt_digests (this is usually intentional, so just ignore this message)
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): Started userdb lookup
doveadm([email protected])<35131><>: Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Connecting
doveadm([email protected])<35131><>: Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Client connected (fd=10)
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): auth USER input: [email protected] uid=1011 gid=8 home=/home/hatlamp quota_rule=*:storage=0M
doveadm([email protected])<35131><>: Debug: auth-master: userdb lookup([email protected]): Finished userdb lookup ([email protected] uid=1011 gid=8 home=/home/hatlamp quota_rule=*:storage=0M)
doveadm([email protected])<35131><>: Debug: Added userdb setting: plugin/quota_rule=*:storage=0M
doveadm([email protected]): Debug: Effective uid=1011, gid=8, home=/home/hatlamp
doveadm([email protected]): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/hatlamp/mail/hatlamp.com/hello
doveadm([email protected]): Debug: maildir++: root=/home/hatlamp/mail/hatlamp.com/hello, index=, indexpvt=, control=, inbox=/home/hatlamp/mail/hatlamp.com/hello, alt=
doveadm([email protected]): Debug: auth-master: conn unix:/var/run/dovecot//auth-userdb: Disconnected: Connection closed (fd=10)

Looking at Solr's interface, there are no articles:

enter image description here

Best Answer

Use the FTS index, then you will see results in your fts backend.

Synchronising the internal state of full text content submitted to the FTS engine via the doveadm fts reindex command does not necessarily do much by itself. In the case of the solr binding, it simply moves the cursor back to zero.

This removes mails from the index that have already been expunged and makes sure that the next doveadm index will index all the missing mails (if any). Note that currently most FTS backends do not implement this properly, but instead they delete all the FTS indexes. This may change in the future versions. -- from man doveadm-fts, emphasis mine

Only when something (issuing a search, posting new mail, manual doveadm index) triggers updating the index after that command, that is when the data is sent (in your case, to solr).

Related Topic