roundcube and managesieve issues

[steve_adams]

Help!,

  I have roundcube installed with managesieve successfully creating filters, but I cannot seem to get the filters to work - even globally.

 

I'm using ubuntu 12.04 LTS, postfix 2.9.6 and dovecot 2.0.19.

 

I think it's my postfix configuration which reads:

 

dovecot          unix  -       n       n       -       -       pipe
        flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -c /etc/dovecot/conf.d/01-dovecot-postfix.conf -d ${recipient}

 

doveconf -n reveals lda is installed and managesieve plugin active:

 

protocols = imap pop3 sieve
service auth {
  unix_listener /var/spool/postfix/private/dovecot-auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-userdb {
    group = vmail
    user = vmail
  }
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }

...

protocol lda {
  deliver_log_format = msgid=%m: %$
  log_path = /var/log/sieve.log
  mail_plugins = sieve quota
  postmaster_address = postmaster@enventive.com
  quota_full_tempfail = yes
  rejection_reason = Your message to <%t> was automatically rejected:%n%r
}

 

any help will be appreciated

[bsc]

Hey junkpad92,

 

Thanks for the hint, really appreciate your help.

 

This also seems to work fine:

  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve

I've applied it to my productive environment, because it worked instantly.

[junkpad92]

Your config looks horrible.

 

A quick how-to - please reset your config before you do this steps!

apt-get install dovecot-managesieved dovecot-sieve

/etc/dovecot/dovecot.conf

plugin {
    sieve = ~/sieve/.dovecot.sieve
    sieve_dir = ~/sieve
    sieve_before = /var/lib/dovecot/sieve/sieve.default
}

protocol lda {
    mail_plugins = $mail_plugins sieve
}

protocols = imap pop3 sieve

Add these new lines e.g. on the bottom:

# Sieve configuration
protocol sieve {
}

service managesieve-login {
    inet_listener sieve {
        port = 4190
    }
}

service managesieve {
}

/etc/init.d/dovecot restart

EDIT: If some passages already exists, only ADD the new parameters.

[steve_adams]

Yes, both dovecot-sieve and dovecot-managesieved installed. Config suggested is the current configuration. Could the issue be that I'm trying to use roundcube's managesive plugin rather than sieverules?

[steve_adams]

I may have located the problem...
Jan 13 12:17:23 managesieve(steve.adams@enventive.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts.
Jan 13 12:17:23 managesieve(steve.adams@enventive.com): Debug: sieve-storage: using active sieve script path: /home/mail/enventive/enventive.com/steve.adams/Maildir//sieve/.dovecot.sieve
Jan 13 12:17:23 managesieve(steve.adams@enventive.com): Debug: sieve-storage: using sieve script storage directory: /home/mail/enventive/enventive.com/steve.adams/Maildir//sieve
Jan 13 12:17:23 managesieve(steve.adams@enventive.com): Debug: sieve-storage: using permissions from /home/mail/enventive/enventive.com/steve.adams/Maildir//sieve: mode=0700 gid=-1
Jan 13 12:17:23 managesieve(steve.adams@enventive.com): Debug: sieve-storage: relative path to sieve storage in active link:
Jan 13 12:17:23 auth: Debug: auth client connected (pid=6628)

 

notice the extra "/" in the active script path?

 

suggestions?

[junkpad92]

No, this isn't the problem - there must be an error or warning somewhere.

 

Please nopaste your COMPELTE config files (Roundcube managesieve plugin, Dovecot) for last time. I will check your configs.

 

Btw: Interesting, that you do not have a .dovecot.sieve inside your folder ... are you really sure that you installed dovecot-sieve?! Do you have restarted dovecot after installing dovecot-sieve? Please paste "ls -la" of your dovecot config folder.

[steve_adams]

attached...amnd thanks for your dedication

dovecot_conf.tar

managesieve_plugin_config.inc.php

[steve_adams]

To add, yes, I restarted dovecot...at least a dozen times today.

 

tha packages are installed:
root@envent:/etc/roundcube/plugins/managesieve# dpkg -l | grep -i dovecot-sieve
ii  dovecot-sieve                    1:2.0.19-0ubuntu2.2                 sieve filters support for Dovecot
root@envent:/etc/roundcube/plugins/managesieve# dpkg -l | grep -i dovecot-managesieved
ii  dovecot-managesieved             1:2.0.19-0ubuntu2.2                 secure ManageSieve server for Dovecot

 

root@envent:/etc/dovecot/conf.d# ls -a
.                            10-auth.conf      10-mail.conf    15-lda.conf   20-managesieve.conf  90-plugin.conf  auth-deny.conf.ext        auth-sql.conf.ext     auth-vpopmail.conf.ext
..                           10-director.conf  10-master.conf  20-imap.conf  20-pop3.conf         90-quota.conf   auth-master.conf.ext      auth-static.conf.ext
01-mail-stack-delivery.conf  10-logging.conf   10-ssl.conf     20-lmtp.conf  90-acl.conf          90-sieve.conf   auth-passwdfile.conf.ext  auth-system.conf.ext
root@envent:/etc/dovecot/conf.d# updatedb
root@envent:/etc/dovecot/conf.d# locate .dovecot.sieve
root@envent:/etc/dovecot/conf.d#

[steve_adams]

I enabled lmtp as a protocol and received the following errors in the dovecot.log:

 

Jan 13 18:19:10 lmtp(29366, steve@enventive.com): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts.
Jan 13 18:19:10 lmtp(29366, steve@enventive.com): Debug: e+dAEAzEtVS2cgAA4fOnYw: sieve: user's script path /home/mail/enventive/enventive.com/steve/Maildir//sieve/.dovecot.sieve doesn't exist (using global script path in stead)
Jan 13 18:19:10 lmtp(29366, steve@enventive.com): Debug: e+dAEAzEtVS2cgAA4fOnYw: sieve: user has no valid personal script
Jan 13 18:19:10 lmtp(29366, steve@enventive.com): Debug: e+dAEAzEtVS2cgAA4fOnYw: sieve: no scripts to execute: reverting to default delivery.

 

This is making me wonder if something in the authentication or the virtual users is amiss

[steve_adams]

Ok, I'm hitting a wall. I completely reformatted the postfix and dovecot according to my server's setting configurations in froxlor and cleaned out tons of bad entries in my config files and followed your instructions. I'm running Ubuntu 12.04, so there is no centrailized /etc/dovecot/dovecot.conf. The dovecot config is broken down into incuded configuration files: 01-mail-stack-delivery.conf, 10-auth.conf, 20-managesieve.conf, 90-sieve.conf, and 15-lda.conf have been altered. I've attached all these files for refernce and the updated doveconf. no matter what I change, the sieve scripts created in roundcube are not activated and I get no errors in the logs. I've even tried making a global sieve script and it does not get activated either...! WTF?

 

 

15-lda.conf.rtf

90-sieve.conf.rtf

20-managesieve.conf.rtf

10-auth.conf.rtf

01-mail-stack-delivery.conf.rtf

doveconf_latest.rtf

[junkpad92]

I have checked your files and have adjusted them. Please remove the 01-mail-stack-delivery.conf and replace the files you found in the folder with your local configs files. ATTENTION: Do not forget to backup the config files!

 

And please check also your "dovecot-dict-sql.conf.ext" - I attached it please replace your Password there.

dovecot.zip

[steve_adams]

There is no attached "dovecot-dict-sql.conf.ext" file? I assume you mean the "dovecot-sql.conf.ext.conf" file?

[junkpad92]

Oh, yes - sorry. I mean "dovecot-sql.conf.ext" (extension ext) of course.

[steve_adams]

Ouch,

 When I remove the 01-mail-stack-delivery.conf and restart dovecot all my users with POP accounts get duplicate emails of everything still in their inboxes on the server. Testing with a send and a receipt indicated the config is brocken. Then restoring the file and restarting dovecot again duplicates every email in the users inbox. 7000 emails later...still non functional filters. Ugh...

 

Has anyone had any luck with other siverules or vacation plugins for roundcube?

[d00p]

This works for me:

require ["fileinto","regex","vacation"];

# rule:[myVacation]
if allof (not header :contains "x-spam-flag" "YES", header :contains "to" "me@domain.tld")
{ 
    vacation :days 2 :subject "Out of the office" text: 
Hello,

i'm out - c ya in hell
.
;
}

[bsc]

Hey steve,

 

want to jump in here because I've looked into the same issue and stumbled across this thread.

I've also anticipated the problem in the variables sieve and sieve_dir and now I know that this was the hint I've needed.

 

For debugging purposes, I've set mail_debug to yes in the /etc/dovecot/dovecot.conf.

mail_debug = yes

I also added logfile output for the lda service (should be pretty much the same for lmtp) in /etc/dovecot/conf.d/15-lda.conf:

protocol lda {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins quota sieve

  # Log files
  log_path = /var/log/dovecot-lda-errors.log
  info_log_path = /var/log/dovecot-lda.log
} 

(This might not be necessary, as the ordinary output could go to the syslog-facility.)

 

After sending a test email to myself, I've noticed that that sieve looks for the sieve script in the wrong path:

Jan 15 11:38:58 lda(xxx@example.com): Debug: sieve: script file /home/xxx/sieve/.roundcube.sieve not found
Jan 15 11:38:58 lda(xxx@example.com): Debug: sieve: user's script /home/xxx/sieve/.roundcube.sieve doesn't exist (using default script location instead)
Jan 15 11:38:58 lda(xxx@example.com): Debug: sieve: no default script configured for user
Jan 15 11:38:58 lda(xxx@example.com): Debug: sieve: user has no valid location for a personal script
Jan 15 11:38:58 lda(xxx@example.com): Debug: sieve: no scripts to execute: reverting to default delivery.

That lead me directly to /etc/dovecot/conf.d/90-sieve.conf, where I changed the paths to my file structure:

  sieve = /var/customers/mail/%n/%d/%n/Maildir/.dovecot.sieve

  sieve_dir = /var/customers/mail/%n/%d/%n/Maildir/sieve/

(with %n is the username and %d the domain name, which translates to /var/customers/mail/xxx/example.com/xxx/Maildir/.dovecot.sieve)

 

After changing these values and restarting the dovecot service, sieve worked like a charm.

 

 

Cheers

  bsc

[junkpad92]

Did you try ~/sieve/? Your config needs extra directories for the sieve scripts. If you use ~/sieve the sieve folder is in the home directory of the mail user for example: /var/customers/mail/WEB/DOMAIN/USER/Maildir/sieve/

 

The problem: With your custom paths you are not able to define the WEB (web1, ...). So the scripts stored in another directory - in your case /var/customers/mail/name/domain.de/name/Maildir/ ...

[junkpad92]

Welcome!

 

With this config I had a problem. Could you check the logs if dovecot throws errors/warnings/etc.?

 

I solved the problem by putting the .dovecot.sieve in the folder /sieve too.

 

sieve = ~/sieve/.dovecot.sieve

sieve_dir = ~/sieve

 

EDIT: I found that interesting thing

"Note: It is not wise to place this link [note: sieve = ~/.dovecot.sieve] inside your mail store, as it may be mistaken for a mail folder. Inside a maildir for instance, the default .dovecot.sieve would show up as phantom folder /dovecot/sieve in your IMAP tree."

 

http://wiki2.dovecot.org/Pigeonhole/ManageSieve/Configuration

[bsc]

Sure thing I fell in this one, too:

Jan 15 20:10:08 hostname dovecot: imap(xxx@example.com): Error: stat(/var/customers/mail/xxx/example.com/xxx/Maildir/.dovecot.sieve/tmp) failed: Not a directory

Roundcube created a symbolic link within the Maildir structure:

# ls -lha | grep .dovecot.sieve
lrwxrwxrwx  1 vmail vmail   21 Jan 15 10:07 .dovecot.sieve -> sieve/roundcube.sieve

That's why 

sieve = ~/.dovecot.sieve 

works for me. (The variable 'sieve' points to the symbolic link which points to the 'roundcube.sieve' in the subdirectory 'sieve'.)

I've also read that this is not the way to configure this, but didn't find the time to look into this. Any hints welcome.

 

I assume that this is not very complicated to fix. However, I'm looking for a solution which is maintainable at all. As I'm currently using the 'managesieve' plugin for Roundcube, there should be a possibility to create the 'roundcube.sieve' file outside of the Maildir structure. 

[junkpad92]

Please change

sieve = ~/.dovecot.sieve
sieve_dir = ~/sieve

to

sieve = ~/sieve/.dovecot.sieve
sieve_dir = ~/sieve

and report, if the error is gone.

 

EDIT: I am also running Dovecot with the Roundcube plugin managesieve and with above settings. I am able to create, delete, change filters. The Roundcube scripts stored at /var/customers/mail/WEB/domain.de/name/Maildir/sieve/. In this directory there is also the .dovecot.sieve:

drwx------  3 vmail vmail 4096 Dec  5 12:32 .
drwx------ 11 vmail vmail 4096 Jan 13 21:44 ..
lrwxrwxrwx  1 vmail vmail   14 Dec  5 02:57 .dovecot.sieve -> Standard.sieve
-rw-------  1 vmail vmail  226 Dec  5 12:32 .dovecot.svbin
-rw-------  1 vmail vmail  167 Dec  5 02:18 Standard.sieve
drwx------  2 vmail vmail 4096 Dec 12 17:04 tmp

[bsc]

As far as I can see that won't fix the issue that the 'sieve' folder is within the Maildir structure and therefore will be displayed as a regular IMAP folder to the user, won't it?

[junkpad92]

No, IMAP folders have a dot (.) as prefix. The sieve folder has no dot.

[bsc]

You're perfectly right, sorry.

 

Here's what I did.

I moved both the .dovecot.sieve and .dovecot.svbin files to the subdirectory ./sieve/

root@hostname:/var/customers/mail/xxx/example.com/xxx/Maildir# mv .dovecot.s* ./sieve/

Afterwards I changed to the sieve subdirectory and deleted the symlink and created a new one:

root@hostname:/var/customers/mail/xxx/example.com/xxx/Maildir# cd sieve
root@hostname:/var/customers/mail/xxx/example.com/xxx/Maildir/sieve# rm -fr .dovecot.sieve
root@hostname:/var/customers/mail/xxx/example.com/xxx/Maildir/sieve# sudo -u vmail ln -s roundcube.sieve .dovecot.sieve  

(The syntax for creating a new symlink is 'ln -s $target $name_of_the_symlink'. The 'sudo -u vmail' part in front of that command executes this command as user 'vmail' which should be the owner of the symlink.)

 

 

Afterwards it should look like:

root@hostname:/var/customers/mail/xxx/example.com/xxx/Maildir/sieve# ls -lha
total 20K
drwx------  3 vmail vmail 4.0K Jan 15 23:09 .
drwx------ 16 vmail vmail 4.0K Jan 15 23:08 ..
lrwxrwxrwx  1 vmail vmail   15 Jan 15 23:09 .dovecot.sieve -> roundcube.sieve
-rw-------  1 vmail vmail  186 Jan 15 14:35 .dovecot.svbin
-rw-------  1 vmail vmail   98 Jan 15 14:16 roundcube.sieve
drwx------  2 vmail vmail 4.0K Jan 15 14:16 tmp

As per junkpad92's proposal, I've changed the sieve parameter in the file /etc/dovecot/conf.d/90-sieve.conf to 

 sieve = ~/sieve/.dovecot.sieve

and restarted dovecot:

# service dovecot restart
[ ok ] Restarting IMAP/POP3 mail server: dovecot.

I'll keep an eye on it, but it seems to work fine and the folder's gone out of the user's view. Great work, young padawan!

 

junkpad92, what if my users create a new ruleset? Roundcube therefore creates a new *.sieve file. How is this filterset going to be addressed in the sieve procedure?

[junkpad92]

Thanks for compliment!

 

It is only possible to activate ONE ruleset in Roundcube. So if your customer creates two rulesets, he must decide which ruleset should be the active one. But you can create rules as many as you want in one ruleset.

 

If you change the ruleset from ruleset1 to ruleset2, also the symlink of .dovecot.sieve will be changed to ruleset2:

lrwxrwxrwx  1 vmail vmail   15 Jan 16 00:05 .dovecot.sieve -> ruleset2.sieve
-rw-------  1 vmail vmail  124 Dec 17 19:11 .dovecot.svbin
-rw-------  1 vmail vmail   18 Jan 16 00:04 ruleset1.sieve
-rw-------  1 vmail vmail   18 Dec 17 17:47 ruleset2.sieve

Hope you unterstand my poor English.

[bsc]

Yes, that's pretty much what I expected it to be.

Thanks for making this clear. (And, your English is pretty fine for me.)

[junkpad92]

You're welcome.