View Issue Details

IDProjectCategoryView StatusLast Update
0001392SOGoBackend Mailpublic2019-11-11 09:12
Reportermra Assigned Toludovic  
PrioritynormalSeverityfeatureReproducibilityN/A
Status resolvedResolutionfixed 
Product Versionnightly v2 
Fixed in Version4.2.0 
Summary0001392: Patch: Inclusion/Embedding of server-side sieve scripts into standard sogo-script
Description

In our company we migrated to SOGo with Dovecot and Postfix for IMAP and SMTP.
Several users used "sophisticated" sieve scripts on the old Groupware Server which they want to use again on SOGo in conjunction with the easy use of the SOGo created sieve scripts (vacation, redirects).
So the best solution seems to be merging the several scripts to one 'big' sogo.sieve, where the merge will be done by SOGo.

REMARK: This feature is IMHO only usable for experienced Users! If the embedded (userwritten) Script has an error, the sogo.sieve-script will fail, too!

Additional Information

Configuration

SOGoSievePrependFilterName - name of user sieve script, which will run before sogo.sieve

SOGoSieveAppendFilterName - name of user sieve script, which will run at the end of sogo.sieve

SOGoSievePrependGlobalFilterName - Dovecot special: name of configured global sieve script, which will run before the user sieve script in sogo.sieve (should be unofficial, maybe removed)

SOGoSieveAppendGlobalFilterName - Dovecot special: name of configured global sieve script, which will run after the use sieve script in sogo.sieve (should be unofficial, maybe removed)

SOGoSieveInsertServerFilter - Default is NO, if YES than all the sieve scripts which current stored in the users sieve directory will be used instead of the configured 'SOGoSieve*FilterName' scripts. The convention is

  • sieve scripts with prefix 'pre_' will be included at the beginning of the standard sogo.sieve
  • sieve scripts with prefix 'post_' will be included at the End of the standard sogo.sieve

Example:

  • pre_01.sieve and pre_02.sieve will be included at the beginning of the sogo.sieve
  • post_01.sieve will be included at the end of the sogo.sieve

SOGoSieveDebugEnabled - enables some debug messages

Dependencies

This patch depends on 0001391

Function

The external scripts inclusion into sogo.sieve will be done in two possible ways

  • with sieve include-plugin or, if 'include' is not available,
  • embedding of the content of this scripts in sogo.sieve

The partly used sieve plugin 'include' is a RFC Proprosal for sieve and is current implemented only in dovecot,
so we check the managesieve server for its capabilities before creating the sogo.sieve script.

REMARK

The plugin uses only the 'include' command with the locations ":personal" and (if configured) ':global'.
In this implementation a global filter is the first (pre) and the last (post) filter.
The personal filter will run directly after the first global, the last personal filter will
be included before the last global filter.

TagsNo tags attached.

Relationships

parent of 0001391 closed Patch for retrieving managesieve capabilities 
related to 0003209 new Skip ManageSieve sync if user has not modified filter Preferences 

Activities

2011-07-20 18:09

 

patch_SOGo_sieve_external_scripts.diff (16,363 bytes)   
#
# old_revision [6573eb6d5779706b5935161822c561cf325f7fa6]
#
# patch "SoObjects/SOGo/SOGoDomainDefaults.h"
#  from [0b2be24987ff61077a11ffe0bf02116625d782d1]
#    to [88b910b96713c21a9d0710066a0f569e8e3eefa7]
# 
# patch "SoObjects/SOGo/SOGoDomainDefaults.m"
#  from [fa3475a65ccea244b1223c960aa0dbd8dd990be4]
#    to [f380c5be1577fe21a0e2c27d1944b3e124623e9c]
# 
# patch "SoObjects/SOGo/SOGoSieveManager.h"
#  from [d3a6cf989982a7dc20ef92ffc786f2e278742951]
#    to [8bc04ad0515f258519b08969699eb3374e24ae2c]
# 
# patch "SoObjects/SOGo/SOGoSieveManager.m"
#  from [3d8e3ce324744a5bb8ebfe9d8fdb738a26bd6b92]
#    to [03246493ff500bf6224324f1820ca235a24972fc]
# 
# patch "UI/PreferencesUI/UIxPreferences.m"
#  from [118f6316688e83962d53584288dd47e498270881]
#    to [495102cb29027e04b934358bd6c3ffbd0cf4f0c2]
#
============================================================
--- SoObjects/SOGo/SOGoDomainDefaults.h	0b2be24987ff61077a11ffe0bf02116625d782d1
+++ SoObjects/SOGo/SOGoDomainDefaults.h	88b910b96713c21a9d0710066a0f569e8e3eefa7
@@ -47,7 +47,17 @@
 - (NSString *) imapFolderSeparator;
 - (BOOL) imapAclConformsToIMAPExt;
 - (BOOL) forceIMAPLoginWithEmail;
+
+
+- (NSString *) sievePrependFilterName;
+- (NSString *) sieveAppendFilterName;
+- (NSString *) sievePrependGlobalFilterName;
+- (NSString *) sieveAppendGlobalFilterName;
+- (BOOL) sieveInsertServerFilter;
 - (BOOL) sieveScriptsEnabled;
+- (BOOL) sieveDebugEnabled;
+
+
 - (BOOL) forwardEnabled;
 - (BOOL) vacationEnabled;
 - (NSString *) mailingMechanism;
============================================================
--- SoObjects/SOGo/SOGoDomainDefaults.m	fa3475a65ccea244b1223c960aa0dbd8dd990be4
+++ SoObjects/SOGo/SOGoDomainDefaults.m	f380c5be1577fe21a0e2c27d1944b3e124623e9c
@@ -176,6 +176,39 @@
   return [self boolForKey: @"SOGoSieveScriptsEnabled"];
 }
 
+
+
+- (NSString *) sievePrependFilterName
+{
+  return [self stringForKey: @"SOGoSievePrependFilterName"];
+}
+
+- (NSString *) sieveAppendFilterName
+{
+  return [self stringForKey: @"SOGoSieveAppendFilterName"];
+}
+
+- (NSString *) sievePrependGlobalFilterName
+{
+  return [self stringForKey: @"SOGoSievePrependGlobalFilterName"];
+}
+
+- (NSString *) sieveAppendGlobalFilterName
+{
+  return [self stringForKey: @"SOGoSieveAppendGlobalFilterName"];
+}
+
+- (BOOL) sieveInsertServerFilter
+{
+  return [self boolForKey: @"SOGoSieveInsertServerFilter"];
+}
+
+- (BOOL) sieveDebugEnabled
+{
+  return [self boolForKey: @"SOGoSieveDebugEnabled"];
+}
+
+
 - (BOOL) forwardEnabled
 {
   return [self boolForKey: @"SOGoForwardEnabled"];
@@ -186,6 +219,10 @@
   return [self boolForKey: @"SOGoVacationEnabled"];
 }
 
+
+
+
+
 - (NSString *) mailingMechanism
 {
   NSString *mailingMechanism;
============================================================
--- SoObjects/SOGo/SOGoSieveManager.h	d3a6cf989982a7dc20ef92ffc786f2e278742951
+++ SoObjects/SOGo/SOGoSieveManager.h	8bc04ad0515f258519b08969699eb3374e24ae2c
@@ -31,6 +31,7 @@
 @class NSString;
 @class SOGoMailAccount;
 @class SOGoUser;
+@class NGSieveClient;
 
 @interface SOGoSieveManager : NSObject
 {
@@ -45,11 +46,24 @@
 - (NSString *) sieveScriptWithRequirements: (NSMutableArray *) newRequirements;
 - (NSString *) lastScriptError;
 
+
 - (BOOL) updateFiltersForLogin: (NSString *) theLogin
 		      authname: (NSString *) theAuthName
 		      password: (NSString *) thePassword
 		       account: (SOGoMailAccount *) theAccount;
 
+- (NSString *) makeScriptEntryWithScript:(NSString*)scriptName 
+				withContent:(BOOL)retrieveContent  
+				  andClient:(NGSieveClient *)client;
+					
+- (NSString *) makeScriptEntryWithScript:(NSString*)scriptName 
+				withContent:(BOOL)retrieveContent 
+				  andClient:(NGSieveClient *)client 
+				   asGlobal:(BOOL) isGlobal;			  	  
+				  	  
+- (NSDictionary *) loadScriptList: (NGSieveClient *) sieveClient;
+
+
 @end
 
 #endif /* SOGOSIEVEMANAGER_H */
============================================================
--- SoObjects/SOGo/SOGoSieveManager.m	3d8e3ce324744a5bb8ebfe9d8fdb738a26bd6b92
+++ SoObjects/SOGo/SOGoSieveManager.m	03246493ff500bf6224324f1820ca235a24972fc
@@ -26,6 +26,7 @@
 #import <Foundation/NSString.h>
 #import <Foundation/NSURL.h>
 #import <Foundation/NSValue.h>
+#import <NGExtensions/NSString+Ext.h>
 
 #import <SOGo/NSArray+Utilities.h>
 #import <SOGo/NSDictionary+Utilities.h>
@@ -618,10 +619,10 @@ static NSString *sieveScriptName = @"sog
 - (BOOL) updateFiltersForLogin: (NSString *) theLogin
 		      authname: (NSString *) theAuthName
 		      password: (NSString *) thePassword
-		       account: (SOGoMailAccount *) theAccount
+		      account: (SOGoMailAccount *) theAccount
 {
   NSMutableArray *req;
-  NSMutableString *script, *header;
+  NSMutableString *script, *header, *appendScriptStr, *prependScriptStr;
   NGInternetSocketAddress *address;
   NSDictionary *result, *values;
   SOGoUserDefaults *ud;
@@ -640,6 +641,8 @@ static NSString *sieveScriptName = @"sog
   b = NO;
 
   script = [NSMutableString string];
+  appendScriptStr = [NSMutableString string]; // needed for include
+  prependScriptStr = [NSMutableString string]; // needed for include
 
   // Right now, we handle Sieve filters here and only for vacation
   // and forwards. Traditional filters support (for fileinto, for
@@ -713,6 +716,9 @@ static NSString *sieveScriptName = @"sog
       if ([[values objectForKey: @"keepCopy"] boolValue])
 	[script appendString: @"keep;\r\n"];
     }
+    
+    
+
   
   filterScript = [self sieveScriptWithRequirements: req];
   if (filterScript)
@@ -729,13 +735,8 @@ static NSString *sieveScriptName = @"sog
       return NO;
     }
 
-  if ([req count])
-    {
-      header = [NSString stringWithFormat: @"require [\"%@\"];\r\n",
-                         [req componentsJoinedByString: @"\",\""]];
-      [script insertString: header  atIndex: 0];
-    }
 
+    
   // We connect to our Sieve server and upload the script.
   //
   // sieveServer might have the following format:
@@ -793,6 +794,9 @@ static NSString *sieveScriptName = @"sog
     [client closeConnection];
     return NO;
   }
+
+  
+  
   result = [client login: theLogin  authname: theAuthName  password: thePassword];
   if (![[result valueForKey:@"result"] boolValue]) {
     NSLog(@"failure. Attempting with a renewed password (no authname supported)");
@@ -807,14 +811,164 @@ static NSString *sieveScriptName = @"sog
     return NO;
   }
 
+  
+  // Handle external scripts for inclusion in sogo.sieve
+  BOOL useExternalScripts = [dd sieveInsertServerFilter]
+  			|| [dd sievePrependFilterName]
+			|| [dd sieveAppendFilterName]
+			|| [dd sievePrependGlobalFilterName]
+			|| [dd sieveAppendGlobalFilterName];
+
+			
+  if(useExternalScripts)
+  {
+	NSMutableDictionary *serverscripts = [NSMutableDictionary dictionary];
+
+	NSMutableArray *_preFiles = [NSMutableArray array];
+	NSMutableArray *_postFiles = [NSMutableArray array];
+	NSMutableArray *_preGlobalFiles = [NSMutableArray array];
+	NSMutableArray *_postGlobalFiles = [NSMutableArray array];		
+
+	// Check for an available 'include' sieve plugin.
+	// When the plugin is not available,
+	// - the script/s will be included directly as source (read from server, lasts some microseconds longer ;-) )
+	// - global scripts not possible (they are only possible, too, when configured in dovecot)
+	//
+  	BOOL withScriptContent = ![client hasCapability: @"include"];
+ 
+  	if(withScriptContent)
+  	{
+  		NSLog(@"WARNING! Configured managesieve server does not support the 'include' plugin. Including scripts directly.");
+  	}
+  	else
+	{		
+		// Usefull if available - so you do not need to include all the stuff in one file
+		[req addObjectUniquely: @"include"];	  
+	}	
+			
+	if([dd sieveInsertServerFilter]) // Insert all current filters on and from the Server, but without sogo.sieve
+	{
+		// Read all current scripts from server, but they must be sorted by a prefix 'pre_' and/or 'post_'
+		// Sorting with number after the prefix will help ...
+		serverscripts = [self loadScriptList: client];
+	}
+	else  // Std case: pre-configured scriptnames for pre-run and post-run
+	{
+		// prepend an external script, name is configured in sievePrependFilterName
+		if([dd sievePrependFilterName])
+		{
+			[_preFiles addObject: [dd sievePrependFilterName]];
+		}		
+		
+		if([dd sieveAppendFilterName])
+		{
+			[_postFiles addObject: [dd sieveAppendFilterName]];
+		}
+		
+		// prepend an global external script, name is configured in sievePrependGlobalFilterName
+		// This runs only when the 'include' sieve-plugin is available
+		if(!withScriptContent)
+		{
+			if([dd sievePrependGlobalFilterName])
+			{
+				[_preGlobalFiles addObject: [dd sievePrependGlobalFilterName]];
+			}	
+	
+			
+			if([dd sieveAppendGlobalFilterName])
+			{
+				[_postGlobalFiles addObject: [dd sieveAppendGlobalFilterName]];
+			}
+		}		
+		
+		[serverscripts setObject: _preFiles forKey: @"prefiles" ];
+		[serverscripts setObject: _preGlobalFiles forKey: @"globalPrefiles" ];		
+		
+		[serverscripts setObject: _postFiles forKey: @"postfiles" ];
+		[serverscripts setObject: _postGlobalFiles forKey: @"globalPostfiles" ];			
+	}
+
+	
+	// Now creating stuff from 'serverscripts' and make scripts
+	if([dd sieveDebugEnabled]) NSLog(@"SIEVE: Now creating stuff from 'serverscripts' and make sieve scripts");
+	NSString *_scriptName;
+	
+	if([[serverscripts objectForKey: @"prefiles"] count] > 0)
+	{
+		NSEnumerator *preFiles = [[serverscripts objectForKey: @"prefiles"] objectEnumerator];
+		
+		//while(_scriptName = (NSString *)[preFiles nextObject])
+		while(_scriptName = [preFiles nextObject])
+		{	
+			[prependScriptStr appendString: [self makeScriptEntryWithScript: _scriptName 
+					withContent:withScriptContent  
+					andClient: client]];
+		}
+	}
+	
+	if([[serverscripts objectForKey: @"postfiles"] count] > 0)
+	{
+		NSEnumerator *postFiles = [[serverscripts objectForKey: @"postfiles"] objectEnumerator];
+		
+		// while(_scriptName = (NSString *)[postFiles nextObject])
+		while(_scriptName = [postFiles nextObject])
+		{
+			[appendScriptStr appendString: [self makeScriptEntryWithScript: _scriptName 
+					withContent:withScriptContent  
+					andClient: client]];
+		}
+	}
+	
+	// Global
+	if(!withScriptContent) // Global scripts could only be used with the sieve-plugin 'include' AND server configuration (known in dovecot)
+	{
+		[prependScriptStr appendString: [self makeScriptEntryWithScript: _scriptName 
+							withContent:NO  
+							andClient: client
+							asGlobal: YES]];
+				
+		[appendScriptStr appendString: [self makeScriptEntryWithScript: _scriptName 
+							withContent:NO  
+							andClient: client
+							asGlobal: YES]];
+	}				
+		
+	
+	// sievePrepend*FilterName MUST be the first entry after file header
+	if([prependScriptStr length])
+	{
+		[script insertString: prependScriptStr atIndex: 0];  	  
+	}    
+	
+	// sieveAppend*FilterName MUST be the last entry
+	if([appendScriptStr length])
+	{
+		[script appendString: appendScriptStr];
+	}
+  }
+  
+  
+  // Creating require-string
+  if ([req count])
+    {
+      header = [NSString stringWithFormat: @"require [\"%@\"];\r\n",
+                         [req componentsJoinedByString: @"\",\""]];
+      [script insertString: header  atIndex: 0];
+    }
+
+  if([dd sieveDebugEnabled]) NSLog(@"SIEVE: Created sieve script:\r\n----\r\n%@", script);
+    
+    
   /* We ensure to deactive the current active script since it could prevent
      its deletion from the server. */
   result = [client setActiveScript: @""];
+  
   // We delete the existing Sieve script
   result = [client deleteScript: sieveScriptName];
   
-  if (![[result valueForKey:@"result"] boolValue]) {
-    NSLog(@"WARNING: Could not delete Sieve script - continuing...: %@", result);
+  if (![[result valueForKey:@"result"] boolValue]) 
+  {
+  	  NSLog(@"WARNING: Could not delete Sieve script - continuing...: %@", result);
   }
 
   // We put and activate the script only if we actually have a script
@@ -836,8 +990,129 @@ static NSString *sieveScriptName = @"sog
 	return NO;
       }
   }
+  
+  return YES;
+}
 
-  return YES;
+
+- (NSString *) makeScriptEntryWithScript:(NSString*)scriptName 
+				withContent:(BOOL)retrieveContent  
+				andClient:(NGSieveClient *)sieveClient
+{
+	return [self makeScriptEntryWithScript: scriptName 
+			withContent:retrieveContent 
+			andClient:sieveClient 
+			asGlobal:NO];
 }
 
+- (NSString *) makeScriptEntryWithScript:(NSString*)scriptName 
+				withContent:(BOOL)retrieveContent 
+				andClient:(NGSieveClient *)sieveClient 
+				asGlobal:(BOOL) isGlobal
+{
+	// Configdata
+	SOGoDomainDefaults *dd = [user domainDefaults];
+	
+	// when a content is given (!= NIL), a "global" file is not "reachable"
+	if(retrieveContent)
+	{
+		isGlobal = NO;
+	}
+	
+	NSString *sieveFileName = [scriptName stringWithoutSuffix:@".sieve"];
+	
+	// Check, if configures local/":personal" configured in 'sieve*FilterName' are on place.
+	// If not, create it
+	if(isGlobal == NO && ![sieveClient getScript: sieveFileName])
+	{
+		if([dd sieveDebugEnabled]) NSLog(@"SIEVE: Script '%@' not found, now creating it.", scriptName);
+		[sieveClient putScript: sieveFileName script:[NSString stringWithFormat: @"# Dummy file '%@' created by SOGo", scriptName]];
+	}
+	
+	
+	// when global, another include-string has to be created
+	if(isGlobal)
+	{
+		if(!scriptName)
+		{
+			return @"";
+		}
+		
+		// create include global configured '*FilterName'
+		return [NSString stringWithFormat: @"\r\ninclude :global \"%@\";\r\n\r\n", scriptName];
+	}
+	else
+	{
+		if(retrieveContent)
+		{
+			// get content of script and return it
+			if(![sieveClient getScript: sieveFileName])  // Create file if not available
+			{
+				if([dd sieveDebugEnabled]) NSLog(@"SIEVE: Script '%@' not found, now creating it as dummy.", scriptName);
+				[sieveClient putScript: sieveFileName script:[NSString stringWithFormat: @"# This file '%@' was created by SOGo", scriptName]];
+			}				
+				
+
+			NSString *loadedScript = [sieveClient getScript: sieveFileName];
+			if(!loadedScript)
+			{
+				if([dd sieveDebugEnabled]) NSLog(@"SIEVE: WARNING: Cannot include script content, script '%@' not found.", scriptName);
+				return nil; // No file, no string.
+			}
+			else
+			{
+				return [NSString stringWithFormat: @"\r\n##### included file content follows #####\r\n%@\r\n##### END OF included file content #####\r\n", loadedScript];
+			}			
+		}
+		else
+		{
+			// create include for personal 'sieve*FilterName'
+			return [NSString stringWithFormat: @"\r\ninclude :personal \"%@\";\r\n\r\n", scriptName];	
+		}
+	}
+}
+
+
+- (NSDictionary *) loadScriptList: (NGSieveClient *)sieveClient
+{
+	NSMutableArray *_preFiles = [NSMutableArray array];
+	NSMutableArray *_postFiles = [NSMutableArray array];
+	NSString *activeFile = @"";
+	
+	// Get list of scripts
+	NSDictionary *scriptList = [sieveClient listScripts];
+
+	if([scriptList count] > 0) // objectForKey
+	{
+		NSArray *keys = [scriptList allKeys];
+		NSArray *values = [scriptList allValues];	
+		
+		int i;
+		for(i=0; i< [keys count]; i++)
+		{
+			if([[keys objectAtIndex: i] hasPrefix: @"pre_"])
+			{
+				[_preFiles addObject: [keys objectAtIndex: i]];
+			}
+			else if([[keys objectAtIndex: i] hasPrefix: @"post_"])
+			{
+				[_postFiles addObject: [keys objectAtIndex: i]];
+			}
+			
+			if([[[values objectAtIndex: i] lowercaseString] isEqualToString: @"active"])
+			{
+				activeFile = [keys objectAtIndex: i];
+			}	
+		}
+	}
+	
+	// Packing results into structure
+	NSMutableDictionary *dict = [NSMutableDictionary dictionaryWithCapacity:4];
+	[dict setObject: [NSArray arrayWithArray:_preFiles ] forKey: @"prefiles"]; 
+	[dict setObject: [NSArray arrayWithArray:_postFiles ] forKey: @"postfiles" ];
+	[dict setObject: activeFile forKey: @"activeFile"];
+	
+	return dict;
+}
+
 @end
============================================================
--- UI/PreferencesUI/UIxPreferences.m	118f6316688e83962d53584288dd47e498270881
+++ UI/PreferencesUI/UIxPreferences.m	495102cb29027e04b934358bd6c3ffbd0cf4f0c2
@@ -975,6 +975,7 @@
       folder = [[self clientObject] mailAccountsFolder: @"Mail"
                                              inContext: context];
       account = [folder lookupName: @"0" inContext: context acquire: NO];
+
       [account updateFilters];
 
       if (hasChanged)
mra

mra

2011-07-20 18:13

reporter   ~0002731

Possible script names in the configuration:

SOGoSievePrependFilterName -> pre.sieve
SOGoSieveAppendFilterName -> post.sieve
SOGoSievePrependGlobalFilterName -> global_pre.sieve
SOGoSieveAppendGlobalFilterName -> global_post.sieve

mra

mra

2011-07-20 18:18

reporter   ~0002732

Maybe this patch will show a solution for 0000809 - toy around with sieve under control of SOGo ;-)

mra

mra

2012-07-11 04:39

reporter   ~0004121

This patch could be seen as a child of 0001391, it depends on it.

ludovic

ludovic

2019-11-11 09:12

administrator   ~0013886

https://github.com/inverse-inc/sogo/commit/4475ac651d1d94513729d6133a70d0e70ea52b87

Issue History

Date Modified Username Field Change
2011-07-20 18:09 mra New Issue
2011-07-20 18:09 mra File Added: patch_SOGo_sieve_external_scripts.diff
2011-07-20 18:13 mra Note Added: 0002731
2011-07-20 18:18 mra Note Added: 0002732
2012-07-11 03:35 Christian Mack Relationship added parent of 0001391
2012-07-11 04:39 mra Note Added: 0004121
2015-07-13 10:55 Christian Mack Relationship added related to 0003209
2019-11-11 09:12 ludovic Note Added: 0013886
2019-11-11 09:12 ludovic Status new => resolved
2019-11-11 09:12 ludovic Fixed in Version => 4.2.0
2019-11-11 09:12 ludovic Resolution open => fixed
2019-11-11 09:12 ludovic Assigned To => ludovic