Dependency Graph

Dependency Graph
related to related to child of child of duplicate of duplicate of

View Issue Details

IDProjectCategoryView StatusLast Update
0002688SOGoActiveSyncpublic2014-05-27 18:47
Reporteronofabio Assigned Toludovic  
PrioritynormalSeverityfeatureReproducibilityalways
Status resolvedResolutionfixed 
Product Version2.2.2 
Target Version2.2.4Fixed in Version2.2.4 
Summary0002688: Missing folders list update
Description

New folders created on webmail or client (like outlook) are not downloaded on the Android Mail app.

Steps To Reproduce

Create new Exchange active sync account on Android device.

Create folder on webmail (or outlook).

Additional Information

If folder was created on Android app, on server I find new folder and I can use it on webmail.

TagsNo tags attached.

Relationships

related to 0001723 resolvedludovic mail: outlook re-creates deleted IMAP folder 
related to 0001739 resolvedludovic cal: Deleting a calendar from SOGo doesn't sync to OL 
related to 0001740 resolvedludovic cal: Renaming a calendar in SOGo doesn't sync to OL 

Activities

ludovic

ludovic

2014-03-27 17:11

administrator   ~0006795

This is normal for now, that feature hasn't been implemented.

onofabio

onofabio

2014-03-28 08:30

reporter   ~0006797

OK thanks.

Same problem in outlook (folder created in webmail isn't downloaded on client). Is this normal?

ludovic

ludovic

2014-03-28 18:22

administrator   ~0006801

Yes.

tfu

tfu

2014-04-13 18:41

reporter  

0001-folder-sync.patch (22,738 bytes)   
From 0cb65ec8fe740b4a9518937f1ff2792f6208ed2b Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Sun, 13 Apr 2014 20:37:14 +0200
Subject: [PATCH] folder sync

---
 ActiveSync/NSString+ActiveSync.m      |    6 +
 ActiveSync/SOGoActiveSyncDispatcher.m |  245 +++++++++++++++++++++++++++++----
 SoObjects/Mailer/SOGoMailAccount.m    |   30 +++-
 SoObjects/Mailer/SOGoMailFolder.h     |    3 +
 SoObjects/Mailer/SOGoMailFolder.m     |   35 +++++
 5 files changed, 291 insertions(+), 28 deletions(-)

diff --git a/ActiveSync/NSString+ActiveSync.m b/ActiveSync/NSString+ActiveSync.m
index b1fba0f..0d4140f 100644
--- a/ActiveSync/NSString+ActiveSync.m
+++ b/ActiveSync/NSString+ActiveSync.m
@@ -107,6 +107,12 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     {
       realCollectionId = [[v stringByUnescapingURL] substringFromIndex: 5];
       *folderType = ActiveSyncMailFolder;
+      
+      // tfu strip uidvalidity from mail-serverId (path-uidvalidity)
+      NSRange r1;
+      r1 = [realCollectionId rangeOfString: @"-"];
+      realCollectionId = [realCollectionId  substringToIndex: r1.location];
+
     }
   else
     {
diff --git a/ActiveSync/SOGoActiveSyncDispatcher.m b/ActiveSync/SOGoActiveSyncDispatcher.m
index 7d99ceb..e2b3941 100644
--- a/ActiveSync/SOGoActiveSyncDispatcher.m
+++ b/ActiveSync/SOGoActiveSyncDispatcher.m
@@ -133,6 +133,49 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   [[[context activeUser] userSettings] synchronize];
 }
 
+- (void) _setFolder: (NSString *) theUIDValidity
+                     foldername: (NSString *) theFolderName
+{
+  NSMutableDictionary *metadata;
+  
+  metadata = [[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]];
+  
+  [metadata setObject: [NSDictionary dictionaryWithObject: theFolderName  forKey: @"FolderName"]  forKey: theUIDValidity];
+
+  [[[context activeUser] userSettings] setMicrosoftActiveSyncMetadata: metadata
+                                                               forDevice: [context objectForKey: @"DeviceId"]];
+
+  [[[context activeUser] userSettings] synchronize];
+}
+
+- (void) _setDeleteFolder: (NSString *) theUIDValidity
+{
+  NSMutableDictionary *metadata;
+  
+  metadata = [[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]];
+  
+  [metadata removeObjectForKey: theUIDValidity];
+
+  [[[context activeUser] userSettings] setMicrosoftActiveSyncMetadata: metadata
+                                                               forDevice: [context objectForKey: @"DeviceId"]];
+
+  [[[context activeUser] userSettings] synchronize];
+}
+
+- (void) _clear
+{
+  NSMutableDictionary *metadata;
+  
+  metadata = [[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]];
+  
+  [metadata removeAllObjects];
+
+  [[[context activeUser] userSettings] setMicrosoftActiveSyncMetadata: metadata
+                                                               forDevice: [context objectForKey: @"DeviceId"]];
+
+  [[[context activeUser] userSettings] synchronize];
+}
+
 //
 //
 //
@@ -190,6 +233,14 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   int type;
 
   parentId = [[(id)[theDocumentElement getElementsByTagName: @"ParentId"] lastObject] textValue];
+ 
+  // strip uidvalidity from parentId (path-uidvalidity)
+  if (![parentId isEqualToString: @"0"]) {
+         NSRange r1;
+         r1 = [parentId rangeOfString: @"-"];
+         parentId = [parentId  substringToIndex: r1.location];
+  }
+
   displayName = [[(id)[theDocumentElement getElementsByTagName: @"DisplayName"] lastObject] textValue];
   type = [[[(id)[theDocumentElement getElementsByTagName: @"Type"] lastObject] textValue] intValue];
   userFolder = [[context activeUser] homeFolderInContext: context];
@@ -213,10 +264,21 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
         accountsFolder = [userFolder lookupName: @"Mail"  inContext: context  acquire: NO];
         currentFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
         
+
+        // tfu if the parrent is 0 -> ok ; otherwise need to build the foldername based on parentId + displayName
+        if ([parentId isEqualToString: @"0"])
         newFolder = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@", [displayName stringByEncodingImap4FolderName]]
                                     inContext: context
                                       acquire: NO];
-        
+
+        else
+		// uidvalidity - for parrent in subfolder  !!!
+        newFolder = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@/%@", [[parentId stringByUnescapingURL] substringFromIndex:5] , [displayName stringByEncodingImap4FolderName]]
+                                    inContext: context
+                                      acquire: NO];
+
+         // FIXME
+
         // FIXME
         // handle exists (status == 2)
         // handle right synckey
@@ -226,12 +288,29 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
             
             // We strip the "folder" prefix
             nameInContainer = [nameInContainer substringFromIndex: 6];
-            nameInContainer = [[NSString stringWithFormat: @"mail/%@", nameInContainer] stringByEscapingURL];
+
+            nameInContainer = [[NSString stringWithFormat: @"mail/%@-%@",  nameInContainer ,[newFolder getUIDValidity]] stringByEscapingURL];
+            // _setFolder to avoid <Add> during foldersync
+            [self _setFolder:    [nameInContainer stringByUnescapingURL]  foldername: [NSString stringWithFormat: @"/%@", [[newFolder nameInContainer]  substringFromIndex: 6]]];
+
           }
         else
           {
-            [theResponse setStatus: 500];
-            [theResponse appendContentString: @"Unable to create folder."];
+  //          [theResponse setStatus: 500];
+  //          [theResponse appendContentString: @"Unable to create folder."];
+
+            // trigger a folderresync if create fails - bad thing is that user doesn't get warned about the failure
+            s = [NSMutableString string];
+            [s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
+            [s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\">"];
+            [s appendString: @"<FolderCreate xmlns=\"FolderHierarchy:\">"];
+            [s appendFormat: @"<Status>%d</Status>", 5]; 
+            [s appendString: @"</FolderCreate>"];
+
+            d = [[s dataUsingEncoding: NSUTF8StringEncoding] xml2wbxml];
+  
+            [theResponse setContent: d];
+
             return;
           }
       }
@@ -347,8 +426,22 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     }
   else
     {
-      [theResponse setStatus: 500];
-      [theResponse appendContentString: @"Unable to delete folder."];
+     // [theResponse setStatus: 500];
+     // [theResponse appendContentString: @"Unable to delete folder."];
+
+      // trigger a folderresync if delete fails - bad thing is that user doesn't get warned about the failure
+      NSMutableString *s;
+      NSData *d;
+      s = [NSMutableString string];
+      [s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
+      [s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\">"];
+      [s appendString: @"<FolderDelete xmlns=\"FolderHierarchy:\">"];
+      [s appendFormat: @"<Status>%d</Status>", 4];
+      [s appendString: @"</FolderDelete>"];
+      d = [[s dataUsingEncoding: NSUTF8StringEncoding] xml2wbxml];
+      [theResponse setContent: d];
+      
+
     }
 }
 
@@ -369,7 +462,15 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   int status;
   
   serverId = [[[(id)[theDocumentElement getElementsByTagName: @"ServerId"] lastObject] textValue] realCollectionIdWithFolderType: &folderType];
+
   parentId = [[(id)[theDocumentElement getElementsByTagName: @"ParentId"] lastObject] textValue];
+
+  if (![parentId isEqualToString: @"0"]) {
+       NSRange r1;
+  r1 = [parentId rangeOfString: @"-"];
+  parentId = [parentId  substringToIndex: r1.location];
+  }
+
   displayName = [[(id)[theDocumentElement getElementsByTagName: @"DisplayName"] lastObject] textValue];
 
   userFolder = [[context activeUser] homeFolderInContext: context];
@@ -380,7 +481,15 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
                                    inContext: context
                                      acquire: NO];
 
-  error = [folderToUpdate renameTo: displayName];
+  // tfu if parrent is 0 or displayname is not changed it is either a rename of a folder in 0 or a move to 0
+  if ([parentId isEqualToString: @"0"] ||([serverId hasSuffix: [NSString stringWithFormat: @"/%@", displayName]] && [parentId isEqualToString: @"0"] ))
+  {
+      error = [folderToUpdate renameTo: [NSString stringWithFormat: @"/%@", [displayName stringByEncodingImap4FolderName] ]];
+  }
+  else
+  {
+      error = [folderToUpdate renameTo: [NSString stringWithFormat: @"%@/%@", [[parentId stringByUnescapingURL] substringFromIndex:5] , [displayName stringByEncodingImap4FolderName] ]];
+  }
 
   // Handle new name exist
   if (!error)
@@ -396,7 +505,8 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
       // See http://msdn.microsoft.com/en-us/library/gg675615(v=exchg.80).aspx
       // we return '9' - we force a FolderSync
-      status = 9;
+      //status = 9;
+      status = 1;
 
       [self _setFolderSyncKey: syncKey];
 
@@ -414,8 +524,21 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     }
   else
     {
-      [theResponse setStatus: 500];
-      [theResponse appendContentString: @"Unable to update folder."];
+      //[theResponse setStatus: 500];
+      //[theResponse appendContentString: @"Unable to update folder."];
+
+      NSMutableString *s;
+      NSData *d;
+      s = [NSMutableString string];
+      [s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
+      [s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\">"];
+      [s appendString: @"<FolderUpdate xmlns=\"FolderHierarchy:\">"];
+      [s appendFormat: @"<Status>%d</Status>", 4]; // issue a folderSync
+      [s appendString: @"</FolderUpdate>"];
+      
+      d = [[s dataUsingEncoding: NSUTF8StringEncoding] xml2wbxml];
+      
+      [theResponse setContent: d];
     }
 }
 
@@ -431,16 +554,17 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
                 inResponse: (WOResponse *) theResponse
 {
   NSMutableDictionary *metadata;
-  NSMutableString *s;
+  NSMutableString *s ,*s1;
   NSString *syncKey;
   NSData *d;
   
-  BOOL first_sync;
+  BOOL first_sync , found;
   int status;
 
   metadata = [[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]];
   syncKey = [[(id)[theDocumentElement getElementsByTagName: @"SyncKey"] lastObject] textValue];
   s = [NSMutableString string];
+  s1 = [NSMutableString string];
 
   first_sync = NO;
   status = 1;
@@ -449,6 +573,8 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     {
       first_sync = YES;
       syncKey = @"1";
+      // tfu clear saved folder structure
+      [self _clear];
     }
   else if (![syncKey isEqualToString: [[metadata objectForKey: @"FolderSync"] objectForKey: @"SyncKey"]])
     {
@@ -458,13 +584,14 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   [self _setFolderSyncKey: syncKey];
 
+
   [s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
   [s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\">"];
   [s appendFormat: @"<FolderSync xmlns=\"FolderHierarchy:\"><Status>%d</Status><SyncKey>%@</SyncKey><Changes>", status, syncKey];
   
   // Initial sync, let's return the complete folder list
-  if (first_sync)
-    {
+  if (status == 1)  {
+
       SOGoMailAccounts *accountsFolder;
       SOGoMailAccount *accountFolder;
       SOGoUserFolder *userFolder;
@@ -474,21 +601,51 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
       NSArray *allFoldersMetadata;
       NSString *name, *serverId, *parentId;
 
-      int i, type;
-      
+      int i, type, command_count;
+      NSMutableString *commands;
+
       userFolder = [[context activeUser] homeFolderInContext: context];
       accountsFolder = [userFolder lookupName: @"Mail"  inContext: context  acquire: NO];
       accountFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
 
       allFoldersMetadata = [accountFolder allFoldersMetadata];
 
-      // See 2.2.3.170.3 Type (FolderSync) - http://msdn.microsoft.com/en-us/library/gg650877(v=exchg.80).aspx
-      [s appendFormat: @"<Count>%d</Count>", [allFoldersMetadata count]+3];
+      // tfu deal with deleted folders
+      NSEnumerator *keyEnum = [ [[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]] keyEnumerator];
+      NSString *key;
+
+      command_count=0;
+      commands = [NSMutableString string];
+
+      while ((key = [keyEnum nextObject]))
+      {
+         found = NO;
+
+         for (i = 0; i < [allFoldersMetadata count]; i++)
+         {
+            folderMetadata = [allFoldersMetadata objectAtIndex: i];
+
+            if ( [key isEqualToString:   [NSString stringWithFormat: @"mail%@-%@", [folderMetadata objectForKey: @"path"], [folderMetadata objectForKey: @"uidvalidity"]]]) {
+               found=YES;
+               break;
+            }
+         }
+
+         // foldersync as a key which should be deleleted here - need to have a better struct
+         if (!found && ![key isEqualToString: @"FolderSync"] ) {
+             [ self _setDeleteFolder: key];
+             [commands appendFormat: @"<Delete><ServerId>%@</ServerId></Delete>", [key stringByEscapingURL]],
 
-      for (i = 0; i < [allFoldersMetadata count]; i++)
+             command_count++;
+         }
+
+     }
+
+     // tfu deail with addition and changes
+     for (i = 0; i < [allFoldersMetadata count]; i++)
         {
           folderMetadata = [allFoldersMetadata objectAtIndex: i];
-          serverId = [NSString stringWithFormat: @"mail%@", [folderMetadata objectForKey: @"path"]];
+          serverId = [NSString stringWithFormat: @"mail%@-%@", [folderMetadata objectForKey: @"path"], [folderMetadata objectForKey: @"uidvalidity"]];
           name = [folderMetadata objectForKey: @"displayName"];
           
           if ([name hasPrefix: @"/"])
@@ -503,16 +660,45 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
           if ([folderMetadata objectForKey: @"parent"])
             {
-              parentId = [NSString stringWithFormat: @"mail%@", [folderMetadata objectForKey: @"parent"]];
+              parentId = [NSString stringWithFormat: @"mail%@-%@", [folderMetadata objectForKey: @"parent"], [folderMetadata objectForKey: @"parentuidvalidity"]];
               name = [[name pathComponents] lastObject];
             }
 
-          [s appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>",
-             [serverId stringByEscapingURL],
-             [parentId stringByEscapingURL],
-             type,
-             [name activeSyncRepresentationInContext: context]];
-        }
+         if ([[[[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]] objectForKey:  serverId] objectForKey: @"FolderName"]) {
+            if (![ [folderMetadata objectForKey: @"path"] isEqualToString: [[[[[context activeUser] userSettings] microsoftActiveSyncMetadataForDevice: [context objectForKey: @"DeviceId"]] objectForKey: serverId] objectForKey: @"FolderName"]]) {
+                [commands appendFormat: @"<Update><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Update>",
+                         [serverId stringByEscapingURL],
+                         [parentId stringByEscapingURL],
+                         type,
+                         [name activeSyncRepresentationInContext: context]];
+
+                command_count++;
+            }
+         }
+         else {
+                [commands appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>",
+                         [serverId stringByEscapingURL],
+                         [parentId stringByEscapingURL],
+                         type,
+                         [name activeSyncRepresentationInContext: context]];
+                 command_count++;
+         }
+
+        [self _setFolder:   serverId  foldername: [folderMetadata objectForKey: @"path"]];
+
+       }
+
+       if (first_sync)
+             [s appendFormat: @"<Count>%d</Count>", command_count+3];
+       else
+             [s appendFormat: @"<Count>%d</Count>", command_count];
+
+       if (command_count > 0)
+         [s appendFormat: @"%@", commands];
+
+  // Initial sync, let's return the complete folder list
+  if (first_sync)
+    {
 
       // We add the personal calendar - events
       // FIXME: add all calendars
@@ -533,6 +719,11 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
       [s appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>", name, @"0", 9, [[currentFolder displayName] activeSyncRepresentationInContext: context]];
     }
 
+// end status = 1
+    }
+
+
+
   [s appendString: @"</Changes></FolderSync>"];
 
   d = [[s dataUsingEncoding: NSUTF8StringEncoding] xml2wbxml];
diff --git a/SoObjects/Mailer/SOGoMailAccount.m b/SoObjects/Mailer/SOGoMailAccount.m
index 5f9a67c..7107e17 100644
--- a/SoObjects/Mailer/SOGoMailAccount.m
+++ b/SoObjects/Mailer/SOGoMailAccount.m
@@ -60,6 +60,7 @@
 #import "SOGoUser+Mailer.h"
 
 #import "SOGoMailAccount.h"
+#import <Foundation/NSProcessInfo.h>
 
 #define XMLNS_INVERSEDAV @"urn:inverse:params:xml:ns:inverse-dav"
 
@@ -435,7 +436,7 @@ static NSString *inboxFolderName = @"INBOX";
 //
 - (NSArray *) allFoldersMetadata
 {
-  NSString *currentFolder, *currentDecodedFolder, *currentDisplayName, *currentFolderType, *login, *fullName, *parent;
+  NSString *currentFolder, *currentDecodedFolder, *currentDisplayName, *currentFolderType, *login, *fullName, *parent, *uidvalidity, *parentuidvalidity;
   NSMutableArray *pathComponents, *folders;
   SOGoUserManager *userManager;
   NSEnumerator *rawFolders;
@@ -497,12 +498,19 @@ static NSString *inboxFolderName = @"INBOX";
 
       parent = [self _parentForFolder: currentFolder  foldersList: allFolderPaths];
       
+      // tfu uidvalidity is used for foldersync 
+      uidvalidity = [self getUIDValidityForFolder: currentFolder];
+      parentuidvalidity = [self getUIDValidityForFolder: parent];
+      
       folderData = [NSDictionary dictionaryWithObjectsAndKeys:
                                    currentFolder, @"path",
                                  currentFolderType, @"type",
                                  currentDisplayName, @"displayName",
+                                 uidvalidity, @"uidvalidity",
+                                 parentuidvalidity, @"parentuidvalidity",
                                  parent, @"parent",
                                  nil];
+
       [folders addObject: folderData];
       [pool release];
     }
@@ -961,4 +969,24 @@ static NSString *inboxFolderName = @"INBOX";
   return [[self _mailAccount] objectForKey: @"name"];
 }
 
+
+// tfu get uidvalidity - to be used in allFoldersMetadata
+- (NSString *) getUIDValidityForFolder: (NSString *) folderName
+{
+  NGImap4Client *client;
+  NSString *uidValidity;
+  NSDictionary *result;
+
+  client = [[self imap4Connection] client];
+  result = [client select: folderName];
+
+  if ([[result objectForKey: @"result"] boolValue])
+    uidValidity = [result objectForKey:@"uidvalidity"];
+  else
+    uidValidity =  @"0"; 
+
+  return uidValidity;
+}
+
+
 @end /* SOGoMailAccount */
diff --git a/SoObjects/Mailer/SOGoMailFolder.h b/SoObjects/Mailer/SOGoMailFolder.h
index eac1aec..baabc95 100644
--- a/SoObjects/Mailer/SOGoMailFolder.h
+++ b/SoObjects/Mailer/SOGoMailFolder.h
@@ -120,6 +120,9 @@
 - (id) appendMessage: (NSData *) message
              usingId: (int *) imap4id;
 
+//tfu
+- (NSString *) getUIDValidity;
+
 @end
 
 @interface SOGoSpecialMailFolder : SOGoMailFolder
diff --git a/SoObjects/Mailer/SOGoMailFolder.m b/SoObjects/Mailer/SOGoMailFolder.m
index 7f45ec8..b16d62c 100644
--- a/SoObjects/Mailer/SOGoMailFolder.m
+++ b/SoObjects/Mailer/SOGoMailFolder.m
@@ -291,10 +291,18 @@ static NSString *defaultUserID =  @"anyone";
           path = [[imap4URL path] stringByDeletingLastPathComponent];
           if (![path hasSuffix: @"/"])
             path = [path stringByAppendingString: @"/"];
+
+          if ([newName rangeOfString: @"/"].location == NSNotFound)
           destURL = [[NSURL alloc] initWithScheme: [imap4URL scheme]
                                              host: [imap4URL host]
                                              path: [NSString stringWithFormat: @"%@%@",
                                                              path, [newName stringByEncodingImap4FolderName]]];
+          else
+          destURL = [[NSURL alloc] initWithScheme: [imap4URL scheme]
+                                             host: [imap4URL host]
+                                             path: [NSString stringWithFormat: @"%@",
+                                                              [newName stringByEncodingImap4FolderName]]];
+
           [destURL autorelease];
           error = [imap4 moveMailboxAtURL: imap4URL
                                     toURL: destURL];
@@ -1973,6 +1981,33 @@ static NSString *defaultUserID =  @"anyone";
   return tag;
 }
 
+
+// tfu
+- (NSString *) getUIDValidity
+{
+  NSString *uidValidity;
+
+  uidValidity = @"0";
+
+  if ([self imap4Connection])
+    {
+      NSString *folderName;
+      NSDictionary *result;
+
+      folderName = [imap4 imap4FolderNameForURL: [self imap4URL]];
+
+      [[imap4 client] unselect];
+
+      result = [[imap4 client] select: folderName];
+
+      uidValidity =  [result objectForKey: @"uidvalidity"];
+    }
+
+  return uidValidity;
+
+}
+
+
 //
 // FIXME - see below for code refactoring with MAPIStoreMailFolder.
 //
-- 
1.7.9.5

0001-folder-sync.patch (22,738 bytes)   
tfu

tfu

2014-04-13 18:41

reporter  

sope-0001-folder-sync-uidvalidity.patch (1,788 bytes)   
From e95cdcedeb65ceec80ad0193e22ecc0cca80372a Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Sat, 12 Apr 2014 22:24:55 +0200
Subject: [PATCH] folder sync uidvalidity

---
 sope-mime/NGImap4/NGImap4Client.m             |    2 ++
 sope-mime/NGImap4/NGImap4ResponseNormalizer.m |    2 ++
 2 files changed, 4 insertions(+)

diff --git a/sope-mime/NGImap4/NGImap4Client.m b/sope-mime/NGImap4/NGImap4Client.m
index 02af32b..e8db54e 100644
--- a/sope-mime/NGImap4/NGImap4Client.m
+++ b/sope-mime/NGImap4/NGImap4Client.m
@@ -931,11 +931,13 @@ static NSMutableDictionary *namespaces;
 - (NSDictionary *)rename:(NSString *)_folder to:(NSString *)_newName {
   NSString *cmd;
   
+NSLog(@"tfu imap rename1 %@ --  %@",  _folder, _newName);
   if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
     return nil;
   if ((_newName = [self _folder2ImapFolder:_newName]) == nil)
     return nil;
   
+NSLog(@"tfu imap rename %@ --  %@",  SaneFolderName(_folder), SaneFolderName(_newName));
   cmd = [NSString stringWithFormat:@"rename \"%@\" \"%@\"",
                   SaneFolderName(_folder), SaneFolderName(_newName)];
   
diff --git a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
index 687c3dc..548e950 100644
--- a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
+++ b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
@@ -268,6 +268,8 @@ static int      LogImapEnabled = -1;
         [result setObject:o forKey:@"highestmodseq"];
       else if ((o = [obj objectForKey:@"UIDNEXT"]))
         [result setObject:o forKey:@"uidnext"];
+      else if ((o = [obj objectForKey:@"UIDVALIDITY"]))
+        [result setObject:o forKey:@"uidvalidity"];
     }
     else
       [self warnWithFormat:@"unexpected OK object: %@", obj];
-- 
1.7.9.5

tfu

tfu

2014-04-13 18:54

reporter   ~0006884

I played a little bit with foldersync and the uploaded patches are the result. I added uidvalidity to the serverid to dedect a folderchange on server. According to the IMAP specs a folder needs to be refreshed when uidvalidity is changed. If that happends the folder is deleted an added back by foldersync.

I'm sure there is some improvement possible but the pachtes should give some ideas on how to deal with foldersync.

Open things (I'm sure there are more): need to dedect folder changes during ping to trigger a foldersync.
On rename outlook appends '[1]' to the foldername - this seems to be caused by a foldersync containing both delete and add actions for the same folder.

ludovic

ludovic

2014-04-13 21:36

administrator   ~0006885

Hi,

Thanks for your patch. While interesting, I think it isn't the right approach.

In my opinion, the right approach would be to use the X-GUID option in Dovecot or an IMAP annotation to correctly set (or get) the unique ID of a mailbox.

Have a look at this:

http://wiki.kolab.org/User:Bruederli/Drafts/KEP:Folder_Display_Names_and_Unique_Identifiers

Would you be willing to work on a patch using this technique? It would be awesome.

tfu

tfu

2014-04-14 07:58

reporter   ~0006886

Is it just approach on how the serverid is build or do you think that other things (e.g. the way I save the synced folderlist) should also be changed.

X-GUID sounds like Dovecot-Only - what imap-server support IMAP annotation(METADATA) available?

ludovic

ludovic

2014-04-14 11:01

administrator   ~0006887

IMAP annotations are well supported by Cyrus and Dovecot (using a plugin: http://hg.dovecot.org/dovecot-metadata-plugin)

If we want to avoid a plugin on Dovecot, nothing would prevent us from using X-GUID on Dovecot and annotations for Cyrus.

tfu

tfu

2014-04-14 19:17

reporter   ~0006898

I tried to get more details on the imap-metadata-extention and cyrus:
metadata (rfc5464) doesn't seem to be available in cyrus yet - should be there in 2.5.
Do you have other infos about the metadata-extention?

ludovic

ludovic

2014-04-22 19:03

administrator   ~0006950

Cyrus 2.4 (and below) implements the ANNOTATEMORE (one of the RFC revision).

I guess we could use that with a /vendor/* tag.

tfu

tfu

2014-04-22 19:18

reporter   ~0006952

That means:

  • dovecot - usie X-GUID to get the unique folder-id
  • cyrus 2.4 and below - use annotatemore and set a unique id and use it as folder-id
  • cyrus 2.5 use '/vendor/cmu/cyrus-imapd/uniqueid' (metadata) for folder id

What about other imap servers?

ludovic

ludovic

2014-04-22 20:29

administrator   ~0006953

Other IMAP server are likely not supported by SOGo anyway due to the lack of QRESYNC support.

tfu

tfu

2014-04-28 20:34

reporter   ~0006966

Please have a look into attached patches. It should be the first step to implement foldersync based on a unique folder id.
The unique-id for the folder is determined in following order:
X-GUID (dovecot)
-> getannotation (currenlty /comment is used for testing purpose)
-> setannotation of a new id if not found by getannotation
-> if setannotation fails build the id based on <foldername>-<uidvalidity>.

After having an accepted way to get the uniqueId it could be used with the foldersync-patch.

tfu

tfu

2014-04-28 20:34

reporter  

0003-annotations.patch (7,662 bytes)   
From 51e76c0311f47f7607218629ae5ccdbd69000bfe Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Mon, 28 Apr 2014 21:14:10 +0200
Subject: [PATCH 3/3] annotations

---
 sope-mime/NGImap4/NGImap4Client.h         |    2 +
 sope-mime/NGImap4/NGImap4Client.m         |   74 ++++++++++++++++++++++++++++-
 sope-mime/NGImap4/NGImap4ResponseParser.m |   71 +++++++++++++++++++++++++--
 3 files changed, 143 insertions(+), 4 deletions(-)

diff --git a/sope-mime/NGImap4/NGImap4Client.h b/sope-mime/NGImap4/NGImap4Client.h
index 8442062..a5c3d17 100644
--- a/sope-mime/NGImap4/NGImap4Client.h
+++ b/sope-mime/NGImap4/NGImap4Client.h
@@ -136,6 +136,8 @@ typedef enum {
 - (NSDictionary *)select:(NSString *)_folder;
 - (NSDictionary *)unselect;
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute  attributeValue:(NSString *)_value;
 - (NSDictionary *)rename:(NSString *)_folder to:(NSString *)_newName;
 - (NSDictionary *)delete:(NSString *)_folder;
 - (NSDictionary *)create:(NSString *)_name;
diff --git a/sope-mime/NGImap4/NGImap4Client.m b/sope-mime/NGImap4/NGImap4Client.m
index af66b0e..9caf861 100644
--- a/sope-mime/NGImap4/NGImap4Client.m
+++ b/sope-mime/NGImap4/NGImap4Client.m
@@ -908,6 +908,77 @@ static NSMutableDictionary *namespaces;
   return [self->normer normalizeResponse:[self processCommand:@"unselect"]];
 }
 
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute {
+/*
+ result dict looks like the following  
+ {"/comment" = {"value.priv" = "My comment"; }; "/vendor/cmu/cyrus-imapd/freespace" = {"value.shared" = 93498228; }; }
+
+ getannotation:
+
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value.priv"];
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value"];
+ result = [client annotation: folderName entryName: @"/*" attributeName: @"value"];
+ result = [client annotation: @"" entryName: @"/*" attributeName: @"value"];
+
+*/
+  NSString *cmd;
+  NGHashMap *_map;
+  NSDictionary        *obj;
+  NSMutableDictionary *result;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"getannotation \"%@\" \"%@\" \"%@\"",
+                      SaneFolderName(_folder), _entry, _attribute];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  _map = [self processCommand:cmd];
+
+  NSEnumerator        *enumerator;
+  enumerator = [_map objectEnumeratorForKey:@"entry"];
+  while ((obj = [enumerator nextObject]) != nil) {
+
+    NSLog(@"tfu getannotation entry %@", obj);
+    [result addEntriesFromDictionary:[NSDictionary dictionaryWithDictionary:obj]];
+  }
+
+  NSLog(@"tfu getannotation result %@", result);
+  return result;
+
+}
+
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute attributeValue:(NSString *)_value {
+  NSString *cmd;
+  NSMutableDictionary *result;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_value == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"setannotation \"%@\" \"%@\" (\"%@\" \"%@\")",
+                      SaneFolderName(_folder), _entry, _attribute, _value];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  result = [self->normer normalizeResponse:[self processCommand:cmd]];
+  NSLog(@"tfu setannotation result %@", result);
+  return result;
+
+}
+
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags {
   NSString *cmd;
   
@@ -918,7 +989,7 @@ static NSMutableDictionary *namespaces;
   if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
     return nil;
   
-  cmd     = [NSString stringWithFormat:@"status \"%@\" (%@)",
+  cmd = [NSString stringWithFormat:@"status \"%@\" (%@)",
                       SaneFolderName(_folder), [_flags componentsJoinedByString:@" "]];
   return [self->normer normalizeStatusResponse:[self processCommand:cmd]];
 }
@@ -990,6 +1061,7 @@ NSLog(@"tfu imap rename %@ --  %@",  SaneFolderName(_folder), SaneFolderName(_ne
   return [_parts componentsJoinedByString:@" "];
 }
 
+
 - (NSDictionary *)fetchUids:(NSArray *)_uids parts:(NSArray *)_parts {
   /*
     eg: 'UID FETCH 1189,1325,1326 ([TODO])'
diff --git a/sope-mime/NGImap4/NGImap4ResponseParser.m b/sope-mime/NGImap4/NGImap4ResponseParser.m
index 952acad..02ffe69 100644
--- a/sope-mime/NGImap4/NGImap4ResponseParser.m
+++ b/sope-mime/NGImap4/NGImap4ResponseParser.m
@@ -11,7 +11,7 @@
   SOPE is distributed in the hope that it will be useful, but WITHOUT ANY
   WARRANTY; without even the implied warranty of MERCHANTABILITY or
   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public
-  License for more details.
+/bin/bash: 58: command not found
 
   You should have received a copy of the GNU Lesser General Public
   License along with SOPE; see the file COPYING.  If not, write to the
@@ -604,8 +604,10 @@ static void _parseUntaggedResponse(NGImap4ResponseParser *self,
   l0 = _la(self, 0);
   switch (l0) {
   case 'A':
-//tfu hier annodate
-    if ([self _parseACLResponseIntoHashMap:result_])
+    l1 = _la(self, 1);
+    if (l1 == 'C' && [self _parseACLResponseIntoHashMap:result_])
+      return;
+    if (l1 == 'N' && [self _parseAnnotationResponseIntoHashMap:result_]) 
       return;
     break;
     
@@ -1452,6 +1454,69 @@ _purifyQuotedString(NSMutableString *quotedString) {
   return YES;
 }
 
+- (BOOL)_parseAnnotationResponseIntoHashMap:(NGMutableHashMap *)result_ {
+  NSString            *name  = nil;
+  NSString            *entry  = nil;
+  NSMutableDictionary *attributes = nil;
+  NSDictionary *d;
+
+  if (!_matchesString(self, "ANNOTATION "))
+    return NO;
+
+  _consume(self, 11);
+
+  if (_la(self, 0) == '"') {
+    name = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    name = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    name = _parseUntil(self, ' ');
+  }
+
+  if (_la(self, 0) == '"') {
+    entry = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    entry = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    entry = _parseUntil(self, ' ');
+  }
+
+  _consumeIfMatch(self, '(');
+
+  attributes = [NSMutableDictionary dictionaryWithCapacity:2];
+  d = [NSMutableDictionary dictionaryWithCapacity:2];
+
+  while (_la(self, 0) != ')') {
+    NSString *key   = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+    NSString *value = [self _parseQuotedString];
+
+    if (_la(self, 0) == ' ')
+      _consume(self, 1);
+
+    [attributes setObject:value
+           forKey:[key lowercaseString]];
+
+    [d setObject:[NSDictionary dictionaryWithDictionary:attributes]
+       forKey:entry];
+  }
+  _consumeIfMatch(self, ')');
+  _parseUntil(self, '\n');
+
+  [result_ addObject:d forKey:@"entry"];
+  [d release];
+  return YES;
+}
+
+
 - (BOOL)_parseByeUntaggedResponseIntoHashMap:(NGMutableHashMap *)result_ {
   NSString *reason;
   
-- 
1.7.9.5

0003-annotations.patch (7,662 bytes)   
tfu

tfu

2014-04-28 20:35

reporter  

0001-uniqueid.patch (3,440 bytes)   
From 0f499f4697b270063792d12d086eb6aff6132679 Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Mon, 28 Apr 2014 21:15:24 +0200
Subject: [PATCH] uniqueid

---
 ActiveSync/SOGoActiveSyncDispatcher.m |    3 +++
 SoObjects/Mailer/SOGoMailFolder.m     |   44 +++++++++++++++++++++++++++++++++
 2 files changed, 47 insertions(+)

diff --git a/ActiveSync/SOGoActiveSyncDispatcher.m b/ActiveSync/SOGoActiveSyncDispatcher.m
index b1b8e4e..9fd800e 100644
--- a/ActiveSync/SOGoActiveSyncDispatcher.m
+++ b/ActiveSync/SOGoActiveSyncDispatcher.m
@@ -618,6 +618,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
       NSCalendarDate *filter;
       NSArray *uids;
 
+//tfu hack
+NSLog(@"tfu unique %@" , [currentCollection uniqueid]);
+
       filter = [NSCalendarDate dateFromFilterType: [[(id)[theDocumentElement getElementsByTagName: @"FilterType"] lastObject] textValue]];
       
       notDeletedQualifier =  [EOQualifier qualifierWithQualifierFormat:
diff --git a/SoObjects/Mailer/SOGoMailFolder.m b/SoObjects/Mailer/SOGoMailFolder.m
index 6408828..492bfd8 100644
--- a/SoObjects/Mailer/SOGoMailFolder.m
+++ b/SoObjects/Mailer/SOGoMailFolder.m
@@ -67,6 +67,8 @@
 #import "SOGoMailManager.h"
 #import "SOGoMailFolder.h"
 #import "SOGoTrashFolder.h"
+#import <Foundation/NSProcessInfo.h>
+
 
 #define XMLNS_INVERSEDAV @"urn:inverse:params:xml:ns:inverse-dav"
 
@@ -757,6 +759,48 @@ static NSString *defaultUserID =  @"anyone";
   return result;
 }
 
+//tfu
+- (NSString *) uniqueid
+{
+  NGImap4Client *client;
+  NSString *folderName, *folderUid, *folderUidValidity ;
+  NSDictionary *result;
+
+  client = [[self imap4Connection] client];
+  folderName = [[self imap4Connection] imap4FolderNameForURL: [self imap4URL]];
+
+  result = [ client select: folderName];
+  
+  folderUid = [result objectForKey: @"x-guid"];
+  folderUidValidity = [result objectForKey: @"uidvalidity"];
+
+  NSLog(@"tfu uniqueid x-guid %@ uidvalidity %@", folderUid, folderUidValidity);
+
+  if (!folderUid) {
+
+     NSLog(@"tfu uniqueid folderName %@", folderName);
+     result = [client annotation: folderName entryName: @"/comment" attributeName: @"value.priv"];
+     //result = [client annotation: folderName entryName: @"/comment" attributeName: @"value"];
+     //result = [client annotation: folderName entryName: @"/*" attributeName: @"value"];
+     result = [client annotation: @"" entryName: @"/*" attributeName: @"value"];
+     if ([result objectForKey: @"/comment"]) {
+        NSLog(@"tfu uniqueid folderuid from annotation %@", [[result objectForKey: @"/comment"] objectForKey: @"value.priv"]);
+        folderUid = [[result objectForKey: @"/comment"] objectForKey: @"value.priv"];
+     } else {
+            folderUid = [[NSProcessInfo processInfo] globallyUniqueString];
+            result = [client annotation: folderName entryName: @"/comment" attributeName: @"value.priv" attributeValue: folderUid ];
+            if (![[result objectForKey: @"result"] boolValue]) {
+               folderUid = [NSString stringWithFormat: @"%@-%@", folderName,  folderUidValidity ];
+               NSLog(@"tfu uniqueid setannotation failed: %@", result );
+               NSLog(@"tfu uniqueid folderid: %@", folderUid );
+            }
+     }
+  } 
+  
+  NSLog(@"tfu annotation folderUid  %@", folderUid );
+  return folderUid;
+}
+
 - (NSDictionary *) statusForFlags: (NSArray *) flags
 {
   NGImap4Client *client;
-- 
1.7.9.5

0001-uniqueid.patch (3,440 bytes)   
tfu

tfu

2014-04-28 20:36

reporter  

0001-annotations.patch (9,002 bytes)   
From 5d3a958c64c46f4c2c628fedd4fe3152a56ea4ce Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Mon, 28 Apr 2014 21:25:01 +0200
Subject: [PATCH] annotations

---
 sope-mime/NGImap4/NGImap4Client.h             |    2 +
 sope-mime/NGImap4/NGImap4Client.m             |   76 ++++++++++++++++++++++++-
 sope-mime/NGImap4/NGImap4ResponseNormalizer.m |    6 ++
 sope-mime/NGImap4/NGImap4ResponseParser.m     |   70 ++++++++++++++++++++++-
 4 files changed, 150 insertions(+), 4 deletions(-)

diff --git a/sope-mime/NGImap4/NGImap4Client.h b/sope-mime/NGImap4/NGImap4Client.h
index 8442062..a5c3d17 100644
--- a/sope-mime/NGImap4/NGImap4Client.h
+++ b/sope-mime/NGImap4/NGImap4Client.h
@@ -136,6 +136,8 @@ typedef enum {
 - (NSDictionary *)select:(NSString *)_folder;
 - (NSDictionary *)unselect;
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute  attributeValue:(NSString *)_value;
 - (NSDictionary *)rename:(NSString *)_folder to:(NSString *)_newName;
 - (NSDictionary *)delete:(NSString *)_folder;
 - (NSDictionary *)create:(NSString *)_name;
diff --git a/sope-mime/NGImap4/NGImap4Client.m b/sope-mime/NGImap4/NGImap4Client.m
index 02af32b..1d84204 100644
--- a/sope-mime/NGImap4/NGImap4Client.m
+++ b/sope-mime/NGImap4/NGImap4Client.m
@@ -118,7 +118,7 @@ static int          ProfileImapEnabled = -1;
 static int          LogImapEnabled     = -1;
 static int          PreventExceptions  = -1;
 static BOOL         fetchDebug         = NO;
-static BOOL         ImapDebugEnabled   = NO;
+static BOOL         ImapDebugEnabled   = YES;
 static NSArray      *Imap4SystemFlags  = nil;
 
 static NSMutableDictionary *capabilities;
@@ -908,6 +908,77 @@ static NSMutableDictionary *namespaces;
   return [self->normer normalizeResponse:[self processCommand:@"unselect"]];
 }
 
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute {
+/*
+ result dict looks like the following  
+ {"/comment" = {"value.priv" = "My comment"; }; "/vendor/cmu/cyrus-imapd/freespace" = {"value.shared" = 93498228; }; }
+
+ getannotation:
+
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value.priv"];
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value"];
+ result = [client annotation: folderName entryName: @"/*" attributeName: @"value"];
+ result = [client annotation: @"" entryName: @"/*" attributeName: @"value"];
+
+*/
+  NSString *cmd;
+  NGHashMap *_map;
+  NSDictionary        *obj;
+  NSMutableDictionary *result;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"getannotation \"%@\" \"%@\" \"%@\"",
+                      SaneFolderName(_folder), _entry, _attribute];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  _map = [self processCommand:cmd];
+
+  NSEnumerator        *enumerator;
+  enumerator = [_map objectEnumeratorForKey:@"entry"];
+  while ((obj = [enumerator nextObject]) != nil) {
+
+    NSLog(@"tfu getannotation entry %@", obj);
+    [result addEntriesFromDictionary:[NSDictionary dictionaryWithDictionary:obj]];
+  }
+
+  NSLog(@"tfu getannotation result %@", result);
+  return result;
+
+}
+
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute attributeValue:(NSString *)_value {
+  NSString *cmd;
+  NSMutableDictionary *result;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_value == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"setannotation \"%@\" \"%@\" (\"%@\" \"%@\")",
+                      SaneFolderName(_folder), _entry, _attribute, _value];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  result = [self->normer normalizeResponse:[self processCommand:cmd]];
+  NSLog(@"tfu setannotation result %@", result);
+  return result;
+
+}
+
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags {
   NSString *cmd;
   
@@ -918,7 +989,7 @@ static NSMutableDictionary *namespaces;
   if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
     return nil;
   
-  cmd     = [NSString stringWithFormat:@"status \"%@\" (%@)",
+  cmd = [NSString stringWithFormat:@"status \"%@\" (%@)",
                       SaneFolderName(_folder), [_flags componentsJoinedByString:@" "]];
   return [self->normer normalizeStatusResponse:[self processCommand:cmd]];
 }
@@ -988,6 +1059,7 @@ static NSMutableDictionary *namespaces;
   return [_parts componentsJoinedByString:@" "];
 }
 
+
 - (NSDictionary *)fetchUids:(NSArray *)_uids parts:(NSArray *)_parts {
   /*
     eg: 'UID FETCH 1189,1325,1326 ([TODO])'
diff --git a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
index 687c3dc..95c9a85 100644
--- a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
+++ b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
@@ -268,6 +268,8 @@ static int      LogImapEnabled = -1;
         [result setObject:o forKey:@"highestmodseq"];
       else if ((o = [obj objectForKey:@"UIDNEXT"]))
         [result setObject:o forKey:@"uidnext"];
+      else if ((o = [obj objectForKey:@"UIDVALIDITY"]))
+        [result setObject:o forKey:@"uidvalidity"];
     }
     else
       [self warnWithFormat:@"unexpected OK object: %@", obj];
@@ -318,6 +320,10 @@ static int      LogImapEnabled = -1;
   }
   if ((o = [obj  objectForKey:@"unseen"]) != nil)
     [result setObject:o forKey:@"unseen"];
+
+//tfu support x-guid (dovecot)
+  if ((o = [obj  objectForKey:@"x-guid"]) != nil)
+    [result setObject:o forKey:@"x-guid"];
   
   return result;
 }
diff --git a/sope-mime/NGImap4/NGImap4ResponseParser.m b/sope-mime/NGImap4/NGImap4ResponseParser.m
index f14c889..02ffe69 100644
--- a/sope-mime/NGImap4/NGImap4ResponseParser.m
+++ b/sope-mime/NGImap4/NGImap4ResponseParser.m
@@ -11,7 +11,7 @@
   SOPE is distributed in the hope that it will be useful, but WITHOUT ANY
   WARRANTY; without even the implied warranty of MERCHANTABILITY or
   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public
-  License for more details.
+/bin/bash: 58: command not found
 
   You should have received a copy of the GNU Lesser General Public
   License along with SOPE; see the file COPYING.  If not, write to the
@@ -604,7 +604,10 @@ static void _parseUntaggedResponse(NGImap4ResponseParser *self,
   l0 = _la(self, 0);
   switch (l0) {
   case 'A':
-    if ([self _parseACLResponseIntoHashMap:result_])
+    l1 = _la(self, 1);
+    if (l1 == 'C' && [self _parseACLResponseIntoHashMap:result_])
+      return;
+    if (l1 == 'N' && [self _parseAnnotationResponseIntoHashMap:result_]) 
       return;
     break;
     
@@ -1451,6 +1454,69 @@ _purifyQuotedString(NSMutableString *quotedString) {
   return YES;
 }
 
+- (BOOL)_parseAnnotationResponseIntoHashMap:(NGMutableHashMap *)result_ {
+  NSString            *name  = nil;
+  NSString            *entry  = nil;
+  NSMutableDictionary *attributes = nil;
+  NSDictionary *d;
+
+  if (!_matchesString(self, "ANNOTATION "))
+    return NO;
+
+  _consume(self, 11);
+
+  if (_la(self, 0) == '"') {
+    name = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    name = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    name = _parseUntil(self, ' ');
+  }
+
+  if (_la(self, 0) == '"') {
+    entry = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    entry = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    entry = _parseUntil(self, ' ');
+  }
+
+  _consumeIfMatch(self, '(');
+
+  attributes = [NSMutableDictionary dictionaryWithCapacity:2];
+  d = [NSMutableDictionary dictionaryWithCapacity:2];
+
+  while (_la(self, 0) != ')') {
+    NSString *key   = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+    NSString *value = [self _parseQuotedString];
+
+    if (_la(self, 0) == ' ')
+      _consume(self, 1);
+
+    [attributes setObject:value
+           forKey:[key lowercaseString]];
+
+    [d setObject:[NSDictionary dictionaryWithDictionary:attributes]
+       forKey:entry];
+  }
+  _consumeIfMatch(self, ')');
+  _parseUntil(self, '\n');
+
+  [result_ addObject:d forKey:@"entry"];
+  [d release];
+  return YES;
+}
+
+
 - (BOOL)_parseByeUntaggedResponseIntoHashMap:(NGMutableHashMap *)result_ {
   NSString *reason;
   
-- 
1.7.9.5

0001-annotations.patch (9,002 bytes)   
tfu

tfu

2014-04-28 20:38

reporter   ~0006967

please ignore 0003-annotations.patch

ludovic

ludovic

2014-04-29 12:28

administrator   ~0006968

Which patched should be used?

Thanks for your excellent work!

tfu

tfu

2014-04-29 13:58

reporter   ~0006972

0001-annotations.patch contains the sope part which implements annotations
0001-uniqueid.patch implements the function to get the uniqueid for a folder

For testing purpose I misused processGetItemEstimate to call uniqueid.

As mentioned before as soon this part is clean and accepted the work can go on ...

ludovic

ludovic

2014-05-05 17:14

administrator   ~0006989

Sorry if it's taking a bit long to review but I've been rewriting the whole "metadata" handling code to synchronize mailboxes. Your patches (this one and the one in 2734) will need to be slightly reworked to handle the new, but simpler code.

ludovic

ludovic

2014-05-15 19:12

administrator   ~0007028

The new caching code has landed. You must re-create the ActiveSync profile if you want to test it.

Could you adapt your patch to the new code?

Please have a look at SOGActiveSyncDispatcher+Sync.m and search for "GUID". I've modified to code to ease your work.

tfu

tfu

2014-05-16 22:12

reporter   ~0007035

Could you please describe what you have in mind with the GUID? I think it is to late to generate the GUID at that location:
// TODO - Generate GUID
//[folderMetadata setObject: @"FOO-BAR-BAZ" forKey: @"GUID"];

Maybe I'm wrong but the GUID should be generated an saved somewhere while introducing the folder to the client via foldersync (ServerId).

It would be great if you could help to adapt the code puzzles I've provided to what you have planed with this GUID.

ludovic

ludovic

2014-05-17 00:01

administrator   ~0007036

You can of course set it earlier, during the FolderSync. It was just to show you "how it's done".

ludovic

ludovic

2014-05-22 13:38

administrator   ~0007067

I don't want to put any kind of pressure but do you expect to have an updated patch soon? I would like to include it in v2.2.4.

tfu

tfu

2014-05-22 20:40

reporter   ~0007081

I'm working on it but it will take some time. Maybe I can provide something until mid next week, but I'm sure there will be some changes based on your review.

Q: What is the right way to retrive a list of folders already stored in cache?
Do you have a suggestion on how to efficiently translate the "serverId"
and "parentId" from guid to the imap folder name?

ludovic

ludovic

2014-05-23 00:30

administrator   ~0007084

First question - you can call - (NSMutableDictionary *) _globalMetadataForDevice in SOGoActiveSyncDispatcher and adapt _setFolderSyncKey: to accept a dictionary so you can set your folders IDs.

Second question - I guess you'll generate a GUID for each IMAP folder - that becomes your key in a hash, value is the real IMAP folder name.

tfu

tfu

2014-05-26 17:40

reporter  

0001-foldersync.patch (31,951 bytes)   
From f0e459ca448eadefd315a7eb952db50fb5de8a0e Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Sun, 25 May 2014 23:57:54 +0200
Subject: [PATCH] foldersync

---
 ActiveSync/SOGoActiveSyncDispatcher+Sync.m |   33 ++--
 ActiveSync/SOGoActiveSyncDispatcher.m      |  254 +++++++++++++++++++++++-----
 SoObjects/Mailer/SOGoMailAccount.h         |    2 +
 SoObjects/Mailer/SOGoMailAccount.m         |   57 +++++++
 SoObjects/SOGo/SOGoCacheGCSObject.h        |    3 +
 SoObjects/SOGo/SOGoCacheGCSObject.m        |   41 +++++
 6 files changed, 340 insertions(+), 50 deletions(-)

diff --git a/ActiveSync/SOGoActiveSyncDispatcher+Sync.m b/ActiveSync/SOGoActiveSyncDispatcher+Sync.m
index 49cc21c..91f5d5d 100644
--- a/ActiveSync/SOGoActiveSyncDispatcher+Sync.m
+++ b/ActiveSync/SOGoActiveSyncDispatcher+Sync.m
@@ -123,7 +123,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   [o setTableUrl: [self folderTableURL]];
   [o reloadIfNeeded];
   
-  [[o properties] removeAllObjects];
+  //[[o properties] removeAllObjects];
+  [[o properties] removeObjectForKey: @"SyncCache"];
+  [[o properties] removeObjectForKey: @"DateCache"];
+
   [[o properties] addEntriesFromDictionary: theFolderMetadata];
   [o save];
 }
@@ -146,7 +149,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 //
 // <?xml version="1.0"?>
-// <!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microsoft.com/">
+// <!DOCTYPE ActiveSync PUBLIC "-//MICROSOFT//DTD ActiveSync//EN" "http://www.microm/">
 // <Sync xmlns="AirSync:">
 //  <Collections>
 //   <Collection>
@@ -627,20 +630,21 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
                                                                             sequence: [[[allMessages objectAtIndex: i] allValues] lastObject]]];
           }
         
-        // If it's a new Sync operation, ignore anything we might have
-        // in our preferences.
+        // If it's a new Sync operation, DateCache and SyncCache need to be deleted
+        // but GUID stored by folderSync shouldn't be touched
+        folderMetadata = [self _folderMetadataForKey: [theCollection nameInContainer]];
         if ([theSyncKey isEqualToString: @"-1"])
           {
-            folderMetadata = [NSMutableDictionary dictionary];
-            
             [folderMetadata setObject: [NSMutableDictionary dictionary]  forKey: @"SyncCache"];
             [folderMetadata setObject: [NSMutableDictionary dictionary]  forKey: @"DateCache"];
-            
-            // TODO - Generate GUID
-            //[folderMetadata setObject: @"FOO-BAR-BAZ"  forKey: @"GUID"];
           }
-        else
-          folderMetadata = [self _folderMetadataForKey: [theCollection nameInContainer]];
+        // check whether GUID in cache is equal to the GUID from imap - this is to avoid cache corruptions if a folder has been renamed and a new folder
+        // with the same name has been created but folderSync has not yet updated the cache
+        if (!([[theCollection nameInContainer] isEqualToString: 
+                      [NSString stringWithFormat: @"folder%@", [self _translateGuidToImapFolderName: [folderMetadata objectForKey: @"GUID"] type: theFolderType]]])) {
+           NSLog(@"guid mismatch don't sync now!"); // not sure whether it is ok just to return
+           return;
+	}
 
         syncCache = [folderMetadata objectForKey: @"SyncCache"];
         dateCache = [folderMetadata objectForKey: @"DateCache"];
@@ -711,6 +715,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
                     [s appendString: @"</Delete>"];
                     
                     [syncCache removeObjectForKey: [aCacheObject uid]];
+                    [dateCache removeObjectForKey: [aCacheObject uid]];
                   }
                 else
                   {
@@ -773,13 +778,15 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
                     //NSLog(@"skipping old deleted UID: %@",  [aCacheObject uid]);
                   }
               }
+
           }
-        
+
         if (more_available)
           [folderMetadata setObject: [NSNumber numberWithBool: YES]  forKey: @"MoreAvailable"];
         else
           [folderMetadata removeObjectForKey: @"MoreAvailable"];
         
+
         [self _setFolderMetadata: folderMetadata
                           forKey: [theCollection nameInContainer]];
       } // default:
@@ -890,6 +897,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   
   collectionId = [[(id)[theDocumentElement getElementsByTagName: @"CollectionId"] lastObject] textValue];
   realCollectionId = [collectionId realCollectionIdWithFolderType: &folderType];
+  realCollectionId = [self _translateGuidToImapFolderName: realCollectionId type: folderType];
   collection = [self collectionFromId: realCollectionId  type: folderType];
   
   syncKey = davCollectionTag = [[(id)[theDocumentElement getElementsByTagName: @"SyncKey"] lastObject] textValue];
@@ -988,6 +996,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
         davCollectionTag = [collection davCollectionTag];
     }
 
+
   // Generate the response buffer
   [theBuffer appendString: @"<Collection>"];
   
diff --git a/ActiveSync/SOGoActiveSyncDispatcher.m b/ActiveSync/SOGoActiveSyncDispatcher.m
index 40cefbf..48d5c53 100644
--- a/ActiveSync/SOGoActiveSyncDispatcher.m
+++ b/ActiveSync/SOGoActiveSyncDispatcher.m
@@ -129,7 +129,6 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   [super init];
 
   folderTableURL = nil;
-  
   return self;
 }
 
@@ -165,6 +164,39 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   return [o properties];
 }
 
+- (id) _translateGuidToImapFolderName: (NSString *) theIdToTranslate  type: (SOGoMicrosoftActiveSyncFolderType) theFolderType
+{
+
+  SOGoMailAccounts *accountsFolder;
+  SOGoMailAccount *accountFolder;
+  SOGoUserFolder *userFolder;
+
+  switch (theFolderType)
+    {
+    case ActiveSyncMailFolder:
+    {
+
+  NSArray *allFoldersMetadata;
+
+  userFolder = [[context activeUser] homeFolderInContext: context];
+  accountsFolder = [userFolder lookupName: @"Mail" inContext: context acquire: NO];
+  accountFolder = [accountsFolder lookupName: @"0" inContext: context acquire: NO];
+
+  // get guids of folder (imap)
+  NSDictionary *imapGuids = [accountFolder getImapFolderGuids];
+  return [[imapGuids allKeysForObject: theIdToTranslate] objectAtIndex: 0];
+}
+break;
+  default: 
+  {
+  return theIdToTranslate;
+}
+  }
+ 
+
+}
+
+
 //
 //
 //
@@ -199,6 +231,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
         accountsFolder = [userFolder lookupName: @"Mail"  inContext: context  acquire: NO];
         currentFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
         
+        SOGoMailAccount *accountFolder;
+        accountFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
+        
         collection = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@", theCollectionId]
                                      inContext: context
                                        acquire: NO];
@@ -250,12 +285,13 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
           newFolder = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@", [displayName stringByEncodingImap4FolderName]]
                                       inContext: context
                                         acquire: NO];
-        else
-          newFolder = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@/%@", [[parentId stringByUnescapingURL] substringFromIndex: 5],
+        else {
+          parentId = [self _translateGuidToImapFolderName: [[parentId stringByUnescapingURL] substringFromIndex: 5] type:ActiveSyncMailFolder];
+          newFolder = [currentFolder lookupName: [NSString stringWithFormat: @"folder%@/%@", [parentId stringByEncodingImap4FolderName],
                                                            [displayName stringByEncodingImap4FolderName]]
                                       inContext: context
                                         acquire: NO];
-
+        }
         // FIXME
         // handle exists (status == 2)
         // handle right synckey
@@ -265,6 +301,24 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
             
             // We strip the "folder" prefix
             nameInContainer = [nameInContainer substringFromIndex: 6];
+
+            // save new guid into cache
+            SOGoMailAccount *accountFolder;
+            accountFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
+
+            // update GUID in cache
+            NSDictionary *imapGuids = [accountFolder getImapFolderGuids];
+
+            NSString *key = [NSString stringWithFormat: @"%@+folder%@", [context objectForKey: @"DeviceId"], nameInContainer ];
+            SOGoCacheGCSObject *o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+            [o setObjectType: ActiveSyncFolderCacheObject];
+            [o setTableUrl: [self folderTableURL]];
+            [o reloadIfNeeded];
+            nameInContainer =[imapGuids objectForKey: nameInContainer];
+
+            [[o properties ]  setObject: nameInContainer   forKey: @"GUID"];
+            [o save];
+
             nameInContainer = [[NSString stringWithFormat: @"mail/%@", nameInContainer] stringByEscapingURL];
           }
         else
@@ -348,6 +402,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   
   serverId = [[[(id)[theDocumentElement getElementsByTagName: @"ServerId"] lastObject] textValue] realCollectionIdWithFolderType: &folderType];
+  serverId = [self _translateGuidToImapFolderName: serverId type:folderType];
 
   userFolder = [[context activeUser] homeFolderInContext: context];
   accountsFolder = [userFolder lookupName: @"Mail"  inContext: context  acquire: NO];
@@ -408,6 +463,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   int status;
   
   serverId = [[[(id)[theDocumentElement getElementsByTagName: @"ServerId"] lastObject] textValue] realCollectionIdWithFolderType: &folderType];
+  serverId = [self _translateGuidToImapFolderName: serverId type:folderType];
   parentId = [[(id)[theDocumentElement getElementsByTagName: @"ParentId"] lastObject] textValue];
   displayName = [[(id)[theDocumentElement getElementsByTagName: @"DisplayName"] lastObject] textValue];
 
@@ -427,7 +483,8 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     }
   else
     {
-      error = [folderToUpdate renameTo: [NSString stringWithFormat: @"%@/%@", [[parentId stringByUnescapingURL] substringFromIndex: 5],
+      parentId = [self _translateGuidToImapFolderName: [[parentId stringByUnescapingURL] substringFromIndex: 5] type:folderType];
+      error = [folderToUpdate renameTo: [NSString stringWithFormat: @"%@/%@", [parentId stringByEncodingImap4FolderName],
                                                   [displayName stringByEncodingImap4FolderName]]];
     }
 
@@ -445,7 +502,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
       // See http://msdn.microsoft.com/en-us/library/gg675615(v=exchg.80).aspx
       // we return '9' - we force a FolderSync
-      status = 9;
+      status = 1;
 
       [self _setFolderSyncKey: syncKey];
 
@@ -481,12 +538,13 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 - (void) processFolderSync: (id <DOMElement>) theDocumentElement
                 inResponse: (WOResponse *) theResponse
 {
+
   NSMutableDictionary *metadata;
   NSMutableString *s;
   NSString *syncKey;
   NSData *d;
-  
-  BOOL first_sync;
+ 
+  BOOL first_sync, found;
   int status;
 
   metadata = [self _globalMetadataForDevice];
@@ -512,10 +570,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   [s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
   [s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\">"];
   [s appendFormat: @"<FolderSync xmlns=\"FolderHierarchy:\"><Status>%d</Status><SyncKey>%@</SyncKey><Changes>", status, syncKey];
-  
-  // Initial sync, let's return the complete folder list
-  if (first_sync)
+ 
+  if (status = 1)
     {
+
       SOGoMailAccounts *accountsFolder;
       SOGoMailAccount *accountFolder;
       SOGoUserFolder *userFolder;
@@ -525,21 +583,86 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
       NSArray *allFoldersMetadata;
       NSString *name, *serverId, *parentId;
 
-      int i, type;
-      
+      int i, type, command_count;
+      NSMutableString *commands;
+
+      SOGoCacheGCSObject *o;
+      NSString *key, *nkey;
+
       userFolder = [[context activeUser] homeFolderInContext: context];
-      accountsFolder = [userFolder lookupName: @"Mail"  inContext: context  acquire: NO];
-      accountFolder = [accountsFolder lookupName: @"0"  inContext: context  acquire: NO];
+      accountsFolder = [userFolder lookupName: @"Mail" inContext: context acquire: NO];
+      accountFolder = [accountsFolder lookupName: @"0" inContext: context acquire: NO];
 
       allFoldersMetadata = [accountFolder allFoldersMetadata];
+  
+      // get guids of folder (imap)
+      // e.g. {INBOX = "sogo_73c_192bd57b_d8"
+      NSDictionary *imapGuids = [accountFolder getImapFolderGuids];
+
+      NSMutableDictionary *cacheGuids = [[NSMutableDictionary alloc] init];
+      NSArray *foldersInCache = [[NSArray alloc] init];
+
+      // no need to read cached folder infos during first sync
+      // e.g. {"sogo_73c_192bd57b_d8" = INBOX} - guid = foldername for easy reverse lookup with imapGuids
+      if (!first_sync) {
+         // get the list of folder stored in cache 
+         key = [NSString stringWithFormat: @"%@+%@", [context objectForKey: @"DeviceId"], @"0"];
+         o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+         [o setObjectType: ActiveSyncFolderCacheObject];
+         [o setTableUrl: [self folderTableURL]];
+         [o reloadIfNeeded];
+         foldersInCache = [o folderList: [context objectForKey: @"DeviceId"] newerThanVersion: -1];
+
+         // get guids of folders stored in cache
+         for (NSString *folderName in foldersInCache) {
+             key = [folderName substringFromIndex: 1];
+   
+             o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+             [o setObjectType: ActiveSyncFolderCacheObject];
+             [o setTableUrl: [self folderTableURL]];
+             [o reloadIfNeeded];
+
+             if ([[o properties ]  objectForKey: @"GUID"])
+                [cacheGuids setObject:  [key substringFromIndex:[key rangeOfString: @"+"].location+7] forKey: [[o properties ]  objectForKey: @"GUID"]];
+         }
+      }
+
+      // handle folders deleted in imap
+      command_count=0;
+      commands = [NSMutableString string];
 
-      // See 2.2.3.170.3 Type (FolderSync) - http://msdn.microsoft.com/en-us/library/gg650877(v=exchg.80).aspx
-      [s appendFormat: @"<Count>%d</Count>", [allFoldersMetadata count]+3];
+      for (NSString *cKey in cacheGuids)
+      {
 
+         if(![imapGuids allKeysForObject: cKey]) {
+            // delete folders cache content to avoid stale data if a new folder gets created with the same name
+            key =  [NSString stringWithFormat: @"%@+folder%@", [context objectForKey: @"DeviceId"], [cacheGuids objectForKey: cKey]];
+            o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+            [o setObjectType: ActiveSyncFolderCacheObject];
+            [o setTableUrl: [self folderTableURL]];
+            [o reloadIfNeeded];
+
+            // only send a delete if GUID is found
+            if ([[o properties ]  objectForKey: @"GUID"])  {
+                [commands appendFormat: @"<Delete><ServerId>%@</ServerId></Delete>", [[NSString stringWithFormat: @"mail/%@", [[o properties ]  objectForKey: @"GUID"]] stringByEscapingURL] ];
+                command_count++; 
+	    }
+
+            [[o properties] removeAllObjects];
+            [o save];
+         }
+      }
+
+      // handle addition and changes
       for (i = 0; i < [allFoldersMetadata count]; i++)
         {
           folderMetadata = [allFoldersMetadata objectAtIndex: i];
-          serverId = [NSString stringWithFormat: @"mail%@", [folderMetadata objectForKey: @"path"]];
+          // no guid - no sync
+          if (!([imapGuids objectForKey: [[folderMetadata objectForKey: @"path"] substringFromIndex: 1]])) {
+             continue;
+          }
+
+          serverId = [NSString stringWithFormat: @"mail/%@", [imapGuids objectForKey: [[folderMetadata objectForKey: @"path"] substringFromIndex: 1] ]];
           name = [folderMetadata objectForKey: @"displayName"];
           
           if ([name hasPrefix: @"/"])
@@ -547,24 +670,75 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
           
           if ([name hasSuffix: @"/"])
             name = [name substringToIndex: [name length]-1];
-
+  
           type = [[folderMetadata objectForKey: @"type"] activeSyncFolderType];
 
           parentId = @"0";
 
           if ([folderMetadata objectForKey: @"parent"])
             {
-              parentId = [NSString stringWithFormat: @"mail%@", [folderMetadata objectForKey: @"parent"]];
+              parentId = [NSString stringWithFormat: @"mail/%@", [imapGuids objectForKey: [[folderMetadata objectForKey: @"parent"] substringFromIndex: 1] ]];
               name = [[name pathComponents] lastObject];
             }
 
-          [s appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>",
-             [serverId stringByEscapingURL],
-             [parentId stringByEscapingURL],
-             type,
-             [name activeSyncRepresentationInContext: context]];
-        }
-
+          // decide between add and change
+          if ([cacheGuids objectForKey: [imapGuids objectForKey: [[folderMetadata objectForKey: @"path"] substringFromIndex: 1]]])  {
+             // search guid to check name change in cache (diff between imap and cache)
+             if ((![ [[folderMetadata objectForKey: @"path"] substringFromIndex: 1] isEqualToString: [imapGuids objectForKey: [cacheGuids objectForKey: 
+                                         [[folderMetadata objectForKey: @"path"] substringFromIndex: 1]]]] )) {
+                 key =  [NSString stringWithFormat: @"%@+folder%@", [context objectForKey: @"DeviceId"], [cacheGuids objectForKey: 
+                             [imapGuids objectForKey: [[folderMetadata objectForKey: @"path"] substringFromIndex: 1]] ]];
+                 nkey = [NSString stringWithFormat: @"%@+folder%@", [context objectForKey: @"DeviceId"], [[folderMetadata objectForKey: @"path"] substringFromIndex: 1] ];
+                if (![key isEqualToString: nkey ] ) {
+                   [commands appendFormat: @"<Update><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Update>",
+                         [serverId stringByEscapingURL],
+                         [parentId stringByEscapingURL],
+                         type,
+                         [name activeSyncRepresentationInContext: context]];
+
+
+                    // change path in cache
+                    o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+                    [o setObjectType: ActiveSyncFolderCacheObject];
+                    [o setTableUrl: [self folderTableURL]];
+                    [o reloadIfNeeded];
+                    [o changePathTo: [NSString stringWithFormat: @"/%@", nkey]]; // ?? why is '/' prefix needed - problem in changePathTo?
+
+                    command_count++;
+                }
+             }
+          }
+          else {
+                    [commands appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>",
+                         [serverId stringByEscapingURL],
+                         [parentId stringByEscapingURL],
+                         type,
+                        [name activeSyncRepresentationInContext: context]];
+
+                    // store folder's guid in cache
+                    key = [NSString stringWithFormat: @"%@+folder%@", [context objectForKey: @"DeviceId"], [[folderMetadata objectForKey: @"path"] substringFromIndex: 1] ];
+                    o = [SOGoCacheGCSObject objectWithName: key inContainer: nil];
+                    [o setObjectType: ActiveSyncFolderCacheObject];
+                    [o setTableUrl: [self folderTableURL]];
+                    [o reloadIfNeeded];
+
+                    [[o properties ]  setObject: [imapGuids objectForKey: [[folderMetadata objectForKey: @"path"] substringFromIndex: 1]]   forKey: @"GUID"];
+                    [o save];
+
+                    command_count++;
+               }
+       }
+
+       if (first_sync)
+             [s appendFormat: @"<Count>%d</Count>", command_count+3];
+       else
+             [s appendFormat: @"<Count>%d</Count>", command_count];
+
+       if (command_count > 0)
+         [s appendFormat: @"%@", commands];
+
+     if (first_sync)
+     {
       // We add the personal calendar - events
       // FIXME: add all calendars
       currentFolder = [[context activeUser] personalCalendarFolderInContext: context];
@@ -582,14 +756,16 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
       currentFolder = [[context activeUser] personalContactsFolderInContext: context];
       name = [NSString stringWithFormat: @"vcard/%@", [currentFolder nameInContainer]];
       [s appendFormat: @"<Add><ServerId>%@</ServerId><ParentId>%@</ParentId><Type>%d</Type><DisplayName>%@</DisplayName></Add>", name, @"0", 9, [[currentFolder displayName] activeSyncRepresentationInContext: context]];
-    }
+     }
+  }
 
   [s appendString: @"</Changes></FolderSync>"];
 
   d = [[s dataUsingEncoding: NSUTF8StringEncoding] xml2wbxml];
 
   [theResponse setContent: d];
-}
+
+} 
 
 //
 // From: http://msdn.microsoft.com/en-us/library/ee157980(v=exchg.80).aspx :
@@ -636,6 +812,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
   collectionId = [[(id)[theDocumentElement getElementsByTagName: @"CollectionId"] lastObject] textValue];
   realCollectionId = [collectionId realCollectionIdWithFolderType: &folderType];
+  realCollectionId = [self _translateGuidToImapFolderName: realCollectionId type:folderType];
   currentCollection = [self collectionFromId: realCollectionId  type: folderType];
   
   //
@@ -710,6 +887,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   fileReference = [[[(id)[theDocumentElement getElementsByTagName: @"FileReference"] lastObject] textValue] stringByUnescapingURL];
 
   realCollectionId = [fileReference realCollectionIdWithFolderType: &folderType];
+  realCollectionId = [self _translateGuidToImapFolderName: realCollectionId type:folderType];
   
   if (folderType == ActiveSyncMailFolder)
     {
@@ -803,6 +981,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   status = 1;
 
   realCollectionId = [[[(id)[theDocumentElement getElementsByTagName: @"CollectionId"] lastObject] textValue] realCollectionIdWithFolderType: &folderType];
+  realCollectionId = [self _translateGuidToImapFolderName: realCollectionId type:folderType];
   userResponse = [[[(id)[theDocumentElement getElementsByTagName: @"UserResponse"] lastObject] textValue] intValue];
   requestId = [[(id)[theDocumentElement getElementsByTagName: @"RequestId"] lastObject] textValue];  
   appointmentObject = nil;
@@ -953,6 +1132,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
           NSDictionary *response;
           NSString *v;
           
+  srcFolderId = [self _translateGuidToImapFolderName: srcFolderId type:srcFolderType];
+  dstFolderId = [self _translateGuidToImapFolderName: dstFolderId type:dstFolderType];
+
           currentCollection = [self collectionFromId: srcFolderId  type: srcFolderType];
           
           client = [[currentCollection imap4Connection] client];
@@ -1476,6 +1658,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   folderId = [[(id)[theDocumentElement getElementsByTagName: @"FolderId"] lastObject] textValue];
   itemId = [[(id)[theDocumentElement getElementsByTagName: @"ItemId"] lastObject] textValue];
   realCollectionId = [folderId realCollectionIdWithFolderType: &folderType];
+  realCollectionId = [self _translateGuidToImapFolderName: realCollectionId type:folderType];
 
   value = [theDocumentElement getElementsByTagName: @"ReplaceMime"];
 
@@ -1705,10 +1888,6 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   if (!folderTableURL)
     {
       user = [context activeUser];
-
-      if (![user loginInDomain])
-        return nil;
-
       urlString = [[user domainDefaults] folderInfoURL];
       parts = [[urlString componentsSeparatedByString: @"/"]
                 mutableCopy];
@@ -1718,7 +1897,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
           /* If "OCSFolderInfoURL" is properly configured, we must have 5
              parts in this url. */
           ocFSTableName = [NSString stringWithFormat: @"sogo_cache_folder_%@",
-                                    [[user loginInDomain] asCSSIdentifier]];
+                                    [[[context activeUser] loginInDomain] asCSSIdentifier]];
           [parts replaceObjectAtIndex: 4 withObject: ocFSTableName];
           folderTableURL
             = [NSURL URLWithString: [parts componentsJoinedByString: @"/"]];
@@ -1746,10 +1925,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
   
   /* FIXME: make use of [EOChannelAdaptor describeTableNames] instead */
   tableName = [[folderTableURL path] lastPathComponent];
-  if (tableName &&
-      [channel evaluateExpressionX:
-                 [NSString stringWithFormat: @"SELECT count(*) FROM %@",
-                           tableName]])
+  if ([channel evaluateExpressionX:
+        [NSString stringWithFormat: @"SELECT count(*) FROM %@",
+                  tableName]])
     {
       queries = [channel specialQueries];
       query = [queries createSOGoCacheGCSFolderTableWithName: tableName];
diff --git a/SoObjects/Mailer/SOGoMailAccount.h b/SoObjects/Mailer/SOGoMailAccount.h
index 2efc05f..37f002c 100644
--- a/SoObjects/Mailer/SOGoMailAccount.h
+++ b/SoObjects/Mailer/SOGoMailAccount.h
@@ -84,6 +84,8 @@ typedef enum {
 - (NSArray *) allFolderPaths;
 - (NSArray *) allFoldersMetadata;
 
+- (NSDictionary *) getImapFolderGuids;
+
 - (BOOL) isInDraftsFolder;
 
 /* special folders */
diff --git a/SoObjects/Mailer/SOGoMailAccount.m b/SoObjects/Mailer/SOGoMailAccount.m
index 5f9a67c..d56ffef 100644
--- a/SoObjects/Mailer/SOGoMailAccount.m
+++ b/SoObjects/Mailer/SOGoMailAccount.m
@@ -60,6 +60,8 @@
 #import "SOGoUser+Mailer.h"
 
 #import "SOGoMailAccount.h"
+#import <Foundation/NSProcessInfo.h>
+
 
 #define XMLNS_INVERSEDAV @"urn:inverse:params:xml:ns:inverse-dav"
 
@@ -659,6 +661,61 @@ static NSString *inboxFolderName = @"INBOX";
   return password;
 }
 
+
+- (NSDictionary *) getImapFolderGuids
+{
+  NSDictionary *result, *nresult, *folderData;
+  NGImap4Client *client;
+  NSMutableDictionary *folders;
+  NSArray *folderList;
+
+  SOGoUserDefaults *ud;
+  BOOL subscribedOnly;
+
+  ud = [[context activeUser] userDefaults];
+  subscribedOnly = [ud mailShowSubscribedFoldersOnly];
+
+  folderList = [[self imap4Connection] allFoldersForURL: [self imap4URL]
+                               onlySubscribedFolders: subscribedOnly];
+
+  folders = [[NSMutableDictionary alloc] init];
+
+  client = [[self imap4Connection] client];
+  result = [client annotation: @"*"  entryName: @"/comment" attributeName: @"value.priv"];
+
+  if (![[result objectForKey: @"result"] boolValue]) {
+     NSLog(@"tfu getannotation failed: x %@", [result objectForKey: @"ResponseResult"]);
+  }
+
+  NSEnumerator *e = [folderList objectEnumerator];
+  id object;
+  NSString *guid;
+  while (object = [e nextObject]) {
+
+   if (!(guid = [[[[result objectForKey: @"FolderList"] objectForKey: [object substringFromIndex: 1]] objectForKey: @"/comment"] objectForKey: @"value.priv"])) {
+            guid = [[NSProcessInfo processInfo] globallyUniqueString];
+            nresult = [client annotation: [object substringFromIndex: 1] entryName: @"/comment" attributeName: @"value.priv" attributeValue: guid ];
+            if (![[nresult objectForKey: @"result"] boolValue]) {
+               // need to implement X-GUID query for dovecot - this requires modification in sope to support following command:
+               // 1 list "" "*" return (status (x-guid)) -> this would avoid firinig a command per folder to imap
+               //guid = [NSString stringWithFormat: @"%@-%@", [object substringFromIndex: 1],  @"xxx" ];
+               nresult = [client status: [object substringFromIndex: 1] flags: [NSArray arrayWithObject: @"x-guid"]];
+               if (!(guid = [nresult objectForKey: @"x-guid"])) {
+                   guid = [NSString stringWithFormat: @"%@", [object substringFromIndex: 1]];
+               }
+               NSLog(@"tfu setannotation failed: %@", nresult );
+               NSLog(@"tfu uniqueid folderid: %@", guid );
+            }
+
+   }
+
+   [folders setObject: guid forKey: [object substringFromIndex: 1]];
+  }
+
+  return folders;
+}
+
+
 /* name lookup */
 
 - (id) lookupName: (NSString *) _key
diff --git a/SoObjects/SOGo/SOGoCacheGCSObject.h b/SoObjects/SOGo/SOGoCacheGCSObject.h
index 8d26a49..3b2a8d4 100644
--- a/SoObjects/SOGo/SOGoCacheGCSObject.h
+++ b/SoObjects/SOGo/SOGoCacheGCSObject.h
@@ -67,6 +67,9 @@ typedef enum {
 - (NSDictionary *) lookupRecord: (NSString *) path
                newerThanVersion: (NSInteger) startVersion;
 
+- (NSArray *) folderList: (NSString *) deviceId
+               newerThanVersion: (NSInteger) startVersion;
+
 - (void) setObjectType: (SOGoCacheObjectType) newObjectType;
 - (SOGoCacheObjectType) objectType; /* message, fai, folder */
 
diff --git a/SoObjects/SOGo/SOGoCacheGCSObject.m b/SoObjects/SOGo/SOGoCacheGCSObject.m
index bb2e690..232ba82 100644
--- a/SoObjects/SOGo/SOGoCacheGCSObject.m
+++ b/SoObjects/SOGo/SOGoCacheGCSObject.m
@@ -375,6 +375,47 @@ static EOAttribute *textColumn = nil;
   return record;
 }
 
+// get a list of all folders
+- (NSArray *) folderList: (NSString *) deviceId
+               newerThanVersion: (NSInteger) startVersion
+{
+  NSMutableArray *recordsOut;
+  NSArray *records;
+  NSString *tableName, *pathValue;
+  NSMutableString *sql;
+  EOAdaptor *adaptor;
+  NSUInteger count, max;
+
+  if ([deviceId hasSuffix: @"/"])
+    [NSException raise: @"MAPIStoreIOException"
+                format: @"path ends with a slash: %@", deviceId];
+
+  tableName = [self tableName];
+  adaptor = [self tableChannelAdaptor];
+  pathValue = [adaptor formatValue: [NSString stringWithFormat: @"/%@+folder%", deviceId]
+                      forAttribute: textColumn];
+
+  /* query */
+  sql = [NSMutableString stringWithFormat:
+                           @"SELECT * FROM %@ WHERE c_path like %@ and c_deleted <> 1",
+                         tableName, pathValue];
+  if (startVersion > -1)
+    [sql appendFormat: @" AND c_version > %d", startVersion];
+
+  /* execution */
+  records = [self performSQLQuery: sql];
+
+  max = [records count];
+  recordsOut = [[NSMutableArray alloc] init];
+  for (count = 0; count < max; count++)
+    {
+      [recordsOut addObject: [[records objectAtIndex: count] objectForKey: @"c_path"]];
+    }
+
+  return recordsOut;
+}
+
+
 - (void) reloadIfNeeded
 {
   /* if object is uninitialized: reload without condition, otherwise, load if
-- 
1.7.9.5

0001-foldersync.patch (31,951 bytes)   
tfu

tfu

2014-05-26 17:41

reporter  

0001-annotation.patch (11,995 bytes)   
From 803bf3b2a07681fc0fea8af98fdf8437095a0c8d Mon Sep 17 00:00:00 2001
From: root <root@example.com>
Date: Sun, 25 May 2014 23:56:59 +0200
Subject: [PATCH] annotation

---
 sope-mime/NGImap4/NGImap4Client.h             |    2 +
 sope-mime/NGImap4/NGImap4Client.m             |   78 +++++++++++++++++++++++-
 sope-mime/NGImap4/NGImap4ResponseNormalizer.m |    7 +++
 sope-mime/NGImap4/NGImap4ResponseParser.m     |   81 +++++++++++++++++++++++--
 4 files changed, 162 insertions(+), 6 deletions(-)

diff --git a/sope-mime/NGImap4/NGImap4Client.h b/sope-mime/NGImap4/NGImap4Client.h
index 8442062..a5c3d17 100644
--- a/sope-mime/NGImap4/NGImap4Client.h
+++ b/sope-mime/NGImap4/NGImap4Client.h
@@ -136,6 +136,8 @@ typedef enum {
 - (NSDictionary *)select:(NSString *)_folder;
 - (NSDictionary *)unselect;
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute;
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute  attributeValue:(NSString *)_value;
 - (NSDictionary *)rename:(NSString *)_folder to:(NSString *)_newName;
 - (NSDictionary *)delete:(NSString *)_folder;
 - (NSDictionary *)create:(NSString *)_name;
diff --git a/sope-mime/NGImap4/NGImap4Client.m b/sope-mime/NGImap4/NGImap4Client.m
index 02af32b..a614515 100644
--- a/sope-mime/NGImap4/NGImap4Client.m
+++ b/sope-mime/NGImap4/NGImap4Client.m
@@ -118,7 +118,7 @@ static int          ProfileImapEnabled = -1;
 static int          LogImapEnabled     = -1;
 static int          PreventExceptions  = -1;
 static BOOL         fetchDebug         = NO;
-static BOOL         ImapDebugEnabled   = NO;
+static BOOL         ImapDebugEnabled   = YES;
 static NSArray      *Imap4SystemFlags  = nil;
 
 static NSMutableDictionary *capabilities;
@@ -908,6 +908,79 @@ static NSMutableDictionary *namespaces;
   return [self->normer normalizeResponse:[self processCommand:@"unselect"]];
 }
 
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute {
+/*
+ result dict looks like the following  
+{FolderList = {INBOX = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_d8"; }; }; "Other Users/sogo2" = {"/comment" = {"value.priv" = "sogo_c0c_192bd7dc_0"; }; }; Sent = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_da"; }; }; Trash = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_dc"; }; }; abczzll = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_d9"; }; }; "abczzll/mmabcmm" = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_de"; }; }; mf1renamedd = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_dd"; }; }; tfu1 = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_db"; }; }; zuk = {"/comment" = {"value.priv" = "sogo_73c_192bd57b_df"; }; }; }; RawResponse = "{FolderList = ({INBOX = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_d8\"; }; }; }, {Sent = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_da\"; }; }; }, {Trash = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_dc\"; }; }; }, {abczzll = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_d9\"; }; }; }, {\"abczzll/mmabcmm\" = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_de\"; }; }; }, {mf1renamedd = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_dd\"; }; }; }, {tfu1 = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_db\"; }; }; }, {zuk = {\"/comment\" = {\"value.priv\" = \"sogo_73c_192bd57b_df\"; }; }; }, {\"Other Users/sogo2\" = {\"/comment\" = {\"value.priv\" = \"sogo_c0c_192bd7dc_0\"; }; }; }); ResponseResult = {description = Completed; result = ok; tagId = 13; }; }"; expunge = (); result = 1; }
+
+ getannotation:
+
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value.priv"];
+ result = [client annotation: folderName entryName: @"/comment" attributeName: @"value"];
+ result = [client annotation: folderName entryName: @"/*" attributeName: @"value"];
+ result = [client annotation: @"" entryName: @"/*" attributeName: @"value"];
+
+*/
+  NSString *cmd;
+  NGHashMap *_map;
+  NSDictionary        *obj;
+  NSMutableDictionary *result, *folderList;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"getannotation \"%@\" \"%@\" \"%@\"",
+                      SaneFolderName(_folder), _entry, _attribute];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  _map = [self processCommand:cmd];
+
+  result = [self->normer normalizeResponse:_map];
+
+  NSEnumerator        *enumerator;
+  enumerator = [_map objectEnumeratorForKey:@"FolderList"];
+  folderList  = [NSMutableDictionary dictionaryWithCapacity:5];
+  while ((obj = [enumerator nextObject]) != nil) {
+
+    [folderList addEntriesFromDictionary:[NSDictionary dictionaryWithDictionary:obj]];
+  }
+
+  [result setObject: folderList forKey: @"FolderList" ];
+
+  return result;
+
+}
+
+- (NSDictionary *)annotation:(NSString *)_folder entryName:(NSString *)_entry attributeName:(NSString *)_attribute attributeValue:(NSString *)_value {
+  NSString *cmd;
+  NSMutableDictionary *result;
+  
+  if (_folder == nil)
+    return nil;
+  if ((_entry == nil))
+    return nil;
+  if ((_attribute == nil))
+    return nil;
+  if ((_value == nil))
+    return nil;
+  if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
+    return nil;
+  
+  cmd     = [NSString stringWithFormat:@"setannotation \"%@\" \"%@\" (\"%@\" \"%@\")",
+                      SaneFolderName(_folder), _entry, _attribute, _value];
+  
+  result  = [NSMutableDictionary dictionaryWithCapacity:2];
+  result = [self->normer normalizeResponse:[self processCommand:cmd]];
+  return result;
+
+}
+
 - (NSDictionary *)status:(NSString *)_folder flags:(NSArray *)_flags {
   NSString *cmd;
   
@@ -918,7 +991,7 @@ static NSMutableDictionary *namespaces;
   if ((_folder = [self _folder2ImapFolder:_folder]) == nil)
     return nil;
   
-  cmd     = [NSString stringWithFormat:@"status \"%@\" (%@)",
+  cmd = [NSString stringWithFormat:@"status \"%@\" (%@)",
                       SaneFolderName(_folder), [_flags componentsJoinedByString:@" "]];
   return [self->normer normalizeStatusResponse:[self processCommand:cmd]];
 }
@@ -988,6 +1061,7 @@ static NSMutableDictionary *namespaces;
   return [_parts componentsJoinedByString:@" "];
 }
 
+
 - (NSDictionary *)fetchUids:(NSArray *)_uids parts:(NSArray *)_parts {
   /*
     eg: 'UID FETCH 1189,1325,1326 ([TODO])'
diff --git a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
index 687c3dc..8e0f0be 100644
--- a/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
+++ b/sope-mime/NGImap4/NGImap4ResponseNormalizer.m
@@ -268,6 +268,8 @@ static int      LogImapEnabled = -1;
         [result setObject:o forKey:@"highestmodseq"];
       else if ((o = [obj objectForKey:@"UIDNEXT"]))
         [result setObject:o forKey:@"uidnext"];
+      else if ((o = [obj objectForKey:@"UIDVALIDITY"]))
+        [result setObject:o forKey:@"uidvalidity"];
     }
     else
       [self warnWithFormat:@"unexpected OK object: %@", obj];
@@ -293,6 +295,7 @@ static int      LogImapEnabled = -1;
   return result;
 }
 
+
 - (NSDictionary *)normalizeStatusResponse:(NGHashMap *)_map {
   /*
     filter for status response
@@ -318,6 +321,10 @@ static int      LogImapEnabled = -1;
   }
   if ((o = [obj  objectForKey:@"unseen"]) != nil)
     [result setObject:o forKey:@"unseen"];
+
+  // support x-guid (dovecot)
+  if ((o = [obj  objectForKey:@"x-guid"]) != nil)
+    [result setObject:o forKey:@"x-guid"];
   
   return result;
 }
diff --git a/sope-mime/NGImap4/NGImap4ResponseParser.m b/sope-mime/NGImap4/NGImap4ResponseParser.m
index f14c889..0fcfe98 100644
--- a/sope-mime/NGImap4/NGImap4ResponseParser.m
+++ b/sope-mime/NGImap4/NGImap4ResponseParser.m
@@ -11,7 +11,7 @@
   SOPE is distributed in the hope that it will be useful, but WITHOUT ANY
   WARRANTY; without even the implied warranty of MERCHANTABILITY or
   FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public
-  License for more details.
+/bin/bash: 58: command not found
 
   You should have received a copy of the GNU Lesser General Public
   License along with SOPE; see the file COPYING.  If not, write to the
@@ -48,6 +48,7 @@
 - (BOOL)_parseVanishedResponseIntoHashMap:(NGMutableHashMap *)result_;
 - (BOOL)_parseByeUntaggedResponseIntoHashMap:(NGMutableHashMap *)result_;
 - (BOOL)_parseACLResponseIntoHashMap:(NGMutableHashMap *)result_;
+- (BOOL)_parseAnnotationResponseIntoHashMap:(NGMutableHashMap *)result_;
 - (BOOL)_parseMyRightsResponseIntoHashMap:(NGMutableHashMap *)result_;
 - (BOOL)_parseListRightsResponseIntoHashMap:(NGMutableHashMap *)result_;
 
@@ -604,7 +605,10 @@ static void _parseUntaggedResponse(NGImap4ResponseParser *self,
   l0 = _la(self, 0);
   switch (l0) {
   case 'A':
-    if ([self _parseACLResponseIntoHashMap:result_])
+    l1 = _la(self, 1);
+    if (l1 == 'C' && [self _parseACLResponseIntoHashMap:result_])
+      return;
+    if (l1 == 'N' && [self _parseAnnotationResponseIntoHashMap:result_]) 
       return;
     break;
     
@@ -1407,7 +1411,7 @@ _purifyQuotedString(NSMutableString *quotedString) {
 - (BOOL)_parseStatusResponseIntoHashMap:(NGMutableHashMap *)result_ {
   NSString            *name  = nil;
   NSMutableDictionary *flags = nil;
-  NSDictionary *d;
+  NSDictionary *d, *f;
     
   if (!_matchesString(self, "STATUS "))
     return NO;
@@ -1437,8 +1441,12 @@ _purifyQuotedString(NSMutableString *quotedString) {
     if (_la(self, 0) == ' ')
       _consume(self, 1);
       
+    if ([[key lowercaseString] isEqualToString:@"x-guid"])
+    [flags setObject:value
+          forKey:[key lowercaseString]];
+    else
     [flags setObject:[NumClass numberWithInt:[value intValue]]
-	   forKey:[key lowercaseString]];
+          forKey:[key lowercaseString]];
   }
   _consumeIfMatch(self, ')');
   _parseUntil(self, '\n');
@@ -1451,6 +1459,71 @@ _purifyQuotedString(NSMutableString *quotedString) {
   return YES;
 }
 
+- (BOOL)_parseAnnotationResponseIntoHashMap:(NGMutableHashMap *)result_ {
+  NSString            *name  = nil;
+  NSString            *entry  = nil;
+  NSMutableDictionary *attributes = nil;
+  NSDictionary *d, *f;
+
+  if (!_matchesString(self, "ANNOTATION "))
+    return NO;
+
+  _consume(self, 11);
+
+  if (_la(self, 0) == '"') {
+    name = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    name = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    name = _parseUntil(self, ' ');
+  }
+
+  if (_la(self, 0) == '"') {
+    entry = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+  }
+  else if (_la(self, 0) == '{') {
+    entry = [self _parseQuotedStringOrNIL];
+    _consumeIfMatch(self, ' ');
+  }
+  else {
+    entry = _parseUntil(self, ' ');
+  }
+
+  _consumeIfMatch(self, '(');
+
+  attributes = [NSMutableDictionary dictionaryWithCapacity:2];
+  d = [NSMutableDictionary dictionaryWithCapacity:2];
+  f = [NSMutableDictionary dictionaryWithCapacity:2];
+
+  while (_la(self, 0) != ')') {
+    NSString *key   = [self _parseQuotedString];
+    _consumeIfMatch(self, ' ');
+    NSString *value = [self _parseQuotedString];
+
+    if (_la(self, 0) == ' ')
+      _consume(self, 1);
+
+    [attributes setObject:value
+           forKey:[key lowercaseString]];
+
+    [d setObject:[NSDictionary dictionaryWithDictionary:attributes]
+       forKey:entry];
+  }
+  _consumeIfMatch(self, ')');
+  _parseUntil(self, '\n');
+
+  [f setObject:d forKey:name];
+  [result_ addObject:f forKey:@"FolderList"];
+
+  return YES;
+}
+
+
 - (BOOL)_parseByeUntaggedResponseIntoHashMap:(NGMutableHashMap *)result_ {
   NSString *reason;
   
-- 
1.7.9.5

0001-annotation.patch (11,995 bytes)   
tfu

tfu

2014-05-26 17:47

reporter   ~0007124

Just uploaded a new patch which impelments foldersync based on new caching code:
0001-foldersync.patch - sogo part
0001-annotation.patch - sope part

ludovic

ludovic

2014-05-27 16:02

administrator   ~0007128

The SOPE patch has just been pushed: https://github.com/inverse-inc/sope/commit/378d023f70f2134f2127a6f8fe72a88042af4c6d

ludovic

ludovic

2014-05-27 17:41

administrator   ~0007129

I'm reviewing the patches. This one will NOT be included as it reverts some fixes I did a few days ago:

@@ -1705,10 +1888,6 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
if (!folderTableURL)
{
user = [context activeUser];

  • if (![user loginInDomain])
  • return nil;
  • urlString = [[user domainDefaults] folderInfoURL];
    parts = [[urlString componentsSeparatedByString: @"/"]
    mutableCopy];
    @@ -1718,7 +1897,7 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
    / If "OCSFolderInfoURL" is properly configured, we must have 5
    parts in this url.
    /
    ocFSTableName = [NSString stringWithFormat: @"sogo_cachefolder%@",

  • [[user loginInDomain] asCSSIdentifier]];
  • [[[context activeUser] loginInDomain] asCSSIdentifier]];
    [parts replaceObjectAtIndex: 4 withObject: ocFSTableName];
    folderTableURL
    = [NSURL URLWithString: [parts componentsJoinedByString: @"/"]];
    @@ -1746,10 +1925,9 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    / FIXME: make use of [EOChannelAdaptor describeTableNames] instead /
    tableName = [[folderTableURL path] lastPathComponent];

  • if (tableName &&
  • [channel evaluateExpressionX:
  • [NSString stringWithFormat: @"SELECT count(*) FROM %@",
  • tableName]])
  • if ([channel evaluateExpressionX:
  • [NSString stringWithFormat: @"SELECT count(*) FROM %@",
  • tableName]])
    {
    queries = [channel specialQueries];
    query = [queries createSOGoCacheGCSFolderTableWithName: tableName];
ludovic

ludovic

2014-05-27 17:43

administrator   ~0007130

@@ -512,10 +570,10 @@ SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
[s appendString: @"<?xml version=\"1.0\" encoding=\"utf-8\"?>"];
[s appendString: @"<!DOCTYPE ActiveSync PUBLIC \"-//MICROSOFT//DTD ActiveSync//EN\" \"http://www.microsoft.com/\&quot;>&quot;];
[s appendFormat: @"<FolderSync xmlns=\"FolderHierarchy:\"><Status>%d</Status><SyncKey>%@</SyncKey><Changes>", status, syncKey];

  • // Initial sync, let's return the complete folder list
  • if (first_sync)
  • if (status = 1)

I guess yout meant a comparison, not an assignment.

tfu

tfu

2014-05-27 18:31

reporter   ~0007131

Yes - it should be: if (status == 1)

ludovic

ludovic

2014-05-27 18:47

administrator   ~0007132

Modified patch pushed: https://github.com/inverse-inc/sogo/commit/d35c52bb38e6bf0aa2f22e9466b66f1bf894b8e6

Thanks for your great work!

Issue History

Date Modified Username Field Change
2014-03-27 14:46 onofabio New Issue
2014-03-27 17:11 ludovic Note Added: 0006795
2014-03-27 17:11 ludovic Severity major => feature
2014-03-28 08:30 onofabio Note Added: 0006797
2014-03-28 18:22 ludovic Note Added: 0006801
2014-04-13 18:41 tfu File Added: 0001-folder-sync.patch
2014-04-13 18:41 tfu File Added: sope-0001-folder-sync-uidvalidity.patch
2014-04-13 18:54 tfu Note Added: 0006884
2014-04-13 21:36 ludovic Note Added: 0006885
2014-04-14 07:58 tfu Note Added: 0006886
2014-04-14 11:01 ludovic Note Added: 0006887
2014-04-14 19:17 tfu Note Added: 0006898
2014-04-22 19:03 ludovic Note Added: 0006950
2014-04-22 19:18 tfu Note Added: 0006952
2014-04-22 20:29 ludovic Note Added: 0006953
2014-04-28 20:34 tfu Note Added: 0006966
2014-04-28 20:34 tfu File Added: 0003-annotations.patch
2014-04-28 20:35 tfu File Added: 0001-uniqueid.patch
2014-04-28 20:36 tfu File Added: 0001-annotations.patch
2014-04-28 20:38 tfu Note Added: 0006967
2014-04-29 12:28 ludovic Note Added: 0006968
2014-04-29 13:58 tfu Note Added: 0006972
2014-05-05 17:14 ludovic Note Added: 0006989
2014-05-05 18:23 ludovic Target Version => 2.2.4
2014-05-15 19:12 ludovic Note Added: 0007028
2014-05-16 22:12 tfu Note Added: 0007035
2014-05-17 00:01 ludovic Note Added: 0007036
2014-05-22 13:38 ludovic Note Added: 0007067
2014-05-22 20:40 tfu Note Added: 0007081
2014-05-23 00:30 ludovic Note Added: 0007084
2014-05-26 17:40 tfu File Added: 0001-foldersync.patch
2014-05-26 17:41 tfu File Added: 0001-annotation.patch
2014-05-26 17:47 tfu Note Added: 0007124
2014-05-26 18:05 ludovic Relationship added related to 0001723
2014-05-26 18:05 ludovic Relationship added related to 0001739
2014-05-26 18:05 ludovic Relationship added related to 0001740
2014-05-27 16:02 ludovic Note Added: 0007128
2014-05-27 17:41 ludovic Note Added: 0007129
2014-05-27 17:43 ludovic Note Added: 0007130
2014-05-27 18:31 tfu Note Added: 0007131
2014-05-27 18:47 ludovic Note Added: 0007132
2014-05-27 18:47 ludovic Status new => resolved
2014-05-27 18:47 ludovic Fixed in Version => 2.2.4
2014-05-27 18:47 ludovic Resolution open => fixed
2014-05-27 18:47 ludovic Assigned To => ludovic