View Issue Details

IDProjectCategoryView StatusLast Update
0004048SOGosogo-toolpublic2017-04-07 14:19
Reporternetzkommune Assigned Toludovic  
PrioritynormalSeverityminorReproducibilityalways
Status resolvedResolutionfixed 
OSFreeBSDOS Version10.3 
Product Version3.2.6 
Fixed in Version3.2.9 
Summary0004048: sogo-tool restore core dumps
Description

we're trying to migrate a sogo 2.3.8 server on Linux to 3.2.6 on FreeBSD and use sogo-tool backup … ALL to dump and then sogo-tool restore -p and -f to import those backups. With some backups, the sogo-tool restore dies without a usable reason. We tracked it to some corrupted entries, in one case a malformed address book entry.

Is there a way to further debug this? I can't share the particular user data on the mailing list, of course, but talking to the user, we could examine the dump together with a developer, maybe?

Steps To Reproduce

use sogo-tool restore to import a corrupted file.

TagsNo tags attached.

Activities

ludovic

ludovic

2017-02-22 06:51

administrator   ~0011348

Provide the backup file privately for us to test.

netzkommune

netzkommune

2017-02-22 07:03

reporter  

badcal_1 (739 bytes)   
PRODID:-//Apple Inc.//iOS 8.4//EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:Europe/Berlin
BEGIN:DAYLIGHT
DTSTART:19810329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
TZNAME:MESZ
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:19961027T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
TZNAME:MEZ
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20150722T075640Z
DTEND;TZID=Europe/Berlin:20150722T110000
DTSTAMP:20150722T075657Z
DTSTART;TZID=Europe/Berlin:20150722T100000
LAST-MODIFIED:20150722T075640Z
SEQUENCE:0
STATUS:CONFIRMED
SUMMARY:Homeoffice ";
        "c_name" = "09326D73-EBBF-4333-831F-92C502F3DB15.ics";
     },
     {
         "c_content" = "BEGIN:VCALENDAR

badcal_1 (739 bytes)   
netzkommune

netzkommune

2017-02-22 07:03

reporter  

badcal_2 (733 bytes)   
PRODID:-//Apple Inc.//iOS 8.4//EN
VERSION:2.0
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:Europe/Berlin
BEGIN:DAYLIGHT
DTSTART:19810329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
TZNAME:MESZ
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:19961027T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
TZNAME:MEZ
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
CREATED:20150723T075952Z
DTEND;TZID=Europe/Berlin:20150723T103000
DTSTAMP:20150723T080045Z
DTSTART;TZID=Europe/Berlin:20150723T100000
LAST-MODIFIED:20150723T080017Z
SEQUENCE:0
SUMMARY:Unterwegs ";
            "c_name" = "24886DB1-3A86-46D2-9431-78E043E8546C.ics";
        },
        {
            "c_content" = "BEGIN:VCALENDAR
badcal_2 (733 bytes)   
netzkommune

netzkommune

2017-02-22 07:07

reporter   ~0011349

I attached 2 malformed calendar entries that let sogo-tool choke, I hope we extracted it correctly. The contents aren't something that needs to be kept privately as it's fortunately pretty generic.
We do have a full backup file from another user that contains a malformed address book entry, where I don't know what exactly made sogo-tool (and OSX addressbook.app btw) choke, it contained German Umlauts for the street address and city, as well as some comments. We deleted the entry thru the web frontend, though the web frontend couldn't thoroughly parse it (it only showed First and Last Name), then we ended up with an empty address book entry, which we deleted and only then we could import the backup with sogo-tool restore.

How do I share that privately?

ludovic

ludovic

2017-02-22 07:10

administrator   ~0011350

Send privately to support@inverse.ca ONLY the backup file of the user(s) for which sogo-tool fails to restore.

ludovic

ludovic

2017-02-22 08:47

administrator   ~0011353

I've imported succesfully all the data you sent privately.

Try to produce a stack trace with sogo-tool:

https://sogo.nu/nc/support/faq/article/how-do-i-debug-sogo.html

Run it like this:

gdb --args sogo-tool restore ...

netzkommune

netzkommune

2017-02-22 09:31

reporter   ~0011357

there you go:

Program received signal SIGABRT, Aborted.
[Switching to Thread 80ac06400 (LWP 101415/sogo-tool)]
0x0000000804df839a in thr_kill () from /lib/libc.so.7
(gdb) bt
#0 0x0000000804df839a in thr_kill () from /lib/libc.so.7
0000001 0x0000000804df8386 in raise () from /lib/libc.so.7
0000002 0x0000000804df8309 in abort () from /lib/libc.so.7
0000003 0x00000008046d13d3 in objc_exception_throw () from /usr/local/lib/libobjc.so.4.6
0000004 0x000000080414886b in -[NSException raise] ()
from /usr/local/GNUstep/System/Library/Libraries/libgnustep-base.so.1.24
0000005 0x00000008041481e4 in +[NSException raise:format:] ()
from /usr/local/GNUstep/System/Library/Libraries/libgnustep-base.so.1.24
0000006 0x000000080418a741 in -[NSObject doesNotRecognizeSelector:] ()
from /usr/local/GNUstep/System/Library/Libraries/libgnustep-base.so.1.24
0000007 0x000000080422cf6c in GSFFIInvocationCallback ()
from /usr/local/GNUstep/System/Library/Libraries/libgnustep-base.so.1.24
0000008 0x00000008078fe5a7 in ffi_closure_unix64_inner () from /usr/local/lib/libffi.so.6
0000009 0x00000008078fedb4 in ffi_closure_unix64 () from /usr/local/lib/libffi.so.6
0000010 0x0000000800c51250 in ?? ()
from /usr/local/GNUstep/Local/Library/Libraries/sogo/libGDLContentStore.so.4.9
0000011 0x0000000000412db3 in main ()

ludovic

ludovic

2017-02-22 09:47

administrator   ~0011358

Install the debugging symbols for SOPE and GNUstep.

netzkommune

netzkommune

2017-02-24 04:59

reporter   ~0011377

2017-02-24 09:57:25.162 sogo-tool[24279:100434] No local time zone specified.
2017-02-24 09:57:25.163 sogo-tool[24279:100434] Using time zone with absolute offset 0.
2017-02-24 09:57:25.161 sogo-tool[24279:100434] restoring record '09326D73-EBBF-4333-831F-92C502F3DB15.ics'

Program received signal SIGSEGV, Segmentation fault.
0x0000000804817479 in objc_selector_type_mismatch (cls=0x7fffffffe6b0,
selector=0x800ebad50 <.objc_selector_list+272>, result=0x2)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c:37
37 /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c: No such file or directory.
(gdb) bt
#0 0x0000000804817479 in objc_selector_type_mismatch (cls=0x7fffffffe6b0,
selector=0x800ebad50 <.objc_selector_list+272>, result=0x2)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c:37
0000001 0x00000008048178a2 in objc_msg_lookup_internal (receiver=0x7fffffffdec0,
selector=0x800ebad50 <.objc_selector_list+272>, sender=0x0)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c:94
0000002 objc_msg_lookup_sender (receiver=0x7fffffffdec0,
selector=0x800ebad50 <.objc_selector_list+272>, sender=0x0)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c:171
0000003 0x00000008048174f7 in slowMsgLookup (receiver=0x7fffffffdec0,
cmd=0x800ebad50 <.objc_selector_list+272>)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/sendmsg2.c:116
0000004 0x00000008048196a8 in objc_msgSend_fpret ()
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/objc_msgSend.x86-64.S:242
0000005 0x0000000800c97ac2 in -[GCSFolder writeContent:fromComponent:container:toName:baseVersion:] (self=0x80b6eed10, _cmd=0x633b20 <.objc_selector_list+1504>, _content=0x80e702fd0,
theComponent=0x0, theContainer=0x0, _name=0x80e703090, _baseVersion=0x7fffffffe664)
at GCSFolder.m:1037
0000006 0x0000000000419195 in -[SOGoToolRestore restoreRecords:ofFolder:] (self=0x80e662bd0,
_cmd=0x633a80 <.objc_selector_list+1344>, records=0x80e6fa150, gcsFolder=0x80b6eed10)
at SOGoToolRestore.m:426
0000007 0x0000000000419668 in -[SOGoToolRestore restoreFolder:withContent:destructive:] (
self=0x80e662bd0, _cmd=0x633ad0 <.objc_selector_list+1424>, folder=0x80f0d4570,
content=0x80e6ad270, isDestructive=0 '\000') at SOGoToolRestore.m:513
0000008 0x00000000004198dd in -[SOGoToolRestore restoreUserFolderFromUserRecord:destructive:] (
self=0x80e662bd0, _cmd=0x633af0 <.objc_selector_list+1456>, userRecord=0x80e663b30,
isDestructive=0 '\000') at SOGoToolRestore.m:553
0000009 0x000000000041a36e in -[SOGoToolRestore proceed] (self=0x80e662bd0,
_cmd=0x6335e0 <.objc_selector_list+160>) at SOGoToolRestore.m:697
0000010 0x000000000041a59c in -[SOGoToolRestore run] (self=0x80e662bd0,
_cmd=0x629db0 <.objc_selector_list+224>) at SOGoToolRestore.m:724
0000011 0x000000000040c431 in +[SOGoTool runToolWithArguments:verbose:] (
self=0x6334b0 <_OBJC_CLASS_SOGoToolRestore>, _cmd=0x6293e0 <.objc_selector_list+368>,
toolArguments=0x80f143840, isVerbose=0 '\000') at SOGoTool.m:54
0000012 0x000000000040bed5 in -[SOGoToolDispatcher run] (self=0x80e647290,
_cmd=0x629270 <.objc_selector_list>) at sogo-tool.m:197
0000013 0x000000000040c0b7 in main (argc=6, argv=0x7fffffffeab0, env=0x7fffffffeae8)
at sogo-tool.m:253

Does that help?

ludovic

ludovic

2017-02-28 08:26

administrator   ~0011386

Are you sure you're using the GNUstep packages we provide?

netzkommune

netzkommune

2017-02-28 08:36

reporter   ~0011387

no, this is running on FreeBSD and we use the FreeBSD port http://www.freshports.org/www/sogo3 and its dependencies

ludovic

ludovic

2017-02-28 14:54

administrator   ~0011389

IMO, it's a GNUstep/FFI issue under FreeBSD.

netzkommune

netzkommune

2017-02-28 16:32

reporter   ~0011390

okay, but this only happened with particular backups of particular users which then imported fine when we removed the offending entries?

netzkommune

netzkommune

2017-03-08 08:26

reporter   ~0011457

I talked to the GNUStep port maintainer and he wrote me this:

It looks as if the issue is calling a selector when the receiving object has a selector with the same name and different types. Doing this can result in stack corruption, so the GNUstep runtime turns this into an error. If you look at GCSFolder.m:1037 and see if the static and dynamic types of the object match. Normally you’ll get a compiler warning if they don’t.

Does this ring a bell?

ludovic

ludovic

2017-03-31 15:46

administrator   ~0011648

Run sogo-tool in gdb. Set a breakpoint to GCSFolder.m:1037

When the breakpoint is raised, do "po _name"

Paste the output here.

netzkommune

netzkommune

2017-04-04 03:08

reporter   ~0011671

Output:

Breakpoint 1, -[GCSFolder writeContent:fromComponent:container:toName:baseVersion:] (
self=0x80b6eec10, _cmd=0x633b20 <.objc_selector_list+1504>, _content=0x812538cd0,
theComponent=0x0, theContainer=0x0, _name=0x812538d50, _baseVersion=0x7fffffffe674)
at GCSFolder.m:1037
1037 GCSFolder.m: No such file or directory.
(gdb) po _name
2F00DFDF-FE63-477A-97E7-ED018EF0F600-ABSPlugin.vcf

ludovic

ludovic

2017-04-06 16:06

administrator   ~0011689

oops.

I meant to do:

"po quickRow"

netzkommune

netzkommune

2017-04-07 09:16

reporter   ~0011691

Output:
Breakpoint 1, -[GCSFolder writeContent:fromComponent:container:toName:baseVersion:] (self=0x80b6eec10, _cmd=0x633b20 <.objc_selector_list+1504>,
_content=0x812538cd0, theComponent=0x0, theContainer=0x0,
_name=0x812538d50, _baseVersion=0x7fffffffe674) at GCSFolder.m:1037
1037 GCSFolder.m: No such file or directory.
(gdb) po quickRow

Program received signal SIGBUS, Bus error.
0x00000008048128ef in objc_test_class_flag (aClass=0x4588c0950f000000,
flag=objc_class_flag_hidden_class)
at /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/class.h:248
248 /wrkdirs/usr/ports/lang/libobjc2/work/libobjc2-1.8.1/class.h: No such file or directory.

ludovic

ludovic

2017-04-07 09:20

administrator   ~0011692

humm..

how about "p quickRow"

and "p *quickRow"

netzkommune

netzkommune

2017-04-07 09:28

reporter   ~0011693

Breakpoint 1, -[GCSFolder writeContent:fromComponent:container:toName:baseVersion:] (self=0x80b6eed10, _cmd=0x633b20 <.objc_selector_list+1504>,
_content=0x812eb1650, theComponent=0x0, theContainer=0x0,
_name=0x812eb16d0, _baseVersion=0x7fffffffe664) at GCSFolder.m:1037
1037 GCSFolder.m: No such file or directory.
(gdb) p quickRow
$1 = (struct NSMutableDictionary ) 0x7fffffffe5a0
(gdb) p
quickRow
$2 = {<> = {<> = {isa = 0x7fffffffe6b0}, <No data fields>}, <No data fields>}

Related Changesets

sogo: master c4ad338a

2017-04-07 14:16:18

ludovic

Details Diff
(fix) initialize variable to avoid strange nil-return value on invocation (fixes 0004048) Affected Issues
0004048
mod - SOPE/GDLContentStore/GCSFolder.m Diff File

Issue History

Date Modified Username Field Change
2017-02-22 03:20 netzkommune New Issue
2017-02-22 06:51 ludovic Note Added: 0011348
2017-02-22 07:03 netzkommune File Added: badcal_1
2017-02-22 07:03 netzkommune File Added: badcal_2
2017-02-22 07:07 netzkommune Note Added: 0011349
2017-02-22 07:10 ludovic Note Added: 0011350
2017-02-22 08:47 ludovic Note Added: 0011353
2017-02-22 08:54 ludovic Priority urgent => normal
2017-02-22 08:54 ludovic Severity crash => minor
2017-02-22 09:31 netzkommune Note Added: 0011357
2017-02-22 09:47 ludovic Note Added: 0011358
2017-02-24 04:59 netzkommune Note Added: 0011377
2017-02-28 08:26 ludovic Note Added: 0011386
2017-02-28 08:36 netzkommune Note Added: 0011387
2017-02-28 14:54 ludovic Note Added: 0011389
2017-02-28 16:32 netzkommune Note Added: 0011390
2017-03-08 08:26 netzkommune Note Added: 0011457
2017-03-31 15:46 ludovic Note Added: 0011648
2017-04-04 03:08 netzkommune Note Added: 0011671
2017-04-06 16:06 ludovic Note Added: 0011689
2017-04-07 09:16 netzkommune Note Added: 0011691
2017-04-07 09:20 ludovic Note Added: 0011692
2017-04-07 09:28 netzkommune Note Added: 0011693
2017-04-07 14:17 ludovic Changeset attached => sogo master c4ad338a
2017-04-07 14:17 ludovic Assigned To => ludovic
2017-04-07 14:17 ludovic Resolution open => fixed
2017-04-07 14:19 ludovic Status new => resolved
2017-04-07 14:19 ludovic Fixed in Version => 3.2.9