Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup #199

ghost · 2017-08-16T11:32:42Z

[REQUIRED] Step 2: Describe your environment

Xcode version: 8.3.3
Firebase SDK version: 4.0.4
Library version: 4.0.4
Firebase Product: Messaging

[REQUIRED] Step 3: Describe the problem

Since the migration to Firebase SDK v4 I'm facing intermittent (2-3% DAU) EXC_BREAKPOINT crashes in [FIRMessagingRmq2PersistentStore openDatabase:]

Steps to reproduce:

Code is setup as in https://github.com/firebase/quickstart-ios/blob/master/messaging/MessagingExampleSwift/AppDelegate.swift

Running on main thread:

FirebaseApp.configure()
Messaging.messaging().delegate = self
Messaging.messaging().shouldEstablishDirectChannel = false

Relevant Code:

Relevant threads in stack trace:

#0. Crashed: com.apple.main-thread
0  MyApp                          0x1006ae45c -[FIRMessagingRmq2PersistentStore openDatabase:] + 4302365788
1  MyApp                          0x1006ad770 -[FIRMessagingRmq2PersistentStore initWithDatabaseName:] + 4302362480
2  MyApp                          0x1006b016c -[FIRMessagingRmqManager initWithDatabaseName:] + 4302373228
3  MyApp                          0x1006977a8 -[FIRMessaging setupRmqManager] + 4302272424
4  MyApp                          0x100697144 -[FIRMessaging start] + 4302270788
5  MyApp                          0x100696e28 __25+[FIRMessaging messaging]_block_invoke + 4302269992
6  libswiftCoreData.dylib         0x1806629a0 (Missing)
7  libswiftCoreData.dylib         0x1806636cc (Missing)
8  MyApp                          0x100696dac +[FIRMessaging messaging] + 4302269868
9  MyApp                          0x100696798 +[FIRMessaging(FIRApp) didReceiveConfigureSDKNotification:] + 4302268312
10 libswiftCoreData.dylib         0x1817455f4 (Missing)
11 libswiftCoreData.dylib         0x181744d08 (Missing)
12 libswiftCoreData.dylib         0x181744a84 (Missing)
13 libswiftCoreData.dylib         0x1817b37a8 (Missing)
14 libswiftCoreData.dylib         0x18168895c (Missing)
15 libswiftCoreData.dylib         0x18219b970 (Missing)
16 MyApp                          0x10066915c +[FIRApp sendNotificationsToSDKs:] + 4302082396
17 MyApp                          0x1006680fc +[FIRApp configureDefaultAppWithOptions:sendingNotifications:] + 4302078204
18 MyApp                          0x100667d90 +[FIRApp configure] + 4302077328
19 MyApp                          0x1000d5944 specialized AppManager.handleFinishLaunching(application : UIApplication, launchOptions : [UIApplicationLaunchOptionsKey : Any]?) -> Bool (AppManager.swift:48)
20 MyApp                          0x1000d2018 AppManager.handleFinishLaunching(application : UIApplication, launchOptions : [UIApplicationLaunchOptionsKey : Any]?) -> Bool (AppManager.swift)
21 MyApp                          0x10038e2e8 specialized AppDelegate.application(UIApplication, didFinishLaunchingWithOptions : [UIApplicationLaunchOptionsKey : Any]?) -> Bool (AppDelegate.swift:36)
22 MyApp                          0x10038cc88 @objc AppDelegate.application(UIApplication, didFinishLaunchingWithOptions : [UIApplicationLaunchOptionsKey : Any]?) -> Bool (AppDelegate.swift)
23 UIKit                          0x18794de48 -[UIApplication _handleDelegateCallbacksWithOptions:isSuspended:restoreState:] + 380
24 UIKit                          0x187b5a37c -[UIApplication _callInitializationDelegatesForMainScene:transitionContext:] + 3452
25 UIKit                          0x187b5fe24 -[UIApplication _runWithMainScene:transitionContext:completion:] + 1684
26 UIKit                          0x187b748b0 __84-[UIApplication _handleApplicationActivationWithScene:transitionContext:completion:]_block_invoke.3147 + 48
27 UIKit                          0x187b5d0b8 -[UIApplication workspaceDidEndTransaction:] + 168
28 FrontBoardServices             0x183354884 __FBSSERIALQUEUE_IS_CALLING_OUT_TO_A_BLOCK__ + 36
29 FrontBoardServices             0x1833546f0 -[FBSSerialQueue _performNext] + 176
30 FrontBoardServices             0x183354aa0 -[FBSSerialQueue _performNextFromRunLoopSource] + 56
31 libswiftCoreData.dylib         0x18175942c (Missing)
32 libswiftCoreData.dylib         0x181758d9c (Missing)
33 libswiftCoreData.dylib         0x1817569a8 (Missing)
34 libswiftCoreData.dylib         0x181686da4 (Missing)
35 UIKit                          0x187946fc8 -[UIApplication _run] + 652
36 UIKit                          0x187941c9c UIApplicationMain + 208
37 MyApp                          0x10038ddb8 main (AppDelegate.swift:12)
38 libswiftCoreData.dylib         0x18069559c (Missing)

#2. FIRAMonitorQueue
0  libswiftCoreData.dylib         0x1807a3e70 (Missing)
1  libswiftCoreData.dylib         0x180872070 (Missing)
2  libswiftCoreData.dylib         0x1802156dc (Missing)
3  libswiftCoreData.dylib         0x1802155cc (Missing)
4  libswiftCoreData.dylib         0x180220478 (Missing)
5  MyApp                          0x1006429f4 -[FIRAMonitor saveMonitoringDataToUserDefaultsOnMonitorQueue] + 4301924852
6  libswiftCoreData.dylib         0x1806629e0 (Missing)
7  libswiftCoreData.dylib         0x1806629a0 (Missing)
8  libswiftCoreData.dylib         0x180670ad4 (Missing)
9  libswiftCoreData.dylib         0x1806662cc (Missing)
10 libswiftCoreData.dylib         0x180672a50 (Missing)
11 libswiftCoreData.dylib         0x1806727d0 (Missing)
12 libswiftCoreData.dylib         0x18086b100 (Missing)
13 libswiftCoreData.dylib         0x18086acac (Missing)

#6. FIRAnalyticsQueue
0  MyApp                          0x100632000 -[FIRAMeasurement initWithOptions:] + 4301856768
1  MyApp                          0x100631310 __55+[FIRAMeasurement initializeSharedInstanceWithOptions:]_block_invoke + 4301853456
2  libswiftCoreData.dylib         0x1806629a0 (Missing)
3  libswiftCoreData.dylib         0x1806636cc (Missing)
4  MyApp                          0x10063127c +[FIRAMeasurement initializeSharedInstanceWithOptions:] + 4301853308
5  MyApp                          0x100648888 __47+[FIRAnalytics startWithConfiguration:options:]_block_invoke_2 + 4301949064
6  libswiftCoreData.dylib         0x1806629e0 (Missing)
7  libswiftCoreData.dylib         0x1806629a0 (Missing)
8  libswiftCoreData.dylib         0x180670ad4 (Missing)
9  libswiftCoreData.dylib         0x1806662cc (Missing)
10 libswiftCoreData.dylib         0x180672a50 (Missing)
11 libswiftCoreData.dylib         0x1806727d0 (Missing)
12 libswiftCoreData.dylib         0x18086b100 (Missing)
13 libswiftCoreData.dylib         0x18086acac (Missing)

The text was updated successfully, but these errors were encountered:

ghost · 2017-08-19T17:33:29Z

Occurring in 4.1 also. Can't reproduce on demand, but hundreds of crashes reported in Crashlytics (still around 2-3% of daily sessions).

paulb777 · 2017-08-31T14:35:42Z

Are there any patterns about the device types with the crashes? Are they low in available storage?

ghost · 2017-08-31T15:27:51Z

Mostly commonly used 6/6s/7(+) iPhones. iOS 10+. Generally low on memory (1-5%), but memory varies up to 50% free.

ghost · 2017-08-31T15:28:22Z

Sorry, disk space up to 50%.

rsattar · 2017-08-31T17:04:14Z

I'd love it if there was a way to reproduce this, but failing that, we can add some additional safety around this. What's strange is that we haven't seen this reported before (and this code has not changed in a very long time).

ghost · 2017-08-31T17:48:17Z

I'd definitely like to have a repro case, I'll see if I can manage to automate the startup sequence and trigger it :)
Version 3.x worked flawlessly. I don't think it's due to an interaction with other libs, Firebase is first in didFinishLaunching in my case anyway.
If it's of any help, I'm only using Analytics and Messaging. Never used swizzling and in 4.x opted out of direct messaging (using the old-school way).

Judging from this community version, could it be crashing in sqlite init part?
If I can help in any other way, name it :)

In the meantime, I'll have to revert to 3.x - do you think that may cause further issues?

rsattar · 2017-08-31T17:51:37Z

Thanks for the info. And it's also interesting that it works in 3.X but is crashy on 4.X. We haven't modified that code in a long time. I don't think you're negatively affected by reverting to 3.X, but obviously we'd like everyone to be on the latest version (much easier for us to provide support!).

bcattle · 2017-09-08T17:32:01Z

I'd also like to flag myself as getting this issue quite consistently. Maybe 8-9% DAUs? Let me know what I can do to help you guys track this down.

My observations:

iOS 10 only
Firebase v4.1.0
Disk: 2%-72% free
Recent iPhones only: 6, 6s, SE, 7

I'm still combing through the stack traces, let me know if there's more information I can provide.

ghost · 2017-09-08T18:39:05Z

After reverting to 3.x, crashes are indeed gone. I’ll make isolated examples of Firebase setups I’m using for both, but there’s nothing special really.

bcattle · 2017-09-08T20:16:34Z

We only started seeing this issue with the recent 4.1.1 release. @VladislavJevremovic did you need to revert all the way back to 3.x? Did you try 4.0.2?

ghost · 2017-09-08T20:22:29Z

Actually, I think 4.0.4 was the first 4.x I used. I noticed the issue then. Can’t vouch for anything before that.

rsattar · 2017-09-08T20:22:51Z

Hmm, this is getting worrisome. Another question: Are your apps Swift-only? What other SDKs are you guys using (wondering if there is some conflict with sqlite)

bcattle · 2017-09-08T20:29:52Z

Ours is 95% Swift. Our bridging header is

#import <FBSDKCoreKit/FBSDKCoreKit.h>
#import <FBSDKLoginKit/FBSDKLoginKit.h>
#import <FBSDKShareKit/FBSDKShareKit.h>
#import <DateTools/DateTools.h>
#import <BugfenderSDK/BugfenderSDK.h>
// UI stuff
#import <TOCropViewController/TOCropViewController.h>
#import "ItemCenteredPagingFlowLayout.h"
#import "YLProgressBar.h"
#import <UIKit/UIGestureRecognizerSubclass.h>

The main dependency there is the Facebook stuff, and I'm not sure why we're still using Objective-C versions of their libraries.

Pod dependencies are

    pod 'Firebase/Core'
    pod 'Firebase/Storage'
    pod 'Firebase/Auth'
    pod 'Firebase/Database'
    pod 'Firebase/Messaging'

    pod 'FacebookCore'
    pod 'FacebookLogin'
    pod 'FacebookShare'
    pod 'FBSDKShareKit', '4.22.1'

    pod 'BugfenderSDK', '~> 0.3'
    pod 'Fabric'
    pod 'Crashlytics'
    pod 'CocoaLumberjack/Swift'
    pod 'PaperTrailLumberjack'

    pod 'SwiftyJSON'
    pod 'SDWebImage', '~> 3.8.0'
    pod 'DateTools'
    pod 'Cartography'
    pod 'MBCircularProgressBar'
    pod 'NVActivityIndicatorView'
    pod 'TOCropViewController'
    pod 'Pulsator'
    pod 'CHTCollectionViewWaterfallLayout'
    pod 'TwilioVideo', '~> 1.0'

ghost · 2017-09-08T20:40:00Z

My code is Swift-only, 3rd party libs most probably not:

pod 'Fabric'
pod 'Crashlytics'

pod 'GoogleSignIn'

pod 'Firebase/Core'
pod 'Firebase/Messaging'

pod 'FBSDKLoginKit'
pod 'FBSDKShareKit'

pod 'Intercom'

(only GoogleSignIn in bridging header)

Grepping the workspace for 'sqlite' revealed only uploadDSYM (from Fabric/Crashlytics) apart from Firebase itself.

bcattle · 2017-09-08T20:42:27Z

Same. Aside from FirebaseMessaging and FirebaseAnalytics I only see sqlite referenced in uploadDSYM.

bcattle · 2017-09-08T21:15:14Z

FYI the function that is crashing is here

bcattle · 2017-09-08T21:29:50Z

I don't know if this is relevant, but the dashboard in Crashlytics shows "App in Focus 25%". I'm not sure what that means, but it seems odd. Especially odd since this crash is happening in the application(application: didFinishLaunchingWithOptions function, which seems like an odd place for the app to be heading to the background.

Could this perhaps be triggered by an app transition to the background during a slow-running disk operation?

Or maybe I'm misunderstanding what the Crashlytics graph means, or perhaps it's just plain wrong.

rsattar · 2017-09-08T21:32:50Z

@bcattle Thanks for the info, and yeah this line is what typically causes a EXC_BREAKPOINT.

I could just add some safety around it, but I'd love to reproduce. Going to see if I can contrive the SDK to trigger when app is going into background. Thanks!

rsattar · 2017-09-08T22:19:21Z

Is anyone here using Core Data for other aspects of their app? Wondering if it's related to this SO post: https://stackoverflow.com/questions/39335386/did-ios10-remove-the-ability-to-read-a-sqlite-database-from-the-bundle

bcattle · 2017-09-08T22:20:30Z

We're not using Core Data.

…

On Fri, Sep 8, 2017 at 6:19 PM Riz ***@***.***> wrote: Is anyone here using Core Data for other aspects of their app? Wondering if it's related to this SO post: https://stackoverflow.com/questions/39335386/did-ios10-remove-the-ability-to-read-a-sqlite-database-from-the-bundle — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#199 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAl2Gpln0PprfM-DKIx2OwYG1DLKF8iqks5sgb1rgaJpZM4O4xwW> .

ghost · 2017-09-09T04:54:36Z

No Core Data here either.

ghost · 2017-09-09T05:00:40Z

App in Focus < 1%

karpelcevs · 2017-09-09T07:58:36Z

For us also app in focus < 1%, but it's something that crashlytics gets wrong, as we get same app in focus rate for definitely in-focus crashes as well.
But we do have core data, plus integrate with Google Maps, I think they used core data additionaly as well.

karpelcevs · 2017-09-09T08:01:37Z

We could add some debugging info in crashlytics logs and flags, such as app state on launch if that helps and get back to you with additional info. Any ideas what might be useful?
For us crash rate is much lower than for others so we won't downgrade yet.

rmanafian · 2017-09-12T22:43:51Z

We're seeing this crash as well. Contrary to everyone else, however, we're getting App in Focus 100%. I'm happy to provide any information needed to help debug this.

rsattar · 2017-09-13T22:02:03Z

Thanks for all the info, everyone. Some more questions: Is it possible to see any logs leading up to the crash? Is there any information like

Could not open rmq database ....

... in the logs?

If someone can create a public link for a particular Crashlytics crash (it hides many of the private fields for a crash, so it's safe to show publicly), that would be great. I'm adding additional logging to figure out the exact result code for the crash.

bcattle · 2017-09-13T22:15:54Z

Hi Riz, thanks for the followup.

I searched all of our available device logs for the token rmq and didn't
find it. However, this doesn't mean it's not there if the device is
crashing before the logs have a chance to be sent to the server. So I'm
afraid this is unhelpful for you.

I'm not sure how to create a public link to a specific crash session (I do
see the share link for the whole issue, just not for a specific instance of the
issue). However here's a full text readout of one of the crash incidents
from the downloadable text file (attached).

Thanks for your help and let me know if there's anything else I can
provide.

com.getnearly.nearly_issue_62_crash_5a3d436e87374294a266c502bf5ddaa3_3ed831fc98c311e7b8e356847afe9799_0_v2.txt

This removes the `FIRMessaging_FAIL` macro which was using `__builtin_trap()`, and replaced with `NSAssert` calls. These `NSAssert` calls may not get called in release builds, and so we also log them with FIRLogger error messages. The RMQ database open error result code is now parsed and included in the error message to help us identify causes for #199

rsattar · 2017-09-13T23:49:34Z

Ok, so this PR should stop the crash from occurring, however it doesn't solve the underlying issue and may still cause an exception down the line. The error code detail for why the database open failed is now being logged as an error in the console, so I'm hoping that if you see this happening, we can learn why it's failing.

I'd also suggest that if you are not using direct-channel data messages, then you should set:

Messaging.messaging().shouldEstablishDirectChannel = false

Since most of the database-related actions are for the direct socket channel.

This removes the `FIRMessaging_FAIL` macro which was using `__builtin_trap()`, and replaced with `NSAssert` calls. These `NSAssert` calls may not get called in release builds, and so we also log them with FIRLogger error messages. The RMQ database open error result code is now parsed and included in the error message to help us identify causes for #199 .

bcattle · 2017-09-14T18:42:26Z

Thanks for the update @rsattar I'll try it.
Note that the default value for shouldEstablishDirectChannel is false. source

MikeKasperlik · 2017-09-17T15:58:36Z

We also see this in our app quite often. We do not use core data and are 100% swift. Currently on FBSKD 4.1.0.

ghost · 2017-10-06T13:58:35Z

The issue seems to be gone in the current release, as expected. However, as said above, I’m not sure how we can pull the logs from devices out there. Never seen it in test yet.

Issue can be closed if you wish. Hoped we could get a definitive reason why it occurred, though.

morganchen12 · 2017-10-06T17:28:12Z

Thanks @VladislavJevremovic. We'll keep an eye out for this issue in the future, and please re-open this ticket if it regresses.

This removes the `FIRMessaging_FAIL` macro which was using `__builtin_trap()`, and replaced with `NSAssert` calls. These `NSAssert` calls may not get called in release builds, and so we also log them with FIRLogger error messages. The RMQ database open error result code is now parsed and included in the error message to help us identify causes for firebase#199 .

ghost changed the title ~~Crash in +[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup~~ Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup Aug 16, 2017

paulb777 assigned rsattar Aug 16, 2017

paulb777 added the api: messaging label Aug 16, 2017

rsattar mentioned this issue Sep 13, 2017

Remove FIRMessaging_FAIL macro, log result code #265

Merged

morganchen12 closed this as completed Oct 6, 2017

antranapp mentioned this issue Oct 23, 2017

FirebaseMessaging can't open database, probably because of "Data Protection" capability. #401

Closed

firebase locked and limited conversation to collaborators Nov 13, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup #199

Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup #199

ghost commented Aug 16, 2017 •

edited by ghost

Loading

ghost commented Aug 19, 2017

paulb777 commented Aug 31, 2017

ghost commented Aug 31, 2017

ghost commented Aug 31, 2017

rsattar commented Aug 31, 2017

ghost commented Aug 31, 2017

rsattar commented Aug 31, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

rsattar commented Sep 8, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

bcattle commented Sep 8, 2017 •

edited

Loading

bcattle commented Sep 8, 2017

bcattle commented Sep 8, 2017 •

edited

Loading

rsattar commented Sep 8, 2017

rsattar commented Sep 8, 2017

bcattle commented Sep 8, 2017 via email

ghost commented Sep 9, 2017

ghost commented Sep 9, 2017

karpelcevs commented Sep 9, 2017

karpelcevs commented Sep 9, 2017

rmanafian commented Sep 12, 2017

rsattar commented Sep 13, 2017

bcattle commented Sep 13, 2017 •

edited

Loading

rsattar commented Sep 13, 2017 •

edited

Loading

bcattle commented Sep 14, 2017

MikeKasperlik commented Sep 17, 2017

ghost commented Oct 6, 2017

morganchen12 commented Oct 6, 2017

Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup #199

Crash in -[FIRMessagingRmq2PersistentStore openDatabase:] caused by Messaging setup #199

Comments

ghost commented Aug 16, 2017 • edited by ghost Loading

[REQUIRED] Step 2: Describe your environment

[REQUIRED] Step 3: Describe the problem

Steps to reproduce:

Relevant Code:

ghost commented Aug 19, 2017

paulb777 commented Aug 31, 2017

ghost commented Aug 31, 2017

ghost commented Aug 31, 2017

rsattar commented Aug 31, 2017

ghost commented Aug 31, 2017

rsattar commented Aug 31, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

rsattar commented Sep 8, 2017

bcattle commented Sep 8, 2017

ghost commented Sep 8, 2017

bcattle commented Sep 8, 2017 • edited Loading

bcattle commented Sep 8, 2017

bcattle commented Sep 8, 2017 • edited Loading

rsattar commented Sep 8, 2017

rsattar commented Sep 8, 2017

bcattle commented Sep 8, 2017 via email

ghost commented Sep 9, 2017

ghost commented Sep 9, 2017

karpelcevs commented Sep 9, 2017

karpelcevs commented Sep 9, 2017

rmanafian commented Sep 12, 2017

rsattar commented Sep 13, 2017

bcattle commented Sep 13, 2017 • edited Loading

rsattar commented Sep 13, 2017 • edited Loading

bcattle commented Sep 14, 2017

MikeKasperlik commented Sep 17, 2017

ghost commented Oct 6, 2017

morganchen12 commented Oct 6, 2017

ghost commented Aug 16, 2017 •

edited by ghost

Loading

bcattle commented Sep 8, 2017 •

edited

Loading

bcattle commented Sep 8, 2017 •

edited

Loading

bcattle commented Sep 13, 2017 •

edited

Loading

rsattar commented Sep 13, 2017 •

edited

Loading