rpc_applicationSentListing missing WIRListingKey - iOS 8.1.2, 8.3

[jchuong]

Sample XML dump from a user:

Unknown app_id PID:534
Invalid message _rpc_applicationSentListing: (dict)
(key)WIRApplicationIdentifierKey(/key)
(string)PID:534(/string)
(key)WIRListingKey(/key)
(dict)
(/dict)
(/dict)
ss.remove_fd(6)
Disconnected :27753 from iPhone5.mobile testing (223f8c744f08c7776c6b38642920749566e396d2)
ss.remove_server_fd(4)
ss.remove_fd(3)

Note that WIRListingKey has no value set. Seems to be an issue in recent iOS updates.

Reported by @LimeDang and @ChuanyuWang. Can you guys tell me what application you're testing? With Mobile Safari, if I recall correctly this would normally return the active tab id so having this break would be an issue with tests on Safari. I also don't have a device on the affected versions of iOS so it may be a while to get a fix for it.

[ChuanyuWang]

Add more information:
Appium clone from master branch
ios-webkit-debug-proxy clone from master branch@jchuong fork.
device: iOS 8.3 by iPad Air and iOS 8.3 by iPad 3
Mac OS 10.10.3

Connection iPad to Mac, and testing Safari browser on iPad by Appium and ios-webkit-debug-proxy.

I added more log in the ios_webkit_debug_proxy.c file and try to figure out the different rpc message purpose, e.g. applicationupdate etc.

@jchuong may you share with me more information about the RPC message specification? I don't know how to debug it on Windows, and Mac OS is killing me these days.

[jchuong]

I'm not sure if you can debug it with Windows (Cygwin or something) so I don't recommend it.

The rpc messages are kind of straight forward if you look at what is being sent. From the design document, applicationSent* is basically the messages from the webinspector being received in iwdp. sentListing would be list of tabs, connectedApps would be applications currently connected, applicationUpdated seems to be updating application PID or names. Most of the functions have the an example plist received.

Upon some thought, it might be an issue with libimobiledevice/libplist being a little out of date. By 1.2.0 of libimobiledevice, their website states that it supports up to 8.2 currently. If the plist has changed then iwdp reading it will need to be updated.

[ChuanyuWang]

Thanks for the information.
I have figured out what applicationUpdated message means, it will be emitted when active tab changed.

I have tested it many times on iOS 8.3 (iPad 3). Every time when I add a new tab or switch tabs, this event is emitted.

So I remove the content of function iwdp_on_applicationUpdated(), and return RPC_SUCCESS; directly.
And also, I comment out the below piece of code in iwdp_on_reportConnectedApplicationList() function

if (*apps == NULL) {
    return RPC_SUCCESS;
}

As result, the test case still fails, and I can't get any error log from console.

[ChuanyuWang]

I am going to show some interesting log below, I have formatted the log for your reference, while the log is still too log, I have to split it into two comments. A little more background:
I have two device, one is iOS 8.3, the other is iOS 7.0.4. I run the same test as below:

1. connect the device to Mac OS 10.10.3
2. launch iOS-webkit-debug-proxy
3. click Safari browser from home screen on iPad (the default page is www.apple.com)
4. add a new tab
5. navigate to www.yahoo.com from address bar
6. switch active tab back to www.apple.com

library info:

• libplist 1.1.2
• libusbmuxd 1.0.10
• libimobiledevice 1.2.0
• iOS-webkit-debug-proxy master@jchuong@ed71f7351dac3f697ba5e3bab96d70249cf1e63f

conclusions:

• new keys are introduced in iOS 8.3
  • WIRIsApplicationActiveKey
  • WIRApplicationBundleIdentifierKey
  • WIRHostApplicationIdentifierKey
• WIRApplicationIdentifierKey contains the PID instead of bundleID in iOS 7.0.4
• WIRApplicationBundleIdentifierKey contains the original WIRApplicationIdentifierKey information
• WIRHostApplicationIdentifierKey is used to describe the hierarchy of browser and its tabs.
• new RPC message _rpc_applicationUpdated is emitted, which is used to update the active tab information, the active tab has key WIRIsApplicationActiveKey with value 1, all the other tabs have value 0
• Application with PID 63 is the host of Safari browser, which is disconnected very soon after browser shows up.
• RPC message _rpc_applicationSentListing only contains the active tab and its app instead of all tabs in iOS 7
• RPC message _rpc_applicationSentListing is emitted multi times with same content

I hope what we did could help someone suffering this on iOS 8.3, which took me almost a week

[ChuanyuWang]

<----     click Safari browser on iOS 7  ---->

<********************_rpc_reportConnectedApplicationList:
<dict>
    <key>WIRApplicationDictionaryKey</key>
    <dict>
        <key>com.apple.mobilesafari</key>
        <dict>
            <key>WIRApplicationIdentifierKey</key>
            <string>com.apple.mobilesafari</string>
            <key>WIRApplicationNameKey</key>
            <string>Safari</string>
            <key>WIRIsApplicationProxyKey</key>
            <false/>
        </dict>
    </dict>
</dict>
<*********************************>

<++++     add app ID:com.apple.mobilesafari  ++++>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>com.apple.mobilesafari</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
            <key>WIRTitleKey</key>
            <string>Apple</string>
            <key>WIRURLKey</key>
            <string>http://www.apple.com/</string>
        </dict>
    </dict>
</dict>
<*********************************>

<----     add a new tab  ---->

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>com.apple.mobilesafari</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
            <key>WIRTitleKey</key>
            <string>Apple</string>
            <key>WIRURLKey</key>
            <string>http://www.apple.com/</string>
        </dict>
        <key>2</key>
        <dict>
            <key>WIRPageIdentifierKey</key>
            <integer>2</integer>
            <key>WIRTitleKey</key>
            <string></string>
            <key>WIRURLKey</key>
            <string>about:blank</string>
        </dict>
    </dict>
</dict>
<*********************************>

<----     navigate to www.yahoo.com in address bar  ---->

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>com.apple.mobilesafari</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
            <key>WIRTitleKey</key>
            <string>Apple</string>
            <key>WIRURLKey</key>
            <string>http://www.apple.com/</string>
        </dict>
        <key>2</key>
        <dict>
            <key>WIRPageIdentifierKey</key>
            <integer>2</integer>
            <key>WIRTitleKey</key>
            <string>Yahoo</string>
            <key>WIRURLKey</key>
            <string>https://www.yahoo.com/</string>
        </dict>
    </dict>
</dict>
<*********************************>

<----     switch the active tab  ---->

Nothing happen

[ChuanyuWang]

<----     click Safari browser on iOS 8.3  ---->

<********************_rpc_reportConnectedApplicationList:
<dict>
    <key>WIRApplicationDictionaryKey</key>
    <dict>
        <key>PID:824</key>
        <dict>
            <key>WIRApplicationIdentifierKey</key>
            <string>PID:824</string>
            <key>WIRIsApplicationProxyKey</key>
            <false/>
            <key>WIRApplicationNameKey</key>
            <string>Safari</string>
            <key>WIRApplicationBundleIdentifierKey</key>
            <string>com.apple.mobilesafari</string>
            <key>WIRIsApplicationActiveKey</key>
            <integer>1</integer>
        </dict>
        <key>PID:827</key>
        <dict>
            <key>WIRApplicationIdentifierKey</key>
            <string>PID:827</string>
            <key>WIRIsApplicationProxyKey</key>
            <true/>
            <key>WIRApplicationNameKey</key>
            <string></string>
            <key>WIRApplicationBundleIdentifierKey</key>
            <string>com.apple.WebKit.WebContent</string>
            <key>WIRIsApplicationActiveKey</key>
            <integer>0</integer>
            <key>WIRHostApplicationIdentifierKey</key>
            <string>PID:824</string>
        </dict>
        <key>PID:63</key>
        <dict>
            <key>WIRApplicationIdentifierKey</key>
            <string>PID:63</string>
            <key>WIRIsApplicationProxyKey</key>
            <false/>
            <key>WIRApplicationNameKey</key>
            <string></string>
            <key>WIRApplicationBundleIdentifierKey</key>
            <string>lockdownd</string>
            <key>WIRIsApplicationActiveKey</key>
            <integer>0</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<++++     add app ID:PID:824  ++++>

<++++     add app ID:PID:827  ++++>

<++++     add app ID:PID:63  ++++>

<********************_rpc_applicationDisconnected:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:63</string>
    <key>WIRIsApplicationProxyKey</key>
    <false/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>lockdownd</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>0</integer>
</dict>
<*********************************>

<++++     remove app ID:PID:63  ++++>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:824</string>
    <key>WIRListingKey</key>
    <dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:827</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string>Apple</string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>http://www.apple.com/</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationUpdated:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:827</string>
    <key>WIRIsApplicationProxyKey</key>
    <true/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>com.apple.WebKit.WebContent</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>1</integer>
    <key>WIRHostApplicationIdentifierKey</key>
    <string>PID:824</string>
</dict>
<*********************************>

<----     add a new tab  ---->

<********************_rpc_applicationConnected:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRIsApplicationProxyKey</key>
    <true/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>com.apple.WebKit.WebContent</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>1</integer>
    <key>WIRHostApplicationIdentifierKey</key>
    <string>PID:824</string>
</dict>
<*********************************>

<++++     add app ID:PID:916  ++++>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string></string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>about:blank</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string></string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>about:blank</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationUpdated:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:827</string>
    <key>WIRIsApplicationProxyKey</key>
    <true/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>com.apple.WebKit.WebContent</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>0</integer>
    <key>WIRHostApplicationIdentifierKey</key>
    <string>PID:824</string>
</dict>
<*********************************>

<----     navigate to www.yahoo.com in address bar  ---->

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string>Yahoo Singapore</string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>https://sg.yahoo.com/searchset?p=us</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string>Yahoo Singapore</string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>https://sg.yahoo.com/searchset?p=us</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<********************_rpc_applicationSentListing:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRListingKey</key>
    <dict>
        <key>1</key>
        <dict>
            <key>WIRTitleKey</key>
            <string>Yahoo Singapore</string>
            <key>WIRTypeKey</key>
            <string>WIRTypeWeb</string>
            <key>WIRURLKey</key>
            <string>https://sg.yahoo.com/searchset?p=us</string>
            <key>WIRPageIdentifierKey</key>
            <integer>1</integer>
        </dict>
    </dict>
</dict>
<*********************************>

<----     switch the active tab  ---->

<********************_rpc_applicationUpdated:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:827</string>
    <key>WIRIsApplicationProxyKey</key>
    <true/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>com.apple.WebKit.WebContent</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>1</integer>
    <key>WIRHostApplicationIdentifierKey</key>
    <string>PID:824</string>
</dict>
<*********************************>

<********************_rpc_applicationUpdated:
<dict>
    <key>WIRApplicationIdentifierKey</key>
    <string>PID:916</string>
    <key>WIRIsApplicationProxyKey</key>
    <true/>
    <key>WIRApplicationNameKey</key>
    <string></string>
    <key>WIRApplicationBundleIdentifierKey</key>
    <string>com.apple.WebKit.WebContent</string>
    <key>WIRIsApplicationActiveKey</key>
    <integer>0</integer>
    <key>WIRHostApplicationIdentifierKey</key>
    <string>PID:824</string>
</dict>
<*********************************>

[jchuong]

Thanks for the information @ChuanyuWang. I do suggest perhaps using a gist for logs but it was very insightful. Just to confirm, there are no new rpc messages being fired (_rpc_applicationUpdated is already being handled), but the contents are different in iOS 8. For the most part, this will just mean that the current rpc_recv_application* functions will only need to handle the new keys. The functionality should stay pretty much the same.

[ChuanyuWang]

@jchuong I saw the rpc message _rpc_applicationUpdated is already being handled in your Feb fix, while as I tested, this message is fired when active tab change, there is no PID change. I think you may need to update the way you handle it.