Big Sur / OSX 11.0 linking problem

[ofZach]

OF 0.11.0 examples are failing to link --

Ld /Users/zachzach/Desktop/of_v0.11.0_osx_release/apps/myApps/emptyExample/bin/emptyExampleDebug.app/Contents/MacOS/emptyExampleDebug normal (in target 'emptyExample' from project 'emptyExample')
    cd /Users/zachzach/Desktop/of_v0.11.0_osx_release/apps/myApps/emptyExample
    /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -target x86_64-apple-macos10.9 -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.16.sdk -L/Users/zachzach/Desktop/of_v0.11.0_osx_release/apps/myApps/emptyExample/bin -F/Users/zachzach/Desktop/of_v0.11.0_osx_release/apps/myApps/emptyExample/bin -filelist /Users/zachzach/Library/Developer/Xcode/DerivedData/emptyExample-gysceiadgyrycjawcqquejltaxcx/Build/Intermediates.noindex/emptyExample.build/Debug/emptyExample.build/Objects-normal/x86_64/emptyExampleDebug.LinkFileList -dead_strip -Xlinker -object_path_lto -Xlinker /Users/zachzach/Library/Developer/Xcode/DerivedData/emptyExample-gysceiadgyrycjawcqquejltaxcx/Build/Intermediates.noindex/emptyExample.build/Debug/emptyExample.build/Objects-normal/x86_64/emptyExampleDebug_lto.o -Xlinker -no_deduplicate -stdlib\=libc++ ../../../libs/tess2/lib/osx/tess2.a ../../../libs/glew/lib/osx/glew.a ../../../libs/cairo/lib/osx/cairo-script-interpreter.a ../../../libs/cairo/lib/osx/cairo.a ../../../libs/cairo/lib/osx/pixman-1.a ../../../libs/fmodex/lib/osx/libfmodex.dylib ../../../libs/rtAudio/lib/osx/rtaudio.a ../../../libs/glfw/lib/osx/glfw3.a ../../../libs/FreeImage/lib/osx/freeimage.a ../../../libs/freetype/lib/osx/freetype.a ../../../libs/boost/lib/osx/boost_filesystem.a ../../../libs/boost/lib/osx/boost_system.a ../../../libs/curl/lib/osx/curl.a ../../../libs/uriparser/lib/osx/uriparser.a ../../../libs/pugixml/lib/osx/pugixml.a -framework Accelerate -framework AGL -framework AppKit -framework ApplicationServices -framework AudioToolbox -framework AVFoundation -framework Cocoa -framework CoreAudio -framework CoreFoundation -framework CoreMedia -framework CoreServices -framework CoreVideo -framework IOKit -framework OpenGL -framework QuartzCore -framework QTKit -framework Security -framework LDAP ../../../libs/openFrameworksCompiled/lib/osx/openFrameworksDebug.a -Xlinker -
dependency_info -Xlinker /Users/zachzach/Library/Developer/Xcode/DerivedData/emptyExample-gysceiadgyrycjawcqquejltaxcx/Build/Intermediates.noindex/emptyExample.build/Debug/emptyExample.build/Objects-normal/x86_64/emptyExampleDebug_dependency_info.dat -o /Users/zachzach/Desktop/of_v0.11.0_osx_release/apps/myApps/emptyExample/bin/emptyExampleDebug.app/Contents/MacOS/emptyExampleDebug

ld: file not found: /usr/lib/libstdc++.6.dylib for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

when you look, /usr/lib/ is full of aliases but the dylibs are removed.

OF compiles and links fine, it's just examples, With Xcode 11.5 and 12 beta.

I dug around but I couldn't see where this particular linking is happening (none of the .a files we link against seem to reference this dylib). I didn't check every lib but can dig a little further....

relevant-ish ? (around /usr/lib changing)

dotnet/msbuild#5454
https://lapcatsoftware.com/articles/bigsur.html

[ofZach]

cc @ofTheo

[ofTheo]

Ahh interesting. Glad you are on top of the 10.16 changes!

@ofZach it could be that the libstdc++ lib might be hardcoded in the examples when it might be better to be the default.

If you search in the build settings for libstdc and change all of the results to compiler default does it then work?
( make sure to do it on the App settings and not the project settings )

eg:

go from this:

to this:

[ofTheo]

@ofZach did that work for you? if so it could be the easiest fix for a new platform so far! :)

[ofZach]

sorry for the slow response -- unfortunately this doesn't help.

If it's useful, I think this is related to fmod, which has this path in it. if I mess with it (using install_name_tool it links but crashes / has other issues)

also this may be useful ?

https://mjtsai.com/blog/2020/06/26/reverse-engineering-macos-11-0/
https://twitter.com/mycoliza/status/1275305517066227712
https://news.ycombinator.com/item?id=23612772

[ofTheo]

@ofZach hmm do you even have an alias to that version of libstdc++6 in /usr/lib ?
I'm curious if its that the dylib doesn't exist in any form or if the linking is broken because of the alias. ( I am guessing the first ).

if there is an another libstdc++ dylib you could always try making an alias to it in /usr/lib with the same name as the missing one ( risky but could work )

[ofZach]

ah let me check -- there is an alias there but it's broken. Let me try to remove the alias (requires SIP to be turned off) -- there are actually no dylibs in the /usr/lib folder, just a handful of orphaned aliases. perhaps the upgrade to Big Sur left those in and the linking is failing since the alias is broken but if the alias is gone maybe it will work? will check...

[thel3l]

Hey @ofZach — was curious if you managed to sort out a temporary fix?

[ofZach]

sorry no I wasn't able to fix this -- it was actually completely crazy hard to adjust /usr/lib (needed to decrypt my drive, and do a lot of system recovery things, etc)

I think three possible solutions would be (a) use openAL instead of fmod (would require compiling kissfft, at least) (b) find some local dylib solution that would work with allow the fmod dylib to be happy (c) update FMOD, the version we are using is very old

[ofZach]

also @ofTheo -- I guess the dylib doesn't exist (at least not in /usr/lib)

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

I don't really know what this note means wrt to linking with the fmod dylib but it's obviously problematic the we have a dylib which includes this path that's missing in bug sur.

[ofTheo]

thanks @ofZach - I think trying the latest fmod would be a good idea.
I imagine they are preparing for this already - it might be that linking is already different in the newer release.

Here's a link to the latest fmod for macOS:
https://drive.google.com/file/d/1PqYb0fDfBi8r2MG_jxQeUZ4dsxPMgtjV/view?usp=sharing

[ofZach]

thx I'll take a look.... I remember that the api changed so it was't possible to just swap in. also these dylibs have the /usr/lib as well :(

[ofZach]

(side note, I can link properly against a dylib that doesn't have those paths, ie when I modify them with install_name_tool, and use the original dylib we ship with OF in the app bundle -- this produces apps that launch outside of Xcode via click or command line, or with a scheme in Xcode that doesn't do debug, but launching w/ debug is unhappy and crashes out somewhere...)

[thel3l]

So I managed to get things working, for anyone who needs a temporary fix, here you go:

Follow these steps at your own risk. This is absolutely not a recommended fix, and since we still don't have all the details of macOS 11's new Signed System Volume feature, I do not know how this will break system updates in the future.

TL;DR: What happened? macOS 11 no longer provides copies of dylibs in the /usr/lib/directory. All that is left behind are broken symlinks:

From the tweet @ofZach mentioned:

SIP now ships with a new mode called authenticated-root. Basically, to allow writing data in the root filesystem, you must disable authenticated-root mode, but this is not possible unless Firevault is also disabled.
Crudely: disabling authenticated-root is the equivalent to csrutil disable in macOS <10.11.

---

I'm not totally sure what the official fix would be ... maybe someone with more experience can chip in with a real fix. But if you really need it to run, here is the awful temporary fix (to write to any system file in–fact):

1. Install Xcode Command Line tools (if it's gone after the updates).
2. Disable Firevault and wait until it has finished decrypting your drive:

3. Reboot into Recovery mode (⌘-R while rebooting), open the Terminal and disable SIP:

csrutil disable
csrutil authenticated-root disable # Filevault depends on this, isn't possible to disable unless you've turned it off.

4. Reboot back into macOS 11. Now identify the name of your root volume and remount it.
   Run mount to get the name of the root mount device:

Then make a mount point and mount the root filesystem:

mkdir /tmp/myVolumeMount
sudo mount -o nobrowse -t apfs /dev/disk1s1 /tmp/myVolumeMount

Now copy over a copy of libstdc++.6.dylib into /usr/lib (don't trust my attachment here, go ahead and get yourself a version from your old install of 10.15 or whatever.):

libstdc-oF.zip

sudo cp libstdc++.6.0.9.dylib libstdc++.6.dylib /tmp/myVolumeMount/usr/lib/

Last step, to make the filesystem bootable again:

sudo bless --folder /tmp/myVolumeMount/System/Library/CoreServices --bootefi --create-snapshot

And reboot. Everything should compile fine now.

---

Seriously, if anyone has a better idea that doesn't involve mucking with system libraries, it would be awesome, even if it's just a thought.
Some references: https://developer.apple.com/news/?id=3xpv8r2m

Best.

[ofZach]

I wonder if it's possible to use this dylib but not in /usr/lib, ie, modify fmod dylib using install_name_tool but put this in libstdc++.6.0.9.dylib in ~/libs or even relative to the fmod dylib. In my experience linking and debugging fails on Big Sur, but using the fmod dylib as is, in the app works fine. It could be that the solution would be something like a modified fmod that searches in a path that's easier to modify than /usr/lib.

I did the same thing you did @thel3l (expect test adding the dylibs) and it's quite an aggressive and scary fix ! ! thanks for documenting it so well...

[thel3l]

modified fmod that searches in a path that's easier to modify than /usr/lib

Ah welp, yeah, that should work. I'll stay subscribed, hopefully something will work out before the GM release :)

Best.