Skip to content

[clang][Driver] Add -fuse-lipo option #121231

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 16, 2025
Merged

Conversation

khyperia
Copy link
Contributor

@khyperia khyperia commented Dec 27, 2024

Partially fixes #59552 by adding a new option -fuse-lipo that can specify the tool name to be used by clang-driver for the lipo action. For example, pass -fuse-lipo=llvm-lipo to use llvm-lipo instead of the default lipo.

Copy link

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

@llvmbot llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' labels Dec 27, 2024
@llvmbot
Copy link
Member

llvmbot commented Dec 27, 2024

@llvm/pr-subscribers-clang-driver

Author: Ashley Hauck (khyperia)

Changes

This is my first LLVM PR! Please feel free to provide feedback/etc. - I am especially unsure about the Options.td change - I just kind of guessed here.

Partially fixes #59552 - opting for -fuse-lipo=llvm-lipo rather than -fuse-llvm-darwin-tools since it solves my use case, and I figure that if -fuse-llvm-darwin-tools is eventually added, it'll still be nice to have the fine-grained control with -fuse-lipo.


My use case is that I'm cross compiling from Windows to Mac (creating an arm/x86 dylib), so I don't have the native lipo.

Additionally, the binaries included in the release file LLVM-19.1.0-Windows-X64.tar.xz only includes llvm-lipo.exe, no lipo.exe alias/link, so clang fails to find lipo when making a universal dylib. The release file LLVM-19.1.6-win64.exe does not include llvm-lipo.exe at all. I'm going to look into including that next.


Full diff: https://github.com/llvm/llvm-project/pull/121231.diff

2 Files Affected:

  • (modified) clang/include/clang/Driver/Options.td (+1)
  • (modified) clang/lib/Driver/ToolChains/Darwin.cpp (+2-1)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index d922709db17786..6cd23de87bacde 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -6654,6 +6654,7 @@ def fbinutils_version_EQ : Joined<["-"], "fbinutils-version=">,
 def fuse_ld_EQ : Joined<["-"], "fuse-ld=">, Group<f_Group>,
   Flags<[LinkOption]>, Visibility<[ClangOption, FlangOption, CLOption]>;
 def ld_path_EQ : Joined<["--"], "ld-path=">, Group<Link_Group>;
+def fuse_lipo_EQ : Joined<["-"], "fuse-lipo=">, Group<f_clang_Group>, Flags<[LinkOption]>;
 
 defm align_labels : BooleanFFlag<"align-labels">, Group<clang_ignored_gcc_optimization_f_Group>;
 def falign_labels_EQ : Joined<["-"], "falign-labels=">, Group<clang_ignored_gcc_optimization_f_Group>;
diff --git a/clang/lib/Driver/ToolChains/Darwin.cpp b/clang/lib/Driver/ToolChains/Darwin.cpp
index 4105d38d15d7d8..c23f6830b8c764 100644
--- a/clang/lib/Driver/ToolChains/Darwin.cpp
+++ b/clang/lib/Driver/ToolChains/Darwin.cpp
@@ -910,7 +910,8 @@ void darwin::Lipo::ConstructJob(Compilation &C, const JobAction &JA,
     CmdArgs.push_back(II.getFilename());
   }
 
-  const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath("lipo"));
+  std::string LipoName = std::string(Args.getLastArgValue(options::OPT_fuse_lipo_EQ, "lipo"));
+  const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath(LipoName.c_str()));
   C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
                                          Exec, CmdArgs, Inputs, Output));
 }

@llvmbot
Copy link
Member

llvmbot commented Dec 27, 2024

@llvm/pr-subscribers-clang

Author: Ashley Hauck (khyperia)

Changes

This is my first LLVM PR! Please feel free to provide feedback/etc. - I am especially unsure about the Options.td change - I just kind of guessed here.

Partially fixes #59552 - opting for -fuse-lipo=llvm-lipo rather than -fuse-llvm-darwin-tools since it solves my use case, and I figure that if -fuse-llvm-darwin-tools is eventually added, it'll still be nice to have the fine-grained control with -fuse-lipo.


My use case is that I'm cross compiling from Windows to Mac (creating an arm/x86 dylib), so I don't have the native lipo.

Additionally, the binaries included in the release file LLVM-19.1.0-Windows-X64.tar.xz only includes llvm-lipo.exe, no lipo.exe alias/link, so clang fails to find lipo when making a universal dylib. The release file LLVM-19.1.6-win64.exe does not include llvm-lipo.exe at all. I'm going to look into including that next.


Full diff: https://github.com/llvm/llvm-project/pull/121231.diff

2 Files Affected:

  • (modified) clang/include/clang/Driver/Options.td (+1)
  • (modified) clang/lib/Driver/ToolChains/Darwin.cpp (+2-1)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index d922709db17786..6cd23de87bacde 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -6654,6 +6654,7 @@ def fbinutils_version_EQ : Joined<["-"], "fbinutils-version=">,
 def fuse_ld_EQ : Joined<["-"], "fuse-ld=">, Group<f_Group>,
   Flags<[LinkOption]>, Visibility<[ClangOption, FlangOption, CLOption]>;
 def ld_path_EQ : Joined<["--"], "ld-path=">, Group<Link_Group>;
+def fuse_lipo_EQ : Joined<["-"], "fuse-lipo=">, Group<f_clang_Group>, Flags<[LinkOption]>;
 
 defm align_labels : BooleanFFlag<"align-labels">, Group<clang_ignored_gcc_optimization_f_Group>;
 def falign_labels_EQ : Joined<["-"], "falign-labels=">, Group<clang_ignored_gcc_optimization_f_Group>;
diff --git a/clang/lib/Driver/ToolChains/Darwin.cpp b/clang/lib/Driver/ToolChains/Darwin.cpp
index 4105d38d15d7d8..c23f6830b8c764 100644
--- a/clang/lib/Driver/ToolChains/Darwin.cpp
+++ b/clang/lib/Driver/ToolChains/Darwin.cpp
@@ -910,7 +910,8 @@ void darwin::Lipo::ConstructJob(Compilation &C, const JobAction &JA,
     CmdArgs.push_back(II.getFilename());
   }
 
-  const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath("lipo"));
+  std::string LipoName = std::string(Args.getLastArgValue(options::OPT_fuse_lipo_EQ, "lipo"));
+  const char *Exec = Args.MakeArgString(getToolChain().GetProgramPath(LipoName.c_str()));
   C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
                                          Exec, CmdArgs, Inputs, Output));
 }

Copy link

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Discourse for more information.

Copy link

github-actions bot commented Dec 27, 2024

✅ With the latest revision this PR passed the C/C++ code formatter.

Copy link
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR! This change needs new tests to be added.

Comment on lines 913 to 916
std::string LipoName =
std::string(Args.getLastArgValue(options::OPT_fuse_lipo_EQ, "lipo"));
const char *Exec =
Args.MakeArgString(getToolChain().GetProgramPath(LipoName.c_str()));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems a bit much to create a throwaway std::string here. Something like this should work:

Suggested change
std::string LipoName =
std::string(Args.getLastArgValue(options::OPT_fuse_lipo_EQ, "lipo"));
const char *Exec =
Args.MakeArgString(getToolChain().GetProgramPath(LipoName.c_str()));
StringRef LipoName =
Args.getLastArgValue(options::OPT_fuse_lipo_EQ, "lipo");
const char *Exec =
Args.MakeArgString(getToolChain().GetProgramPath(LipoName.data()));

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, StringRef::data() is documented as "data - Get a pointer to the start of the string (which may not be null terminated)". Because we're using it as a null-terminated string, I thought that making a copy is necessary to ensure it's null terminated, in case getLastArgValue ever changes to not return a null-terminated string. Depending on it always returning a null terminated string seems like what a lot of other code does already, though, I'll make that change.

@khyperia
Copy link
Contributor Author

@carlocab I've removed the temporary std::string and added a guess at basic tests. Let me know if there's additional tests in particular that you're thinking of!

Copy link
Collaborator

@cachemeifyoucan cachemeifyoucan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM in general with a comment in test.

For discussion. Is it better if the option supplies the full path to lipo or just the name? Full path seems to be easy to use, but might deserve a warning if the tool doesn't exist.
If just the name, it might be better to rename the option to something like -fuse-lipo-program=.

@khyperia
Copy link
Contributor Author

khyperia commented Jan 2, 2025

LGTM in general with a comment in test.

For discussion. Is it better if the option supplies the full path to lipo or just the name? Full path seems to be easy to use, but might deserve a warning if the tool doesn't exist. If just the name, it might be better to rename the option to something like -fuse-lipo-program=.

As a prefix: I am a new contributor, if you or someone else experienced has an opinion here I will gladly blindly follow it.

I tried to take precedence from -fuse-ld and --ld-path, so -fuse-lipo takes a name, and a hypothetical future --lipo-path would be a full path. Perhaps these are only named this way due to legacy compatibility though, and new naming conventions should be something else, fuse-lipo-program as you say? I can't find any existing option that ends with -program=, though. I don't know the history and context here. Let me know what I should do!

Nit: Add a test case to check when the flag is not supplied.

Pushed

@khyperia
Copy link
Contributor Author

khyperia commented Jan 2, 2025

(For what it's worth, just confirming that the current code doesn't work with a full path - it surprisingly kind of almost does, but not quite, due to quirks of GetProgramPath)

@cachemeifyoucan
Copy link
Collaborator

I tried to take precedence from -fuse-ld and --ld-path, so -fuse-lipo takes a name, and a hypothetical future --lipo-path would be a full path. Perhaps these are only named this way due to legacy compatibility though, and new naming conventions should be something else, fuse-lipo-program as you say? I can't find any existing option that ends with -program=, though. I don't know the history and context here. Let me know what I should do!

Make sense to me.

@Andarwinux
Copy link
Contributor

It is better to add a CLANG_DEFAULT_LIPO cmake option to match the CLANG_DEFAULT_LINKER for -fuse-ld.

@khyperia
Copy link
Contributor Author

khyperia commented Jan 6, 2025

It is better to add a CLANG_DEFAULT_LIPO cmake option to match the CLANG_DEFAULT_LINKER for -fuse-ld.

Sure, that's a possibility, but in my opinion can be done in a future PR. There's a lot of additional features that can be done in the future:

  • CLANG_DEFAULT_LIPO cmake option
  • --lipo-path commandline argument
  • -fuse-llvm-darwin-tools, described in the original issue
  • better error handling when the lipo tool isn't found
  • probably more...

I have a motivation and use case for -fuse-lipo and I'd like to get this in first, and we can talk about additional features later - there's no backcompat issues/etc. that require these additional features to be done within this PR.

@khyperia
Copy link
Contributor Author

@cachemeifyoucan Could you possibly help me get this merged? I'm not sure if your approval is enough, or if more reviewers are needed - and if so, who. Are you able to merge this if just your approval is enough?

@cachemeifyoucan
Copy link
Collaborator

@khyperia Sure, I can help you merge. Code change looks good as I don't see any open review feedback and no objection from adding the option.

Can you please update the PR message to be a better formatted commit message? The squash commit will use that as commit message, and I don't want to rewrite that on your behalf. More specifically, it will read much better if it talks about what is the option added and how to use it, rather formulated around your use case. Just ping me again once you updated the message.

@khyperia
Copy link
Contributor Author

@cachemeifyoucan thank you, done! (feel free to rewrite it as well, if you want or have any nitpicks)

@cachemeifyoucan cachemeifyoucan changed the title Add -fuse-lipo option [clang][Driver] Add -fuse-lipo option Jan 16, 2025
@cachemeifyoucan
Copy link
Collaborator

I updated the title and PR message. Let me know if that looks ok for you.

@khyperia
Copy link
Contributor Author

Looks great, thanks!

@cachemeifyoucan cachemeifyoucan merged commit 9be358f into llvm:main Jan 16, 2025
8 checks passed
@cachemeifyoucan
Copy link
Collaborator

Merged. Thanks for your contribution!

Copy link

@khyperia Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

@khyperia khyperia deleted the fuse-lipo branch January 16, 2025 21:58
@sztomi
Copy link

sztomi commented Jan 17, 2025

Thank you! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang Clang issues not falling into any other category
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow selecting llvm-lipo (and possibly other llvm tools) before the host toolchain equivalents
6 participants