Add support for Windows hot-patching #138972
Conversation
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
llvmbot
added
clang
|
@llvm/pr-subscribers-platform-windows @llvm/pr-subscribers-clang-driver Author: None (sivadeilra) ChangesThis PR adds some of the support needed for Windows hot-patching. Windows implements a form of hot-patching. This allows patches to be applied to Windows apps, drivers, and the kernel, without rebooting or restarting any of these components. Hot-patching is a complex technology and requires coordination between the OS, compilers, linkers, and additional tools. This PR adds support to Clang and LLVM for part of the hot-patching process. It enables LLVM to generate the required code changes and to generate CodeView symbols which identify hot-patched functions. The PR provides new command-line arguments to Clang which allow developers to identify the list of functions that need to be hot-patched. This PR also allows LLVM to directly receive the list of functions to be modified, so that language front-ends which have not yet been modified (such as Rust) can still make use of hot-patching. This PR:
Although the flags are redundant between Clang and LLVM, this allows additional languages (such as Rust) to take advantage of hot-patching support before they have been modified to generate the required attributes. Credit to @dpaoliello, who wrote the original form of this patch. Patch is 39.25 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/138972.diff 34 Files Affected:
diff --git a/clang/include/clang/Basic/CodeGenOptions.h b/clang/include/clang/Basic/CodeGenOptions.h
index e39a73bdb13ac..3e102d47312ec 100644
--- a/clang/include/clang/Basic/CodeGenOptions.h
+++ b/clang/include/clang/Basic/CodeGenOptions.h
@@ -493,6 +493,14 @@ class CodeGenOptions : public CodeGenOptionsBase {
/// The name of a file to use with \c .secure_log_unique directives.
std::string AsSecureLogFile;
+ /// The name of a file that contains functions which will be compiled for
+ /// hotpatching. See -fms-hot-patch-functions-file.
+ std::string MSHotPatchFunctionsFile;
+
+ /// A list of functions which will be compiled for hotpatching.
+ /// See -fms-hot-patch-functions-list.
+ std::vector<std::string> MSHotPatchFunctionsList;
+
public:
// Define accessors/mutators for code generation options of enumeration type.
#define CODEGENOPT(Name, Bits, Default)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 30ea75bb108d5..062c8544746c0 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3793,6 +3793,20 @@ def fms_hotpatch : Flag<["-"], "fms-hotpatch">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Ensure that all functions can be hotpatched at runtime">,
MarshallingInfoFlag<CodeGenOpts<"HotPatch">>;
+def fms_hotpatch_functions_file
+ : Joined<["-"], "fms-hotpatch-functions-file=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoString<CodeGenOpts<"MSHotPatchFunctionsFile">>,
+ HelpText<"Path to a file that contains a list of mangled symbol names of "
+ "functions that should be hot-patched">;
+def fms_hotpatch_functions_list
+ : CommaJoined<["-"], "fms-hotpatch-functions-list=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoStringVector<CodeGenOpts<"MSHotPatchFunctionsList">>,
+ HelpText<"List of mangled symbol names of functions that should be "
+ "hot-patched">;
def fpcc_struct_return : Flag<["-"], "fpcc-struct-return">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>,
HelpText<"Override the default ABI to return all structs on the stack">;
diff --git a/clang/lib/CodeGen/CGCall.cpp b/clang/lib/CodeGen/CGCall.cpp
index 2f1c7699d27c3..14dcdbd79e577 100644
--- a/clang/lib/CodeGen/CGCall.cpp
+++ b/clang/lib/CodeGen/CGCall.cpp
@@ -2636,6 +2636,15 @@ void CodeGenModule::ConstructAttributeList(StringRef Name,
// CPU/feature overrides. addDefaultFunctionDefinitionAttributes
// handles these separately to set them based on the global defaults.
GetCPUAndFeaturesAttributes(CalleeInfo.getCalleeDecl(), FuncAttrs);
+
+ // Windows hotpatching support
+ if (!MSHotPatchFunctions.empty()) {
+ bool IsHotPatched = std::binary_search(MSHotPatchFunctions.begin(),
+ MSHotPatchFunctions.end(), Name);
+ if (IsHotPatched) {
+ FuncAttrs.addAttribute(llvm::Attribute::MarkedForWindowsHotPatching);
+ }
+ }
}
// Collect attributes from arguments and return values.
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index c27817604f6ca..36a1f838ba414 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -453,6 +453,37 @@ CodeGenModule::CodeGenModule(ASTContext &C,
if (Context.getTargetInfo().getTriple().getArch() == llvm::Triple::x86)
getModule().addModuleFlag(llvm::Module::Error, "NumRegisterParameters",
CodeGenOpts.NumRegisterParameters);
+
+ // If there are any functions that are marked for Windows hot-patching,
+ // then build the list of functions now.
+ if (!CGO.MSHotPatchFunctionsFile.empty() ||
+ !CGO.MSHotPatchFunctionsList.empty()) {
+ if (!CGO.MSHotPatchFunctionsFile.empty()) {
+ auto BufOrErr = llvm::MemoryBuffer::getFile(CGO.MSHotPatchFunctionsFile);
+ if (BufOrErr) {
+ const llvm::MemoryBuffer &FileBuffer = **BufOrErr;
+ for (llvm::line_iterator I(FileBuffer.getMemBufferRef(), true), E;
+ I != E; ++I) {
+ this->MSHotPatchFunctions.push_back(std::string{*I});
+ }
+ } else {
+ auto &DE = Context.getDiagnostics();
+ unsigned DiagID =
+ DE.getCustomDiagID(DiagnosticsEngine::Error,
+ "failed to open hotpatch functions file "
+ "(-fms-hotpatch-functions-file): %0 : %1");
+ DE.Report(DiagID) << CGO.MSHotPatchFunctionsFile
+ << BufOrErr.getError().message();
+ }
+ }
+
+ for (const auto &FuncName : CGO.MSHotPatchFunctionsList) {
+ this->MSHotPatchFunctions.push_back(FuncName);
+ }
+
+ std::sort(this->MSHotPatchFunctions.begin(),
+ this->MSHotPatchFunctions.end());
+ }
}
CodeGenModule::~CodeGenModule() {}
diff --git a/clang/lib/CodeGen/CodeGenModule.h b/clang/lib/CodeGen/CodeGenModule.h
index 1db5c3bc4e4ef..45411cb95b7d2 100644
--- a/clang/lib/CodeGen/CodeGenModule.h
+++ b/clang/lib/CodeGen/CodeGenModule.h
@@ -678,6 +678,11 @@ class CodeGenModule : public CodeGenTypeCache {
AtomicOptions AtomicOpts;
+ // A set of functions which should be hot-patched; see
+ // -fms-hotpatch-functions-file (and -list). This will nearly always be empty.
+ // The list is sorted for binary-searching.
+ std::vector<std::string> MSHotPatchFunctions;
+
public:
CodeGenModule(ASTContext &C, IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS,
const HeaderSearchOptions &headersearchopts,
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index f87549baff5e1..a733c50e56b89 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -6945,6 +6945,16 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch);
+ if (Arg *A = Args.getLastArg(options::OPT_fms_hotpatch_functions_file)) {
+ Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch_functions_file);
+ }
+
+ for (const auto &A :
+ Args.getAllArgValues(options::OPT_fms_hotpatch_functions_list)) {
+ CmdArgs.push_back(
+ Args.MakeArgString("-fms-hotpatch-functions-list=" + Twine(A)));
+ }
+
if (TC.SupportsProfiling()) {
Args.AddLastArg(CmdArgs, options::OPT_pg);
diff --git a/clang/test/CodeGen/ms-hotpatch-bad-file.c b/clang/test/CodeGen/ms-hotpatch-bad-file.c
new file mode 100644
index 0000000000000..55304488175df
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-bad-file.c
@@ -0,0 +1,16 @@
+// This verifies that we correctly handle a -fms-hotpatch-functions-file argument that points
+// to a missing file.
+//
+// RUN: not %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/this-file-is-intentionally-missing-do-not-create-it.txt /Fo%t.obj %s 2>&1 | FileCheck %s
+// CHECK: failed to open hotpatch functions file
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch-functions.txt b/clang/test/CodeGen/ms-hotpatch-functions.txt
new file mode 100644
index 0000000000000..2512ba3881c66
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-functions.txt
@@ -0,0 +1 @@
+this_gets_hotpatched
diff --git a/clang/test/CodeGen/ms-hotpatch-lto.c b/clang/test/CodeGen/ms-hotpatch-lto.c
new file mode 100644
index 0000000000000..290f7695b02f6
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-lto.c
@@ -0,0 +1,18 @@
+// This verifies that hotpatch function attributes are correctly propagated through LLVM IR when compiling with LTO.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt -flto /Fo%t.bc %s
+// RUN: llvm-dis %t.bc -o - | FileCheck %s
+//
+// CHECK: ; Function Attrs: marked_for_windows_hot_patching mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_gets_hotpatched() local_unnamed_addr #0 !dbg !13 {
+//
+// CHECK: ; Function Attrs: mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_does_not_get_hotpatched() local_unnamed_addr #1 !dbg !19 {
+
+int __declspec(noinline) this_gets_hotpatched() {
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch.c b/clang/test/CodeGen/ms-hotpatch.c
new file mode 100644
index 0000000000000..c11b8ffc2f87a
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch.c
@@ -0,0 +1,17 @@
+// This verifies that hotpatch function attributes are correctly propagated when compiling directly to OBJ.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt /Fo%t.obj %s
+// RUN: llvm-readobj --codeview %t.obj | FileCheck %s
+// CHECK: Kind: S_HOTPATCHFUNC (0x1169)
+// CHECK-NEXT: Function: this_gets_hotpatched
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index 9317e6aa793de..d32191d51023a 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -793,6 +793,8 @@ enum AttributeKindCodes {
ATTR_KIND_NO_DIVERGENCE_SOURCE = 100,
ATTR_KIND_SANITIZE_TYPE = 101,
ATTR_KIND_CAPTURES = 102,
+ ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION = 103,
+ ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING = 104,
};
enum ComdatSelectionKindCodes {
diff --git a/llvm/include/llvm/CodeGen/Passes.h b/llvm/include/llvm/CodeGen/Passes.h
index d214ab9306c2f..c4163617da5ae 100644
--- a/llvm/include/llvm/CodeGen/Passes.h
+++ b/llvm/include/llvm/CodeGen/Passes.h
@@ -617,6 +617,9 @@ namespace llvm {
/// Lowers KCFI operand bundles for indirect calls.
FunctionPass *createKCFIPass();
+
+ /// Creates Windows Hot Patch pass. \see WindowsHotPatch.cpp
+ ModulePass *createWindowsHotPatch();
} // End llvm namespace
#endif
diff --git a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
index 9d85acc49fa02..b38bdb482df43 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
+++ b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
@@ -256,6 +256,8 @@ SYMBOL_RECORD_ALIAS(S_GTHREAD32 , 0x1113, GlobalTLS, ThreadLocalDataSym)
SYMBOL_RECORD(S_UNAMESPACE , 0x1124, UsingNamespaceSym)
SYMBOL_RECORD(S_ANNOTATION , 0x1019, AnnotationSym)
+SYMBOL_RECORD(S_HOTPATCHFUNC , 0x1169, HotPatchFuncSym)
+
#undef CV_SYMBOL
#undef SYMBOL_RECORD
#undef SYMBOL_RECORD_ALIAS
diff --git a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
index 3cce40dcf735a..de80d37999890 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
+++ b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
@@ -176,6 +176,21 @@ class CallerSym : public SymbolRecord {
uint32_t RecordOffset = 0;
};
+class HotPatchFuncSym : public SymbolRecord {
+public:
+ explicit HotPatchFuncSym(SymbolRecordKind Kind) : SymbolRecord(Kind) {}
+ HotPatchFuncSym(uint32_t RecordOffset)
+ : SymbolRecord(SymbolRecordKind::HotPatchFuncSym),
+ RecordOffset(RecordOffset) {}
+
+ // This is an ItemID in the IPI stream, which points to an LF_FUNC_ID or
+ // LF_MFUNC_ID record.
+ TypeIndex Function;
+ StringRef Name;
+
+ uint32_t RecordOffset = 0;
+};
+
struct DecodedAnnotation {
StringRef Name;
ArrayRef<uint8_t> Bytes;
diff --git a/llvm/include/llvm/IR/Attributes.td b/llvm/include/llvm/IR/Attributes.td
index fb94926043fc7..d2766a5346946 100644
--- a/llvm/include/llvm/IR/Attributes.td
+++ b/llvm/include/llvm/IR/Attributes.td
@@ -389,6 +389,17 @@ def CoroDestroyOnlyWhenComplete : EnumAttr<"coro_only_destroy_when_complete", In
/// pipeline to perform elide on the call or invoke instruction.
def CoroElideSafe : EnumAttr<"coro_elide_safe", IntersectPreserve, [FnAttr]>;
+/// Function is marked for Windows Hot Patching
+def MarkedForWindowsHotPatching
+ : EnumAttr<"marked_for_windows_hot_patching", IntersectPreserve, [FnAttr]>;
+
+/// Global variable should not be accessed through a "__ref_" global variable in
+/// a hot patching function This attribute is applied to the global variable
+/// decl, not the hotpatched function.
+def AllowDirectAccessInHotPatchFunction
+ : EnumAttr<"allow_direct_access_in_hot_patch_function",
+ IntersectPreserve, []>;
+
/// Target-independent string attributes.
def LessPreciseFPMAD : StrBoolAttr<"less-precise-fpmad">;
def NoInfsFPMath : StrBoolAttr<"no-infs-fp-math">;
diff --git a/llvm/include/llvm/InitializePasses.h b/llvm/include/llvm/InitializePasses.h
index bff0526d4177a..1a067e6526eb2 100644
--- a/llvm/include/llvm/InitializePasses.h
+++ b/llvm/include/llvm/InitializePasses.h
@@ -322,6 +322,7 @@ void initializeVirtRegMapWrapperLegacyPass(PassRegistry &);
void initializeVirtRegRewriterLegacyPass(PassRegistry &);
void initializeWasmEHPreparePass(PassRegistry &);
void initializeWinEHPreparePass(PassRegistry &);
+void initializeWindowsHotPatchPass(PassRegistry &);
void initializeWriteBitcodePassPass(PassRegistry &);
void initializeXRayInstrumentationLegacyPass(PassRegistry &);
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 4074ed65885c7..46d1f883d5543 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2248,6 +2248,10 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoExt;
case bitc::ATTR_KIND_CAPTURES:
return Attribute::Captures;
+ case bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION:
+ return Attribute::AllowDirectAccessInHotPatchFunction;
+ case bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING:
+ return Attribute::MarkedForWindowsHotPatching;
}
}
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 27ada0ddcd831..7e418a522a39e 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -919,6 +919,10 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_EXT;
case Attribute::Captures:
return bitc::ATTR_KIND_CAPTURES;
+ case Attribute::AllowDirectAccessInHotPatchFunction:
+ return bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION;
+ case Attribute::MarkedForWindowsHotPatching:
+ return bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING;
case Attribute::EndAttrKinds:
llvm_unreachable("Can not encode end-attribute kinds marker.");
case Attribute::None:
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
index df4e48571692c..c643bd269f3b1 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
@@ -627,8 +627,6 @@ void CodeViewDebug::beginModule(Module *M) {
CurrentSourceLanguage = MapDWLangToCVLang(CU->getSourceLanguage());
- collectGlobalVariableInfo();
-
// Check if we should emit type record hashes.
ConstantInt *GH =
mdconst::extract_or_null<ConstantInt>(M->getModuleFlag("CodeViewGHash"));
@@ -639,6 +637,8 @@ void CodeViewDebug::endModule() {
if (!Asm || !Asm->hasDebugInfo())
return;
+ collectGlobalVariableInfo();
+
// The COFF .debug$S section consists of several subsections, each starting
// with a 4-byte control code (e.g. 0xF1, 0xF2, etc) and then a 4-byte length
// of the payload followed by the payload itself. The subsections are 4-byte
@@ -653,6 +653,8 @@ void CodeViewDebug::endModule() {
emitCompilerInformation();
endCVSubsection(CompilerInfo);
+ emitHotPatchInformation();
+
emitInlineeLinesSubsection();
// Emit per-function debug information.
@@ -807,6 +809,32 @@ void CodeViewDebug::emitObjName() {
endSymbolRecord(CompilerEnd);
}
+void CodeViewDebug::emitHotPatchInformation() {
+ MCSymbol *hotPatchInfo = nullptr;
+ for (const auto &F : MMI->getModule()->functions()) {
+ if (!F.isDeclarationForLinker() &&
+ F.hasFnAttribute(Attribute::MarkedForWindowsHotPatching)) {
+ if (hotPatchInfo == nullptr) {
+ hotPatchInfo = beginCVSubsection(DebugSubsectionKind::Symbols);
+ }
+ MCSymbol *HotPatchEnd = beginSymbolRecord(SymbolKind::S_HOTPATCHFUNC);
+ auto *SP = F.getSubprogram();
+ OS.AddComment("Function");
+ OS.emitInt32(getFuncIdForSubprogram(SP).getIndex());
+ OS.AddComment("Name");
+ llvm::StringRef Name = SP->getLinkageName();
+ if (Name.empty()) {
+ Name = F.getName();
+ }
+ emitNullTerminatedSymbolName(OS, Name);
+ endSymbolRecord(HotPatchEnd);
+ }
+ }
+ if (hotPatchInfo != nullptr) {
+ endCVSubsection(hotPatchInfo);
+ }
+}
+
namespace {
struct Version {
int Part[4];
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
index 7a138a0332b6d..3cc47b79b5b32 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
@@ -333,6 +333,8 @@ class LLVM_LIBRARY_VISIBILITY CodeViewDebug : public DebugHandlerBase {
void emitCompilerInformation();
+ void emitHotPatchInformation();
+
void emitBuildInfo();
void emitInlineeLinesSubsection();
diff --git a/llvm/lib/CodeGen/CMakeLists.txt b/llvm/lib/CodeGen/CMakeLists.txt
index 5dd6413431255..f18f085b0450b 100644
--- a/llvm/lib/CodeGen/CMakeLists.txt
+++ b/llvm/lib/CodeGen/CMakeLists.txt
@@ -250,6 +250,7 @@ add_llvm_component_library(LLVMCodeGen
VirtRegMap.cpp
WasmEHPrepare.cpp
WindowScheduler.cpp
+ WindowsHotPatch.cpp
WinEHPrepare.cpp
XRayInstrumentation.cpp
${GeneratedMLSources}
diff --git a/llvm/lib/CodeGen/TargetPassConfig.cpp b/llvm/lib/CodeGen/TargetPassConfig.cpp
index 0095ce3d96277..1d93efc21fbab 100644
--- a/llvm/lib/CodeGen/TargetPassConfig.cpp
+++ b/llvm/lib/CodeGen/TargetPassConfig.cpp
@@ -893,6 +893,9 @@ void TargetPassConfig::addIRPasses() {
if (EnableGlobalMergeFunc)
addPass(createGlobalMergeFuncPass());
+
+ if (TM->getTargetTriple().isOSBinFormatCOFF())
+ addPass(createWindowsHotPatch());
}
/// Turn exception handling constructs into something the code generators can
diff --git a/llvm/lib/CodeGen/WindowsHotPatch.cpp b/llvm/lib/CodeGen/WindowsHotPatch.cpp
new file mode 100644
index 0000000000000..853670d64d28f
--- /dev/null
+++ b/llvm/lib/CodeGen/WindowsHotPatch.cpp
@@ -0,0 +1,251 @@
+//===------ WindowsHotPatch.cpp - Support for Windows hotpatching ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Marks functions with the `marked_for_windows_hot_patching` attribute.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallSet.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/Attributes.h"
+#include "llvm/IR/DIBuilder.h"
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/Module.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/LineIterator.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "windows-hot-patch"
+
+// A file containing list of mangled function names to mark for hot patching.
+static cl:...
[truncated]
|
|
@llvm/pr-subscribers-debuginfo Author: None (sivadeilra) ChangesThis PR adds some of the support needed for Windows hot-patching. Windows implements a form of hot-patching. This allows patches to be applied to Windows apps, drivers, and the kernel, without rebooting or restarting any of these components. Hot-patching is a complex technology and requires coordination between the OS, compilers, linkers, and additional tools. This PR adds support to Clang and LLVM for part of the hot-patching process. It enables LLVM to generate the required code changes and to generate CodeView symbols which identify hot-patched functions. The PR provides new command-line arguments to Clang which allow developers to identify the list of functions that need to be hot-patched. This PR also allows LLVM to directly receive the list of functions to be modified, so that language front-ends which have not yet been modified (such as Rust) can still make use of hot-patching. This PR:
Although the flags are redundant between Clang and LLVM, this allows additional languages (such as Rust) to take advantage of hot-patching support before they have been modified to generate the required attributes. Credit to @dpaoliello, who wrote the original form of this patch. Patch is 39.25 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/138972.diff 34 Files Affected:
diff --git a/clang/include/clang/Basic/CodeGenOptions.h b/clang/include/clang/Basic/CodeGenOptions.h
index e39a73bdb13ac..3e102d47312ec 100644
--- a/clang/include/clang/Basic/CodeGenOptions.h
+++ b/clang/include/clang/Basic/CodeGenOptions.h
@@ -493,6 +493,14 @@ class CodeGenOptions : public CodeGenOptionsBase {
/// The name of a file to use with \c .secure_log_unique directives.
std::string AsSecureLogFile;
+ /// The name of a file that contains functions which will be compiled for
+ /// hotpatching. See -fms-hot-patch-functions-file.
+ std::string MSHotPatchFunctionsFile;
+
+ /// A list of functions which will be compiled for hotpatching.
+ /// See -fms-hot-patch-functions-list.
+ std::vector<std::string> MSHotPatchFunctionsList;
+
public:
// Define accessors/mutators for code generation options of enumeration type.
#define CODEGENOPT(Name, Bits, Default)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 30ea75bb108d5..062c8544746c0 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3793,6 +3793,20 @@ def fms_hotpatch : Flag<["-"], "fms-hotpatch">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Ensure that all functions can be hotpatched at runtime">,
MarshallingInfoFlag<CodeGenOpts<"HotPatch">>;
+def fms_hotpatch_functions_file
+ : Joined<["-"], "fms-hotpatch-functions-file=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoString<CodeGenOpts<"MSHotPatchFunctionsFile">>,
+ HelpText<"Path to a file that contains a list of mangled symbol names of "
+ "functions that should be hot-patched">;
+def fms_hotpatch_functions_list
+ : CommaJoined<["-"], "fms-hotpatch-functions-list=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoStringVector<CodeGenOpts<"MSHotPatchFunctionsList">>,
+ HelpText<"List of mangled symbol names of functions that should be "
+ "hot-patched">;
def fpcc_struct_return : Flag<["-"], "fpcc-struct-return">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>,
HelpText<"Override the default ABI to return all structs on the stack">;
diff --git a/clang/lib/CodeGen/CGCall.cpp b/clang/lib/CodeGen/CGCall.cpp
index 2f1c7699d27c3..14dcdbd79e577 100644
--- a/clang/lib/CodeGen/CGCall.cpp
+++ b/clang/lib/CodeGen/CGCall.cpp
@@ -2636,6 +2636,15 @@ void CodeGenModule::ConstructAttributeList(StringRef Name,
// CPU/feature overrides. addDefaultFunctionDefinitionAttributes
// handles these separately to set them based on the global defaults.
GetCPUAndFeaturesAttributes(CalleeInfo.getCalleeDecl(), FuncAttrs);
+
+ // Windows hotpatching support
+ if (!MSHotPatchFunctions.empty()) {
+ bool IsHotPatched = std::binary_search(MSHotPatchFunctions.begin(),
+ MSHotPatchFunctions.end(), Name);
+ if (IsHotPatched) {
+ FuncAttrs.addAttribute(llvm::Attribute::MarkedForWindowsHotPatching);
+ }
+ }
}
// Collect attributes from arguments and return values.
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index c27817604f6ca..36a1f838ba414 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -453,6 +453,37 @@ CodeGenModule::CodeGenModule(ASTContext &C,
if (Context.getTargetInfo().getTriple().getArch() == llvm::Triple::x86)
getModule().addModuleFlag(llvm::Module::Error, "NumRegisterParameters",
CodeGenOpts.NumRegisterParameters);
+
+ // If there are any functions that are marked for Windows hot-patching,
+ // then build the list of functions now.
+ if (!CGO.MSHotPatchFunctionsFile.empty() ||
+ !CGO.MSHotPatchFunctionsList.empty()) {
+ if (!CGO.MSHotPatchFunctionsFile.empty()) {
+ auto BufOrErr = llvm::MemoryBuffer::getFile(CGO.MSHotPatchFunctionsFile);
+ if (BufOrErr) {
+ const llvm::MemoryBuffer &FileBuffer = **BufOrErr;
+ for (llvm::line_iterator I(FileBuffer.getMemBufferRef(), true), E;
+ I != E; ++I) {
+ this->MSHotPatchFunctions.push_back(std::string{*I});
+ }
+ } else {
+ auto &DE = Context.getDiagnostics();
+ unsigned DiagID =
+ DE.getCustomDiagID(DiagnosticsEngine::Error,
+ "failed to open hotpatch functions file "
+ "(-fms-hotpatch-functions-file): %0 : %1");
+ DE.Report(DiagID) << CGO.MSHotPatchFunctionsFile
+ << BufOrErr.getError().message();
+ }
+ }
+
+ for (const auto &FuncName : CGO.MSHotPatchFunctionsList) {
+ this->MSHotPatchFunctions.push_back(FuncName);
+ }
+
+ std::sort(this->MSHotPatchFunctions.begin(),
+ this->MSHotPatchFunctions.end());
+ }
}
CodeGenModule::~CodeGenModule() {}
diff --git a/clang/lib/CodeGen/CodeGenModule.h b/clang/lib/CodeGen/CodeGenModule.h
index 1db5c3bc4e4ef..45411cb95b7d2 100644
--- a/clang/lib/CodeGen/CodeGenModule.h
+++ b/clang/lib/CodeGen/CodeGenModule.h
@@ -678,6 +678,11 @@ class CodeGenModule : public CodeGenTypeCache {
AtomicOptions AtomicOpts;
+ // A set of functions which should be hot-patched; see
+ // -fms-hotpatch-functions-file (and -list). This will nearly always be empty.
+ // The list is sorted for binary-searching.
+ std::vector<std::string> MSHotPatchFunctions;
+
public:
CodeGenModule(ASTContext &C, IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS,
const HeaderSearchOptions &headersearchopts,
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index f87549baff5e1..a733c50e56b89 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -6945,6 +6945,16 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch);
+ if (Arg *A = Args.getLastArg(options::OPT_fms_hotpatch_functions_file)) {
+ Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch_functions_file);
+ }
+
+ for (const auto &A :
+ Args.getAllArgValues(options::OPT_fms_hotpatch_functions_list)) {
+ CmdArgs.push_back(
+ Args.MakeArgString("-fms-hotpatch-functions-list=" + Twine(A)));
+ }
+
if (TC.SupportsProfiling()) {
Args.AddLastArg(CmdArgs, options::OPT_pg);
diff --git a/clang/test/CodeGen/ms-hotpatch-bad-file.c b/clang/test/CodeGen/ms-hotpatch-bad-file.c
new file mode 100644
index 0000000000000..55304488175df
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-bad-file.c
@@ -0,0 +1,16 @@
+// This verifies that we correctly handle a -fms-hotpatch-functions-file argument that points
+// to a missing file.
+//
+// RUN: not %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/this-file-is-intentionally-missing-do-not-create-it.txt /Fo%t.obj %s 2>&1 | FileCheck %s
+// CHECK: failed to open hotpatch functions file
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch-functions.txt b/clang/test/CodeGen/ms-hotpatch-functions.txt
new file mode 100644
index 0000000000000..2512ba3881c66
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-functions.txt
@@ -0,0 +1 @@
+this_gets_hotpatched
diff --git a/clang/test/CodeGen/ms-hotpatch-lto.c b/clang/test/CodeGen/ms-hotpatch-lto.c
new file mode 100644
index 0000000000000..290f7695b02f6
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-lto.c
@@ -0,0 +1,18 @@
+// This verifies that hotpatch function attributes are correctly propagated through LLVM IR when compiling with LTO.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt -flto /Fo%t.bc %s
+// RUN: llvm-dis %t.bc -o - | FileCheck %s
+//
+// CHECK: ; Function Attrs: marked_for_windows_hot_patching mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_gets_hotpatched() local_unnamed_addr #0 !dbg !13 {
+//
+// CHECK: ; Function Attrs: mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_does_not_get_hotpatched() local_unnamed_addr #1 !dbg !19 {
+
+int __declspec(noinline) this_gets_hotpatched() {
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch.c b/clang/test/CodeGen/ms-hotpatch.c
new file mode 100644
index 0000000000000..c11b8ffc2f87a
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch.c
@@ -0,0 +1,17 @@
+// This verifies that hotpatch function attributes are correctly propagated when compiling directly to OBJ.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt /Fo%t.obj %s
+// RUN: llvm-readobj --codeview %t.obj | FileCheck %s
+// CHECK: Kind: S_HOTPATCHFUNC (0x1169)
+// CHECK-NEXT: Function: this_gets_hotpatched
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index 9317e6aa793de..d32191d51023a 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -793,6 +793,8 @@ enum AttributeKindCodes {
ATTR_KIND_NO_DIVERGENCE_SOURCE = 100,
ATTR_KIND_SANITIZE_TYPE = 101,
ATTR_KIND_CAPTURES = 102,
+ ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION = 103,
+ ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING = 104,
};
enum ComdatSelectionKindCodes {
diff --git a/llvm/include/llvm/CodeGen/Passes.h b/llvm/include/llvm/CodeGen/Passes.h
index d214ab9306c2f..c4163617da5ae 100644
--- a/llvm/include/llvm/CodeGen/Passes.h
+++ b/llvm/include/llvm/CodeGen/Passes.h
@@ -617,6 +617,9 @@ namespace llvm {
/// Lowers KCFI operand bundles for indirect calls.
FunctionPass *createKCFIPass();
+
+ /// Creates Windows Hot Patch pass. \see WindowsHotPatch.cpp
+ ModulePass *createWindowsHotPatch();
} // End llvm namespace
#endif
diff --git a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
index 9d85acc49fa02..b38bdb482df43 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
+++ b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
@@ -256,6 +256,8 @@ SYMBOL_RECORD_ALIAS(S_GTHREAD32 , 0x1113, GlobalTLS, ThreadLocalDataSym)
SYMBOL_RECORD(S_UNAMESPACE , 0x1124, UsingNamespaceSym)
SYMBOL_RECORD(S_ANNOTATION , 0x1019, AnnotationSym)
+SYMBOL_RECORD(S_HOTPATCHFUNC , 0x1169, HotPatchFuncSym)
+
#undef CV_SYMBOL
#undef SYMBOL_RECORD
#undef SYMBOL_RECORD_ALIAS
diff --git a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
index 3cce40dcf735a..de80d37999890 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
+++ b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
@@ -176,6 +176,21 @@ class CallerSym : public SymbolRecord {
uint32_t RecordOffset = 0;
};
+class HotPatchFuncSym : public SymbolRecord {
+public:
+ explicit HotPatchFuncSym(SymbolRecordKind Kind) : SymbolRecord(Kind) {}
+ HotPatchFuncSym(uint32_t RecordOffset)
+ : SymbolRecord(SymbolRecordKind::HotPatchFuncSym),
+ RecordOffset(RecordOffset) {}
+
+ // This is an ItemID in the IPI stream, which points to an LF_FUNC_ID or
+ // LF_MFUNC_ID record.
+ TypeIndex Function;
+ StringRef Name;
+
+ uint32_t RecordOffset = 0;
+};
+
struct DecodedAnnotation {
StringRef Name;
ArrayRef<uint8_t> Bytes;
diff --git a/llvm/include/llvm/IR/Attributes.td b/llvm/include/llvm/IR/Attributes.td
index fb94926043fc7..d2766a5346946 100644
--- a/llvm/include/llvm/IR/Attributes.td
+++ b/llvm/include/llvm/IR/Attributes.td
@@ -389,6 +389,17 @@ def CoroDestroyOnlyWhenComplete : EnumAttr<"coro_only_destroy_when_complete", In
/// pipeline to perform elide on the call or invoke instruction.
def CoroElideSafe : EnumAttr<"coro_elide_safe", IntersectPreserve, [FnAttr]>;
+/// Function is marked for Windows Hot Patching
+def MarkedForWindowsHotPatching
+ : EnumAttr<"marked_for_windows_hot_patching", IntersectPreserve, [FnAttr]>;
+
+/// Global variable should not be accessed through a "__ref_" global variable in
+/// a hot patching function This attribute is applied to the global variable
+/// decl, not the hotpatched function.
+def AllowDirectAccessInHotPatchFunction
+ : EnumAttr<"allow_direct_access_in_hot_patch_function",
+ IntersectPreserve, []>;
+
/// Target-independent string attributes.
def LessPreciseFPMAD : StrBoolAttr<"less-precise-fpmad">;
def NoInfsFPMath : StrBoolAttr<"no-infs-fp-math">;
diff --git a/llvm/include/llvm/InitializePasses.h b/llvm/include/llvm/InitializePasses.h
index bff0526d4177a..1a067e6526eb2 100644
--- a/llvm/include/llvm/InitializePasses.h
+++ b/llvm/include/llvm/InitializePasses.h
@@ -322,6 +322,7 @@ void initializeVirtRegMapWrapperLegacyPass(PassRegistry &);
void initializeVirtRegRewriterLegacyPass(PassRegistry &);
void initializeWasmEHPreparePass(PassRegistry &);
void initializeWinEHPreparePass(PassRegistry &);
+void initializeWindowsHotPatchPass(PassRegistry &);
void initializeWriteBitcodePassPass(PassRegistry &);
void initializeXRayInstrumentationLegacyPass(PassRegistry &);
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 4074ed65885c7..46d1f883d5543 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2248,6 +2248,10 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoExt;
case bitc::ATTR_KIND_CAPTURES:
return Attribute::Captures;
+ case bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION:
+ return Attribute::AllowDirectAccessInHotPatchFunction;
+ case bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING:
+ return Attribute::MarkedForWindowsHotPatching;
}
}
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 27ada0ddcd831..7e418a522a39e 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -919,6 +919,10 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_EXT;
case Attribute::Captures:
return bitc::ATTR_KIND_CAPTURES;
+ case Attribute::AllowDirectAccessInHotPatchFunction:
+ return bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION;
+ case Attribute::MarkedForWindowsHotPatching:
+ return bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING;
case Attribute::EndAttrKinds:
llvm_unreachable("Can not encode end-attribute kinds marker.");
case Attribute::None:
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
index df4e48571692c..c643bd269f3b1 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
@@ -627,8 +627,6 @@ void CodeViewDebug::beginModule(Module *M) {
CurrentSourceLanguage = MapDWLangToCVLang(CU->getSourceLanguage());
- collectGlobalVariableInfo();
-
// Check if we should emit type record hashes.
ConstantInt *GH =
mdconst::extract_or_null<ConstantInt>(M->getModuleFlag("CodeViewGHash"));
@@ -639,6 +637,8 @@ void CodeViewDebug::endModule() {
if (!Asm || !Asm->hasDebugInfo())
return;
+ collectGlobalVariableInfo();
+
// The COFF .debug$S section consists of several subsections, each starting
// with a 4-byte control code (e.g. 0xF1, 0xF2, etc) and then a 4-byte length
// of the payload followed by the payload itself. The subsections are 4-byte
@@ -653,6 +653,8 @@ void CodeViewDebug::endModule() {
emitCompilerInformation();
endCVSubsection(CompilerInfo);
+ emitHotPatchInformation();
+
emitInlineeLinesSubsection();
// Emit per-function debug information.
@@ -807,6 +809,32 @@ void CodeViewDebug::emitObjName() {
endSymbolRecord(CompilerEnd);
}
+void CodeViewDebug::emitHotPatchInformation() {
+ MCSymbol *hotPatchInfo = nullptr;
+ for (const auto &F : MMI->getModule()->functions()) {
+ if (!F.isDeclarationForLinker() &&
+ F.hasFnAttribute(Attribute::MarkedForWindowsHotPatching)) {
+ if (hotPatchInfo == nullptr) {
+ hotPatchInfo = beginCVSubsection(DebugSubsectionKind::Symbols);
+ }
+ MCSymbol *HotPatchEnd = beginSymbolRecord(SymbolKind::S_HOTPATCHFUNC);
+ auto *SP = F.getSubprogram();
+ OS.AddComment("Function");
+ OS.emitInt32(getFuncIdForSubprogram(SP).getIndex());
+ OS.AddComment("Name");
+ llvm::StringRef Name = SP->getLinkageName();
+ if (Name.empty()) {
+ Name = F.getName();
+ }
+ emitNullTerminatedSymbolName(OS, Name);
+ endSymbolRecord(HotPatchEnd);
+ }
+ }
+ if (hotPatchInfo != nullptr) {
+ endCVSubsection(hotPatchInfo);
+ }
+}
+
namespace {
struct Version {
int Part[4];
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
index 7a138a0332b6d..3cc47b79b5b32 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
@@ -333,6 +333,8 @@ class LLVM_LIBRARY_VISIBILITY CodeViewDebug : public DebugHandlerBase {
void emitCompilerInformation();
+ void emitHotPatchInformation();
+
void emitBuildInfo();
void emitInlineeLinesSubsection();
diff --git a/llvm/lib/CodeGen/CMakeLists.txt b/llvm/lib/CodeGen/CMakeLists.txt
index 5dd6413431255..f18f085b0450b 100644
--- a/llvm/lib/CodeGen/CMakeLists.txt
+++ b/llvm/lib/CodeGen/CMakeLists.txt
@@ -250,6 +250,7 @@ add_llvm_component_library(LLVMCodeGen
VirtRegMap.cpp
WasmEHPrepare.cpp
WindowScheduler.cpp
+ WindowsHotPatch.cpp
WinEHPrepare.cpp
XRayInstrumentation.cpp
${GeneratedMLSources}
diff --git a/llvm/lib/CodeGen/TargetPassConfig.cpp b/llvm/lib/CodeGen/TargetPassConfig.cpp
index 0095ce3d96277..1d93efc21fbab 100644
--- a/llvm/lib/CodeGen/TargetPassConfig.cpp
+++ b/llvm/lib/CodeGen/TargetPassConfig.cpp
@@ -893,6 +893,9 @@ void TargetPassConfig::addIRPasses() {
if (EnableGlobalMergeFunc)
addPass(createGlobalMergeFuncPass());
+
+ if (TM->getTargetTriple().isOSBinFormatCOFF())
+ addPass(createWindowsHotPatch());
}
/// Turn exception handling constructs into something the code generators can
diff --git a/llvm/lib/CodeGen/WindowsHotPatch.cpp b/llvm/lib/CodeGen/WindowsHotPatch.cpp
new file mode 100644
index 0000000000000..853670d64d28f
--- /dev/null
+++ b/llvm/lib/CodeGen/WindowsHotPatch.cpp
@@ -0,0 +1,251 @@
+//===------ WindowsHotPatch.cpp - Support for Windows hotpatching ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Marks functions with the `marked_for_windows_hot_patching` attribute.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallSet.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/Attributes.h"
+#include "llvm/IR/DIBuilder.h"
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/Module.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/LineIterator.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "windows-hot-patch"
+
+// A file containing list of mangled function names to mark for hot patching.
+static cl:...
[truncated]
|
|
@llvm/pr-subscribers-objectyaml Author: None (sivadeilra) ChangesThis PR adds some of the support needed for Windows hot-patching. Windows implements a form of hot-patching. This allows patches to be applied to Windows apps, drivers, and the kernel, without rebooting or restarting any of these components. Hot-patching is a complex technology and requires coordination between the OS, compilers, linkers, and additional tools. This PR adds support to Clang and LLVM for part of the hot-patching process. It enables LLVM to generate the required code changes and to generate CodeView symbols which identify hot-patched functions. The PR provides new command-line arguments to Clang which allow developers to identify the list of functions that need to be hot-patched. This PR also allows LLVM to directly receive the list of functions to be modified, so that language front-ends which have not yet been modified (such as Rust) can still make use of hot-patching. This PR:
Although the flags are redundant between Clang and LLVM, this allows additional languages (such as Rust) to take advantage of hot-patching support before they have been modified to generate the required attributes. Credit to @dpaoliello, who wrote the original form of this patch. Patch is 39.25 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/138972.diff 34 Files Affected:
diff --git a/clang/include/clang/Basic/CodeGenOptions.h b/clang/include/clang/Basic/CodeGenOptions.h
index e39a73bdb13ac..3e102d47312ec 100644
--- a/clang/include/clang/Basic/CodeGenOptions.h
+++ b/clang/include/clang/Basic/CodeGenOptions.h
@@ -493,6 +493,14 @@ class CodeGenOptions : public CodeGenOptionsBase {
/// The name of a file to use with \c .secure_log_unique directives.
std::string AsSecureLogFile;
+ /// The name of a file that contains functions which will be compiled for
+ /// hotpatching. See -fms-hot-patch-functions-file.
+ std::string MSHotPatchFunctionsFile;
+
+ /// A list of functions which will be compiled for hotpatching.
+ /// See -fms-hot-patch-functions-list.
+ std::vector<std::string> MSHotPatchFunctionsList;
+
public:
// Define accessors/mutators for code generation options of enumeration type.
#define CODEGENOPT(Name, Bits, Default)
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 30ea75bb108d5..062c8544746c0 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3793,6 +3793,20 @@ def fms_hotpatch : Flag<["-"], "fms-hotpatch">, Group<f_Group>,
Visibility<[ClangOption, CC1Option, CLOption]>,
HelpText<"Ensure that all functions can be hotpatched at runtime">,
MarshallingInfoFlag<CodeGenOpts<"HotPatch">>;
+def fms_hotpatch_functions_file
+ : Joined<["-"], "fms-hotpatch-functions-file=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoString<CodeGenOpts<"MSHotPatchFunctionsFile">>,
+ HelpText<"Path to a file that contains a list of mangled symbol names of "
+ "functions that should be hot-patched">;
+def fms_hotpatch_functions_list
+ : CommaJoined<["-"], "fms-hotpatch-functions-list=">,
+ Group<f_Group>,
+ Visibility<[ClangOption, CC1Option, CLOption]>,
+ MarshallingInfoStringVector<CodeGenOpts<"MSHotPatchFunctionsList">>,
+ HelpText<"List of mangled symbol names of functions that should be "
+ "hot-patched">;
def fpcc_struct_return : Flag<["-"], "fpcc-struct-return">, Group<f_Group>,
Visibility<[ClangOption, CC1Option]>,
HelpText<"Override the default ABI to return all structs on the stack">;
diff --git a/clang/lib/CodeGen/CGCall.cpp b/clang/lib/CodeGen/CGCall.cpp
index 2f1c7699d27c3..14dcdbd79e577 100644
--- a/clang/lib/CodeGen/CGCall.cpp
+++ b/clang/lib/CodeGen/CGCall.cpp
@@ -2636,6 +2636,15 @@ void CodeGenModule::ConstructAttributeList(StringRef Name,
// CPU/feature overrides. addDefaultFunctionDefinitionAttributes
// handles these separately to set them based on the global defaults.
GetCPUAndFeaturesAttributes(CalleeInfo.getCalleeDecl(), FuncAttrs);
+
+ // Windows hotpatching support
+ if (!MSHotPatchFunctions.empty()) {
+ bool IsHotPatched = std::binary_search(MSHotPatchFunctions.begin(),
+ MSHotPatchFunctions.end(), Name);
+ if (IsHotPatched) {
+ FuncAttrs.addAttribute(llvm::Attribute::MarkedForWindowsHotPatching);
+ }
+ }
}
// Collect attributes from arguments and return values.
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp b/clang/lib/CodeGen/CodeGenModule.cpp
index c27817604f6ca..36a1f838ba414 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -453,6 +453,37 @@ CodeGenModule::CodeGenModule(ASTContext &C,
if (Context.getTargetInfo().getTriple().getArch() == llvm::Triple::x86)
getModule().addModuleFlag(llvm::Module::Error, "NumRegisterParameters",
CodeGenOpts.NumRegisterParameters);
+
+ // If there are any functions that are marked for Windows hot-patching,
+ // then build the list of functions now.
+ if (!CGO.MSHotPatchFunctionsFile.empty() ||
+ !CGO.MSHotPatchFunctionsList.empty()) {
+ if (!CGO.MSHotPatchFunctionsFile.empty()) {
+ auto BufOrErr = llvm::MemoryBuffer::getFile(CGO.MSHotPatchFunctionsFile);
+ if (BufOrErr) {
+ const llvm::MemoryBuffer &FileBuffer = **BufOrErr;
+ for (llvm::line_iterator I(FileBuffer.getMemBufferRef(), true), E;
+ I != E; ++I) {
+ this->MSHotPatchFunctions.push_back(std::string{*I});
+ }
+ } else {
+ auto &DE = Context.getDiagnostics();
+ unsigned DiagID =
+ DE.getCustomDiagID(DiagnosticsEngine::Error,
+ "failed to open hotpatch functions file "
+ "(-fms-hotpatch-functions-file): %0 : %1");
+ DE.Report(DiagID) << CGO.MSHotPatchFunctionsFile
+ << BufOrErr.getError().message();
+ }
+ }
+
+ for (const auto &FuncName : CGO.MSHotPatchFunctionsList) {
+ this->MSHotPatchFunctions.push_back(FuncName);
+ }
+
+ std::sort(this->MSHotPatchFunctions.begin(),
+ this->MSHotPatchFunctions.end());
+ }
}
CodeGenModule::~CodeGenModule() {}
diff --git a/clang/lib/CodeGen/CodeGenModule.h b/clang/lib/CodeGen/CodeGenModule.h
index 1db5c3bc4e4ef..45411cb95b7d2 100644
--- a/clang/lib/CodeGen/CodeGenModule.h
+++ b/clang/lib/CodeGen/CodeGenModule.h
@@ -678,6 +678,11 @@ class CodeGenModule : public CodeGenTypeCache {
AtomicOptions AtomicOpts;
+ // A set of functions which should be hot-patched; see
+ // -fms-hotpatch-functions-file (and -list). This will nearly always be empty.
+ // The list is sorted for binary-searching.
+ std::vector<std::string> MSHotPatchFunctions;
+
public:
CodeGenModule(ASTContext &C, IntrusiveRefCntPtr<llvm::vfs::FileSystem> FS,
const HeaderSearchOptions &headersearchopts,
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index f87549baff5e1..a733c50e56b89 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -6945,6 +6945,16 @@ void Clang::ConstructJob(Compilation &C, const JobAction &JA,
Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch);
+ if (Arg *A = Args.getLastArg(options::OPT_fms_hotpatch_functions_file)) {
+ Args.AddLastArg(CmdArgs, options::OPT_fms_hotpatch_functions_file);
+ }
+
+ for (const auto &A :
+ Args.getAllArgValues(options::OPT_fms_hotpatch_functions_list)) {
+ CmdArgs.push_back(
+ Args.MakeArgString("-fms-hotpatch-functions-list=" + Twine(A)));
+ }
+
if (TC.SupportsProfiling()) {
Args.AddLastArg(CmdArgs, options::OPT_pg);
diff --git a/clang/test/CodeGen/ms-hotpatch-bad-file.c b/clang/test/CodeGen/ms-hotpatch-bad-file.c
new file mode 100644
index 0000000000000..55304488175df
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-bad-file.c
@@ -0,0 +1,16 @@
+// This verifies that we correctly handle a -fms-hotpatch-functions-file argument that points
+// to a missing file.
+//
+// RUN: not %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/this-file-is-intentionally-missing-do-not-create-it.txt /Fo%t.obj %s 2>&1 | FileCheck %s
+// CHECK: failed to open hotpatch functions file
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch-functions.txt b/clang/test/CodeGen/ms-hotpatch-functions.txt
new file mode 100644
index 0000000000000..2512ba3881c66
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-functions.txt
@@ -0,0 +1 @@
+this_gets_hotpatched
diff --git a/clang/test/CodeGen/ms-hotpatch-lto.c b/clang/test/CodeGen/ms-hotpatch-lto.c
new file mode 100644
index 0000000000000..290f7695b02f6
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch-lto.c
@@ -0,0 +1,18 @@
+// This verifies that hotpatch function attributes are correctly propagated through LLVM IR when compiling with LTO.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt -flto /Fo%t.bc %s
+// RUN: llvm-dis %t.bc -o - | FileCheck %s
+//
+// CHECK: ; Function Attrs: marked_for_windows_hot_patching mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_gets_hotpatched() local_unnamed_addr #0 !dbg !13 {
+//
+// CHECK: ; Function Attrs: mustprogress nofree noinline norecurse nosync nounwind sspstrong willreturn memory(none) uwtable
+// CHECK-NEXT: define dso_local noundef i32 @this_does_not_get_hotpatched() local_unnamed_addr #1 !dbg !19 {
+
+int __declspec(noinline) this_gets_hotpatched() {
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/clang/test/CodeGen/ms-hotpatch.c b/clang/test/CodeGen/ms-hotpatch.c
new file mode 100644
index 0000000000000..c11b8ffc2f87a
--- /dev/null
+++ b/clang/test/CodeGen/ms-hotpatch.c
@@ -0,0 +1,17 @@
+// This verifies that hotpatch function attributes are correctly propagated when compiling directly to OBJ.
+//
+// RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt /Fo%t.obj %s
+// RUN: llvm-readobj --codeview %t.obj | FileCheck %s
+// CHECK: Kind: S_HOTPATCHFUNC (0x1169)
+// CHECK-NEXT: Function: this_gets_hotpatched
+
+void this_might_have_side_effects();
+
+int __declspec(noinline) this_gets_hotpatched() {
+ this_might_have_side_effects();
+ return 42;
+}
+
+int __declspec(noinline) this_does_not_get_hotpatched() {
+ return this_gets_hotpatched() + 100;
+}
diff --git a/llvm/include/llvm/Bitcode/LLVMBitCodes.h b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
index 9317e6aa793de..d32191d51023a 100644
--- a/llvm/include/llvm/Bitcode/LLVMBitCodes.h
+++ b/llvm/include/llvm/Bitcode/LLVMBitCodes.h
@@ -793,6 +793,8 @@ enum AttributeKindCodes {
ATTR_KIND_NO_DIVERGENCE_SOURCE = 100,
ATTR_KIND_SANITIZE_TYPE = 101,
ATTR_KIND_CAPTURES = 102,
+ ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION = 103,
+ ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING = 104,
};
enum ComdatSelectionKindCodes {
diff --git a/llvm/include/llvm/CodeGen/Passes.h b/llvm/include/llvm/CodeGen/Passes.h
index d214ab9306c2f..c4163617da5ae 100644
--- a/llvm/include/llvm/CodeGen/Passes.h
+++ b/llvm/include/llvm/CodeGen/Passes.h
@@ -617,6 +617,9 @@ namespace llvm {
/// Lowers KCFI operand bundles for indirect calls.
FunctionPass *createKCFIPass();
+
+ /// Creates Windows Hot Patch pass. \see WindowsHotPatch.cpp
+ ModulePass *createWindowsHotPatch();
} // End llvm namespace
#endif
diff --git a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
index 9d85acc49fa02..b38bdb482df43 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
+++ b/llvm/include/llvm/DebugInfo/CodeView/CodeViewSymbols.def
@@ -256,6 +256,8 @@ SYMBOL_RECORD_ALIAS(S_GTHREAD32 , 0x1113, GlobalTLS, ThreadLocalDataSym)
SYMBOL_RECORD(S_UNAMESPACE , 0x1124, UsingNamespaceSym)
SYMBOL_RECORD(S_ANNOTATION , 0x1019, AnnotationSym)
+SYMBOL_RECORD(S_HOTPATCHFUNC , 0x1169, HotPatchFuncSym)
+
#undef CV_SYMBOL
#undef SYMBOL_RECORD
#undef SYMBOL_RECORD_ALIAS
diff --git a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
index 3cce40dcf735a..de80d37999890 100644
--- a/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
+++ b/llvm/include/llvm/DebugInfo/CodeView/SymbolRecord.h
@@ -176,6 +176,21 @@ class CallerSym : public SymbolRecord {
uint32_t RecordOffset = 0;
};
+class HotPatchFuncSym : public SymbolRecord {
+public:
+ explicit HotPatchFuncSym(SymbolRecordKind Kind) : SymbolRecord(Kind) {}
+ HotPatchFuncSym(uint32_t RecordOffset)
+ : SymbolRecord(SymbolRecordKind::HotPatchFuncSym),
+ RecordOffset(RecordOffset) {}
+
+ // This is an ItemID in the IPI stream, which points to an LF_FUNC_ID or
+ // LF_MFUNC_ID record.
+ TypeIndex Function;
+ StringRef Name;
+
+ uint32_t RecordOffset = 0;
+};
+
struct DecodedAnnotation {
StringRef Name;
ArrayRef<uint8_t> Bytes;
diff --git a/llvm/include/llvm/IR/Attributes.td b/llvm/include/llvm/IR/Attributes.td
index fb94926043fc7..d2766a5346946 100644
--- a/llvm/include/llvm/IR/Attributes.td
+++ b/llvm/include/llvm/IR/Attributes.td
@@ -389,6 +389,17 @@ def CoroDestroyOnlyWhenComplete : EnumAttr<"coro_only_destroy_when_complete", In
/// pipeline to perform elide on the call or invoke instruction.
def CoroElideSafe : EnumAttr<"coro_elide_safe", IntersectPreserve, [FnAttr]>;
+/// Function is marked for Windows Hot Patching
+def MarkedForWindowsHotPatching
+ : EnumAttr<"marked_for_windows_hot_patching", IntersectPreserve, [FnAttr]>;
+
+/// Global variable should not be accessed through a "__ref_" global variable in
+/// a hot patching function This attribute is applied to the global variable
+/// decl, not the hotpatched function.
+def AllowDirectAccessInHotPatchFunction
+ : EnumAttr<"allow_direct_access_in_hot_patch_function",
+ IntersectPreserve, []>;
+
/// Target-independent string attributes.
def LessPreciseFPMAD : StrBoolAttr<"less-precise-fpmad">;
def NoInfsFPMath : StrBoolAttr<"no-infs-fp-math">;
diff --git a/llvm/include/llvm/InitializePasses.h b/llvm/include/llvm/InitializePasses.h
index bff0526d4177a..1a067e6526eb2 100644
--- a/llvm/include/llvm/InitializePasses.h
+++ b/llvm/include/llvm/InitializePasses.h
@@ -322,6 +322,7 @@ void initializeVirtRegMapWrapperLegacyPass(PassRegistry &);
void initializeVirtRegRewriterLegacyPass(PassRegistry &);
void initializeWasmEHPreparePass(PassRegistry &);
void initializeWinEHPreparePass(PassRegistry &);
+void initializeWindowsHotPatchPass(PassRegistry &);
void initializeWriteBitcodePassPass(PassRegistry &);
void initializeXRayInstrumentationLegacyPass(PassRegistry &);
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index 4074ed65885c7..46d1f883d5543 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -2248,6 +2248,10 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoExt;
case bitc::ATTR_KIND_CAPTURES:
return Attribute::Captures;
+ case bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION:
+ return Attribute::AllowDirectAccessInHotPatchFunction;
+ case bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING:
+ return Attribute::MarkedForWindowsHotPatching;
}
}
diff --git a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
index 27ada0ddcd831..7e418a522a39e 100644
--- a/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
+++ b/llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
@@ -919,6 +919,10 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_EXT;
case Attribute::Captures:
return bitc::ATTR_KIND_CAPTURES;
+ case Attribute::AllowDirectAccessInHotPatchFunction:
+ return bitc::ATTR_KIND_ALLOW_DIRECT_ACCESS_IN_HOT_PATCH_FUNCTION;
+ case Attribute::MarkedForWindowsHotPatching:
+ return bitc::ATTR_KIND_MARKED_FOR_WINDOWS_HOT_PATCHING;
case Attribute::EndAttrKinds:
llvm_unreachable("Can not encode end-attribute kinds marker.");
case Attribute::None:
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
index df4e48571692c..c643bd269f3b1 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.cpp
@@ -627,8 +627,6 @@ void CodeViewDebug::beginModule(Module *M) {
CurrentSourceLanguage = MapDWLangToCVLang(CU->getSourceLanguage());
- collectGlobalVariableInfo();
-
// Check if we should emit type record hashes.
ConstantInt *GH =
mdconst::extract_or_null<ConstantInt>(M->getModuleFlag("CodeViewGHash"));
@@ -639,6 +637,8 @@ void CodeViewDebug::endModule() {
if (!Asm || !Asm->hasDebugInfo())
return;
+ collectGlobalVariableInfo();
+
// The COFF .debug$S section consists of several subsections, each starting
// with a 4-byte control code (e.g. 0xF1, 0xF2, etc) and then a 4-byte length
// of the payload followed by the payload itself. The subsections are 4-byte
@@ -653,6 +653,8 @@ void CodeViewDebug::endModule() {
emitCompilerInformation();
endCVSubsection(CompilerInfo);
+ emitHotPatchInformation();
+
emitInlineeLinesSubsection();
// Emit per-function debug information.
@@ -807,6 +809,32 @@ void CodeViewDebug::emitObjName() {
endSymbolRecord(CompilerEnd);
}
+void CodeViewDebug::emitHotPatchInformation() {
+ MCSymbol *hotPatchInfo = nullptr;
+ for (const auto &F : MMI->getModule()->functions()) {
+ if (!F.isDeclarationForLinker() &&
+ F.hasFnAttribute(Attribute::MarkedForWindowsHotPatching)) {
+ if (hotPatchInfo == nullptr) {
+ hotPatchInfo = beginCVSubsection(DebugSubsectionKind::Symbols);
+ }
+ MCSymbol *HotPatchEnd = beginSymbolRecord(SymbolKind::S_HOTPATCHFUNC);
+ auto *SP = F.getSubprogram();
+ OS.AddComment("Function");
+ OS.emitInt32(getFuncIdForSubprogram(SP).getIndex());
+ OS.AddComment("Name");
+ llvm::StringRef Name = SP->getLinkageName();
+ if (Name.empty()) {
+ Name = F.getName();
+ }
+ emitNullTerminatedSymbolName(OS, Name);
+ endSymbolRecord(HotPatchEnd);
+ }
+ }
+ if (hotPatchInfo != nullptr) {
+ endCVSubsection(hotPatchInfo);
+ }
+}
+
namespace {
struct Version {
int Part[4];
diff --git a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
index 7a138a0332b6d..3cc47b79b5b32 100644
--- a/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
+++ b/llvm/lib/CodeGen/AsmPrinter/CodeViewDebug.h
@@ -333,6 +333,8 @@ class LLVM_LIBRARY_VISIBILITY CodeViewDebug : public DebugHandlerBase {
void emitCompilerInformation();
+ void emitHotPatchInformation();
+
void emitBuildInfo();
void emitInlineeLinesSubsection();
diff --git a/llvm/lib/CodeGen/CMakeLists.txt b/llvm/lib/CodeGen/CMakeLists.txt
index 5dd6413431255..f18f085b0450b 100644
--- a/llvm/lib/CodeGen/CMakeLists.txt
+++ b/llvm/lib/CodeGen/CMakeLists.txt
@@ -250,6 +250,7 @@ add_llvm_component_library(LLVMCodeGen
VirtRegMap.cpp
WasmEHPrepare.cpp
WindowScheduler.cpp
+ WindowsHotPatch.cpp
WinEHPrepare.cpp
XRayInstrumentation.cpp
${GeneratedMLSources}
diff --git a/llvm/lib/CodeGen/TargetPassConfig.cpp b/llvm/lib/CodeGen/TargetPassConfig.cpp
index 0095ce3d96277..1d93efc21fbab 100644
--- a/llvm/lib/CodeGen/TargetPassConfig.cpp
+++ b/llvm/lib/CodeGen/TargetPassConfig.cpp
@@ -893,6 +893,9 @@ void TargetPassConfig::addIRPasses() {
if (EnableGlobalMergeFunc)
addPass(createGlobalMergeFuncPass());
+
+ if (TM->getTargetTriple().isOSBinFormatCOFF())
+ addPass(createWindowsHotPatch());
}
/// Turn exception handling constructs into something the code generators can
diff --git a/llvm/lib/CodeGen/WindowsHotPatch.cpp b/llvm/lib/CodeGen/WindowsHotPatch.cpp
new file mode 100644
index 0000000000000..853670d64d28f
--- /dev/null
+++ b/llvm/lib/CodeGen/WindowsHotPatch.cpp
@@ -0,0 +1,251 @@
+//===------ WindowsHotPatch.cpp - Support for Windows hotpatching ---------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Marks functions with the `marked_for_windows_hot_patching` attribute.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallSet.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/Attributes.h"
+#include "llvm/IR/DIBuilder.h"
+#include "llvm/IR/DiagnosticInfo.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/Module.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/LineIterator.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "windows-hot-patch"
+
+// A file containing list of mangled function names to mark for hot patching.
+static cl:...
[truncated]
|
|
For context, here are recent Microsoft announcements concerning hot-patching: |
llvm/lib/CodeGen/WindowsHotPatch.cpp
Outdated
| } | ||
|
|
||
| llvm::StringRef FuncName{}; | ||
| if (auto *SP = F.getSubprogram(); SP != nullptr) { |
There was a problem hiding this comment.
Please add an explanation for why this needs to dig through debug info.
There was a problem hiding this comment.
Ahh, I think this is incorrect. I just need the mangled name of the symbol, and at one point this was the only way I could get it. I think this can simply be F.getName(); I'll verify and update.
There was a problem hiding this comment.
Updated to use simply `F.getName().
| // marked for hotpatching using the list provided directly to LLVM. | ||
| for (auto &F : M.functions()) { | ||
| // Ignore declarations that are not definitions. | ||
| if (F.isDeclarationForLinker()) { |
There was a problem hiding this comment.
Do we want to exclude functions with internal/private linkage?
There was a problem hiding this comment.
No, I think we want to include them. Current hot-patch tools cannot patch symbols that are marked as private, but that is a limitation of the hot-patch tools. Correlating private symbols in before/after builds of the same module is feasible and desirable, so I don't see any reason to exclude them from hot-patching here.
There was a problem hiding this comment.
The primary problem here is that optimization passes can manipulate internal function symbols. What do you do if you want to hotpatch a symbol, but it's disappeared, or has a different signature?
There was a problem hiding this comment.
That's a problem we already have with MSVC. Our hot-patch tooling doesn't make guarantees, and developers are currently required to do a moderate amount of "homework" while preparing hot-patches. They already have to deal with a function getting inlined into callers (which is why the S_INLINEES record was added, months ago) and several other situations.
Right now, we're aiming for parity with the experience using MSVC. Once we reach that, we can work on doing better than MSVC. Getting to parity will be no small achievement in the first place.
Developers already have to avoid changing function signatures, for example, for any call that crosses the patched / not-patched boundary.
There was a problem hiding this comment.
I think I don't follow.
There are hot-patch tools that can deal with symbols with internal linkage just fine (Live++), and it doesn't require developers to do any "homework" at all.
Furthermore, you mentioned the experience using MSVC - are you referring to Edit & Continue or something else?
There was a problem hiding this comment.
"We" in this context is Microsoft, specifically the Windows organization. We have in-house tools that generate our hot-patches. Currently, these tools only work with MSVC. The aim of this PR (and others, in the future) is to allow these tools to work with LLVM, specifically with Clang and Rust.
Our tools currently do not handle static functions. However, I don't want to exclude the possibility of supporting static functions in the future, which is why I responded to @efriedma-quic 's comment by saying that I don't want to restrict this code to handle only public functions.
There was a problem hiding this comment.
Furthermore, you mentioned the experience using MSVC - are you referring to Edit & Continue or something else?
This is not for Edit-and-Continue, this is for hot-patching deployed Windows machines running retail bits: https://www.microsoft.com/en-us/windows-server/blog/2025/04/24/tired-of-all-the-restarts-get-hotpatching-for-windows-server/?msockid=19a6f8f09bd160ac0b18ed449afc614b
There are hot-patch tools that can deal with symbols with internal linkage just fine (Live++), and it doesn't require developers to do any "homework" at all.
True, but tools like that have their own restrictions. From their own docs:
Linker -> Optimization -> References must be set to No (/OPT:NOREF)
Linker -> Optimization -> Enable COMDAT Folding must be set to No (/OPT:NOICF)
For Windows hot-patching one of the goals was to enable its use with fully-optimized binaries, the downside of this (as Eli called out) that that the optimizer can do lots of things that mess with the resulting symbols and binaries, hence why there is some "homework" required (i.e., we'd rather have Microsoft devs spend time to craft hot-patching changes that meet these requirements rather than deploy suboptimal binaries to customers).
| for (auto &I : instructions(F)) { | ||
| for (auto &U : I.operands()) { | ||
| // Discover all uses of GlobalVariable, these will need to be replaced. | ||
| GlobalVariable *GV = dyn_cast<GlobalVariable>(&U); |
There was a problem hiding this comment.
You probably need some kind of handling ConstantExpr here.
llvm/lib/CodeGen/WindowsHotPatch.cpp
Outdated
| RefMapping.try_emplace(GVUse.GV).first->second; | ||
| if (ReplaceWithRefGV == nullptr) { | ||
| Constant *AddrOfOldGV = ConstantExpr::getGetElementPtr( | ||
| Builder.getPtrTy(), GVUse.GV, ArrayRef<Value *>{}); |
There was a problem hiding this comment.
The getGetElementPtr call here just returns GVUse.GV, I think? We do constant folding on constant expressions.
llvm/lib/CodeGen/WindowsHotPatch.cpp
Outdated
| SmallDenseMap<GlobalVariable *, GlobalVariable *> &RefMapping, | ||
| DIBuilder &DebugInfo) { | ||
| for (auto &GVUse : GVUses) { | ||
| IRBuilder<> Builder(GVUse.User); |
There was a problem hiding this comment.
You can't insert code before an arbitrary instruction. For example, a PHI node.
There was a problem hiding this comment.
I see. What would you suggest? What IR could I build that would repro the problem? Would a PHI node directly consume a GlobalVariable node?
There was a problem hiding this comment.
Something like the following gives a PHI with a global as an operand:
int g1, g2;
int* f(int b, void g()) {
int* ret;
if (b) {
g();
ret = &g1;
} else {
ret = &g2;
}
return ret;
}
For PHI nodes, you can just insert the load into the predecessor BB, or something like that. Well, unless there's a catchswitch involved.
Most other instructions won't be a problem here... some less common edge cases to consider: catchpad, llvm.threadlocal.address.
* Simply use F.getName() for function name. * Fix llvm/test/CodeGen/Generic/ms-hotpatch-direct-global-access.ll * Add negative testing for functions that are not supposed to be hotpatched
|
Since the MSVC flag I'd like to understand the history here. It seems Can you provide a more technical insight about how is this gonna be used? What are the equivalent MSVC flags? Why is it that users (programmers) need to select specific functions to hotpatch? As a side-note, we (at Sony) have been using Live++ for this purpose for quite a while now on large game projects, sometimes going for a day long, on hundered of iterations, without stopping the game. This scenario (patching global variables) is already well supported by Live++ out of the box without the changes proposed in this PR. However I understand that Microsoft has a different technology that requires these changes. Also, this paragraph is a bit confusing:
"hot-patching/hot-patched" usually refers to a process that happens at runtime, not at compile time. Can you explain more in (technical) detail the whole pipeline for this new hotpatch feature? What does exactly the kernel/the system do after a new Windows update is downloaded? Does it iterate on the debug infos for each DLL in the update package, and if it finds
Second side-note: |
clang/test/CodeGen/ms-hotpatch.c
Outdated
| @@ -0,0 +1,20 @@ | |||
| // This verifies that hotpatch function attributes are correctly propagated when compiling directly to OBJ. | |||
| // | |||
| // RUN: %clang_cl -c --target=x86_64-windows-msvc -O2 /Z7 -fms-hotpatch-functions-file=%S/ms-hotpatch-functions.txt /Fo%t.obj %s | |||
There was a problem hiding this comment.
You might also want to test and support (here and in LLVM) target variants that might be unsupported with these new flags. Unsupported targets should warn if these flags are used.
There was a problem hiding this comment.
Thanks. I've added checks for targeting Windows/COFF to the code.
There was a problem hiding this comment.
Thank you! Are you able to exclude (and warn for) 32-bit (i386) and ARM/ARM64? We have users that target these archs on Windows.
| @@ -176,6 +176,21 @@ class CallerSym : public SymbolRecord { | |||
| uint32_t RecordOffset = 0; | |||
| }; | |||
|
|
|||
| class HotPatchFuncSym : public SymbolRecord { | |||
There was a problem hiding this comment.
Out of curiosity (related to my other questions), historically Windows updates only deliver binaries (DLLs, EXEs), not PDBs. PDBs are usually fetched by users when debugging, through the Microsoft symbol server. Then how is the kernel gonna find this record if the PDB isn't there by default?
There was a problem hiding this comment.
Our hot-patch tools read PDBs and then generate metadata that is placed into the final DLL/SYS/EXE, into a COFF section that is reserved for this purpose. The PDBs are not distributed with the hot-patch, and are not needed by the OS that is installing the hot-patch.
It sounds like the compiled code gets altered so that it no longer refers to global variables directly, but through a ref* indirection, similar to the GOT in ELFs. If this is only applied to global variables though, hot-patching of anything using function static variables (your typical singleton implementation) or TLS will fail. |
This PR is intended to allow LLVM-generated code to work with Windows hot-patching, also known as "Windows secure hot-patching". The "secure" part is relevant because the hot-patches generated by Windows meet Microsoft's requirements for binary signing and attestation. We use hot-patching for the Windows kernel, kernel-mode components (file systems, networking), device drivers, user-mode system services, user-mode applications, etc. The workflow that Microsoft uses relies on more information and behaviors than are currently publicly documented by the MSVC The The "double-indirection" when access global variables allows hot-patched functions to access the instance of global variables in the original base image (which is actively executing), instead of accessing the second copy of the global variable that is in the hot-patch image. For const data, this is unimportant, but this is very important for mutable global data. This works by generating a We are well aware of the limitations of hot-patching that this implies. Hot-patching is intended to be an optimization that can be applied in many circumstances, not a solution to every problem. Our tools inspect generated machine code and report errors when these requirements cannot be met. For example, hot-patching code that directly accesses TLS variables is out-of-scope. Hot-patching function statics is also currently out-of-scope, but with some work could be made to work. Our hot-patching tools and workflow allow us to generate hot-patches for the vast majority of CVEs that we fix, and this is the main focus for Windows hot-patching support. Windows hot-patching is not intended to be a replacement for Edit-and-Continue or to serve as a tool for interactive development. |
Also tolerate blank lines and trim whitespace in patch function files.
|
Thanks for the answer. Are these new flags If I understand correctly, what you describe as "symbols that were hot-patched" means functions that are ready/candidates for hotpatching at runtime on the OS that installs the hotpatch, is that right? I get the part about the security and verification, but I still don't get the It'd be really nice if someone at Microsoft could write at some point a supporting documentation to understand how all this is to be used (a tech blog or a white paper describing it?) |
This is simply how Windows Secure Hotpatching works, and has for years. It relies on different codegen for the hotpatched functions, rather than using fixups to modify code. This way, code segments are not modified (so hashes and code integrity still work). Ordinary fixups cannot be used (since they refer to relocations within a single image, not across multiple images (i.e. base image vs. patch image)), so using a different form of fixup would have been required. Microsoft (MSVC + Windows) chose to implement this using the global indirection technique. This is not directly related to synchronization, although if synchronization objects are stored in global variables it would certainly apply to them. It might help to emphasize the scenario that this is meant to support: hot-patching CVEs (vulnerabilities) in system services, the OS kernel, and device drivers. It's not meant for general development.
No, the target process is not stopped. In fact, we apply hot-patches to the kernel itself, without halting any processes or CPUs. The order of operations that Windows hot-patching uses guarantees the consistency requirements that we need. One of those requirements is that all code sees the right data (always points into global variables in the original image, not the hot-patched image).
We're working in that direction. This PR is one of our first steps in publicizing any part of the toolchain and workflow, which is the nature of the
Yes. This is a manual part of our workflow. Generally a developer identifies a function (or functions) that must be hot-patched, usually because they contain a defect. Then, the developer must manually identify the set of functions which get "dragged in" to hot-patching, usually because of inlining. Then the developer provides the list of functions which must be hot-patched to the compiler. |
|
Oh so all this dance ( |
This PR adds some of the support needed for Windows hot-patching.
Windows implements a form of hot-patching. This allows patches to be applied to Windows apps, drivers, and the kernel, without rebooting or restarting any of these components. Hot-patching is a complex technology and requires coordination between the OS, compilers, linkers, and additional tools.
This PR adds support to Clang and LLVM for part of the hot-patching process. It enables LLVM to generate the required code changes and to generate CodeView symbols which identify hot-patched functions. The PR provides new command-line arguments to Clang which allow developers to identify the list of functions that need to be hot-patched. This PR also allows LLVM to directly receive the list of functions to be modified, so that language front-ends which have not yet been modified (such as Rust) can still make use of hot-patching.
This PR:
MarkedForWindowsHotPatchingLLVM function attribute. This attribute indicates that a function should be hot-patched. This generates a new CodeView symbol,S_HOTPATCHFUNC, which identifies any function that has been hot-patched. This attribute also causes accesses to global variables to be indirected through a_ref_*global variable. This allows hot-patched functions to access the correct version of a global variable; the hot-patched code needs to access the variable in the original image, not the patch image.AllowDirectAccessInHotPatchFunctionLLVM attribute. This attribute may be placed on global variable declarations. It indicates that the variable may be safely accessed without the_ref_*indirection.-fms-hotpatch-functions-fileand-fms-hotpatch-functions-list. The-fileflag may point to a text file, which contains a list of functions to be hot-patched (one function name per line). The-listflag simply directly identifies functions to be patched, using a comma-separated list. These two command-line parameters may also be combined; the final set of functions to be hot-patched is the union of the two sets.--ms-hotpatch-functions-fileand--ms-hotpatch-functions-list.S_HOTPATCHFUNCCodeView symbol.Although the flags are redundant between Clang and LLVM, this allows additional languages (such as Rust) to take advantage of hot-patching support before they have been modified to generate the required attributes.
Credit to @dpaoliello, who wrote the original form of this patch.