[RISCV] Support XSfmm C intrinsics #143070
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This patch supports the naive vset* insertion. If the state(tm, tn, tk, sew, widen) changes, it emits all of the vset* instructions that are needed, partial compatibility is not supported yet. Co-authored-by: Piyou Chen <[email protected]>
Co-authored-by: Piyou Chen <[email protected]>
In this version of intrinsics, users need to manage the life time of tiles on their own, compiler doesn't have tile type for variables not only for design simplicity but also preventing users to write bad performance code that could potentially having tile spills which are quite expensive in terms of cycles.
llvmbot
added
clang
|
@llvm/pr-subscribers-backend-x86 @llvm/pr-subscribers-clang Author: Brandon Wu (4vtomat) ChangesIn this version of intrinsics, users need to manage the life time of Patch is 242.12 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/143070.diff 93 Files Affected:
diff --git a/clang/include/clang/Basic/riscv_sifive_vector.td b/clang/include/clang/Basic/riscv_sifive_vector.td
index f7996f362378a..9740f98f1285a 100644
--- a/clang/include/clang/Basic/riscv_sifive_vector.td
+++ b/clang/include/clang/Basic/riscv_sifive_vector.td
@@ -14,6 +14,10 @@
include "riscv_vector_common.td"
+class IsFloat<string type> {
+ bit val = !or(!eq(type, "x"), !eq(type, "f"), !eq(type, "d"), !eq(type, "y"));
+}
+
//===----------------------------------------------------------------------===//
// Instruction definitions
//===----------------------------------------------------------------------===//
@@ -198,3 +202,169 @@ let ManualCodegen = [{
defm sf_vfnrclip_xu_f_qf : RVVVFNRCLIPBuiltinSet<"Uv", "UvFqf", "c">;
}
}
+
+multiclass RVVSFTileLoadStoreBuiltinSet<list<string> types,
+ list<string> RequiredFeatures = []> {
+ let OverloadedName = NAME,
+ Name = NAME,
+ IRName = NAME,
+ Log2LMUL = [0],
+ HasMasked = false,
+ ManualCodegen = [{IntrinsicTypes = {Ops.back()->getType()};}] in
+ foreach type = types in {
+ let RequiredFeatures = !listconcat(RequiredFeatures,
+ !cond(!eq(type, "x"): ["Zvfhmin"],
+ !eq(type, "y"): ["Zvfbfmin"],
+ true: []<string>)) in {
+ def : RVVBuiltin<"e", "0zPCe", type>;
+ if !not(IsFloat<type>.val) then
+ def : RVVBuiltin<"Ue", "0zPCUe", type>;
+ }
+ }
+}
+
+multiclass RVVSFTileMoveBuiltinSet<list<list<string>> suffixes_prototypes,
+ list<int> intrinsic_types,
+ string type,
+ list<string> RequiredFeatures = []> {
+ foreach sp = suffixes_prototypes in
+ let RequiredFeatures = !listconcat(RequiredFeatures,
+ !cond(!eq(type, "x"): ["Zvfhmin"],
+ !eq(type, "y"): ["Zvfbfmin"],
+ true: []<string>)),
+ SupportOverloading = false,
+ HasMasked = false,
+ Name = NAME,
+ IRName = NAME,
+ HasVL = true,
+ Log2LMUL = [3],
+ IntrinsicTypes = intrinsic_types in
+ def : RVVBuiltin<sp[0], sp[1], type>;
+}
+
+multiclass RVVSFTileMoveVTBuiltinSet<list<string> RequiredFeatures = []> {
+ foreach type = ["c", "s", "i", "l"] in
+ defm NAME :
+ RVVSFTileMoveBuiltinSet<[["v", "vz"], ["Uv", "Uvz"]], [-1], type,
+ RequiredFeatures>;
+ foreach type = ["x", "y", "f", "d"] in
+ defm NAME :
+ RVVSFTileMoveBuiltinSet<[["v", "vz"]], [-1], type, RequiredFeatures>;
+}
+
+multiclass RVVSFTileMoveTVBuiltinSet<list<string> RequiredFeatures = []> {
+ let SupportOverloading = true, OverloadedName = NAME in {
+ foreach type = ["c", "s", "i", "l"] in
+ defm NAME :
+ RVVSFTileMoveBuiltinSet<[["v", "0zv"], ["Uv", "0zUv"]], [1], type,
+ RequiredFeatures>;
+ foreach type = ["x", "y", "f", "d"] in
+ defm NAME :
+ RVVSFTileMoveBuiltinSet<[["v", "0zv"]], [1], type, RequiredFeatures>;
+ }
+}
+
+multiclass RVVOp0Op1Op2BuiltinSet<string intrinsic_name, string type_range,
+ list<list<string>> suffixes_prototypes>
+ : RVVBuiltinSet<intrinsic_name, type_range, suffixes_prototypes, [0, 1, 2]>;
+
+multiclass RVVSFMatMulBuiltinSet<string prototype, string suffix,
+ string type_range, list<int> widens> {
+ foreach widen = widens in
+ let OverloadedName = NAME,
+ TWiden = widen,
+ HasVL = false,
+ Log2LMUL = [3],
+ HasMasked = false in
+ defm NAME : RVVOp0Op1Op2BuiltinSet<NAME, type_range,
+ [[!strconcat("w", !cast<string>(widen)), suffix, prototype]]>;
+}
+
+multiclass RVVSFMatMulFloatBuiltinSet<string name, string prototype, string suffix,
+ list<string> type_range, int widen> {
+ // Currently the XSfmm spec doesn't support w8.
+ foreach type = type_range in
+ let OverloadedName = name # !strconcat("_w", !cast<string>(widen)),
+ TWiden = widen,
+ HasVL = false,
+ Log2LMUL = [3],
+ Name = name # "_" # !strconcat("w", !cast<string>(widen)),
+ HasMasked = false in
+ defm : RVVOp0Op1BuiltinSet<name, type, [["", suffix, prototype]]>;
+}
+
+multiclass RVVSFVTZeroBuiltinSet {
+ let SupportOverloading = false,
+ HasVL = false,
+ HasMasked = false,
+ Name = NAME,
+ IRName = NAME,
+ Log2LMUL = [0] in
+ defm : RVVOp0BuiltinSet<NAME, "i", [["", "", "0Kzzzzz"]]>;
+}
+
+multiclass RVVSFVTDiscardBuiltinSet {
+ let SupportOverloading = false,
+ HasVL = false,
+ HasMasked = false,
+ Name = NAME,
+ IRName = NAME,
+ Log2LMUL = [0] in
+ defm : RVVBuiltinSet<NAME, "i", [["", "", "0"]], []>;
+}
+
+let RequiredFeatures = ["Xsfmmbase"] in {
+ let SupportOverloading = false,
+ HasVL = false,
+ HasMasked = false,
+ Log2LMUL = [0],
+ ManualCodegen = [{IntrinsicTypes = {ResultType};}] in // Set XLEN type
+ {
+ // let HasBuiltinAlias = false in
+ def sf_vsettnt : RVVBuiltin<"", "zzKzKz", "i">;
+ def sf_vsettm : RVVBuiltin<"", "zzKzKz", "i">;
+ let IRName = "sf_vsettnt" in
+ def sf_vsettn : RVVBuiltin<"", "zzKzKz", "i">;
+ def sf_vsettk : RVVBuiltin<"", "zzKzKz", "i">;
+ }
+ defm sf_vtzero_t : RVVSFVTZeroBuiltinSet;
+ defm sf_vtdiscard : RVVSFVTDiscardBuiltinSet;
+}
+
+defm sf_vtmv_v_t : RVVSFTileMoveVTBuiltinSet<["Xsfmmbase"]>;
+defm sf_vtmv_t_v : RVVSFTileMoveTVBuiltinSet<["Xsfmmbase"]>;
+
+defm sf_vlte8 : RVVSFTileLoadStoreBuiltinSet<["c"], ["Xsfmmbase"]>;
+defm sf_vlte16 : RVVSFTileLoadStoreBuiltinSet<["s", "x", "y"], ["Xsfmmbase"]>;
+defm sf_vlte32 : RVVSFTileLoadStoreBuiltinSet<["i", "f"], ["Xsfmmbase"]>;
+defm sf_vlte64 : RVVSFTileLoadStoreBuiltinSet<["l", "d"], ["Xsfmmbase"]>;
+
+defm sf_vste8 : RVVSFTileLoadStoreBuiltinSet<["c"], ["Xsfmmbase"]>;
+defm sf_vste16 : RVVSFTileLoadStoreBuiltinSet<["s", "x", "y"], ["Xsfmmbase"]>;
+defm sf_vste32 : RVVSFTileLoadStoreBuiltinSet<["i", "f"], ["Xsfmmbase"]>;
+defm sf_vste64 : RVVSFTileLoadStoreBuiltinSet<["l", "d"], ["Xsfmmbase"]>;
+
+let RequiredFeatures = ["Xsfmm32a8i"] in {
+ defm sf_mm_u_u : RVVSFMatMulBuiltinSet<"0KzUvUvzzz", "UvUv", "c", [4]>;
+ defm sf_mm_s_u : RVVSFMatMulBuiltinSet<"0KzvUvzzz", "vUv", "c", [4]>;
+ defm sf_mm_u_s : RVVSFMatMulBuiltinSet<"0KzUvvzzz", "Uvv", "c", [4]>;
+ defm sf_mm_s_s : RVVSFMatMulBuiltinSet<"0Kzvvzzz", "vv", "c", [4]>;
+
+}
+
+let RequiredFeatures = ["Xsfmm32a16f"] in
+ defm : RVVSFMatMulFloatBuiltinSet<"sf_mm_f_f", "0Kzvvzzz", "v", ["x", "y"], 2>;
+
+let RequiredFeatures = ["Xsfmm32a32f"] in
+ defm : RVVSFMatMulFloatBuiltinSet<"sf_mm_f_f", "0Kzvvzzz", "v", ["f"], 1>;
+
+let RequiredFeatures = ["Xsfmm32a8f"] in
+ foreach e1 = [5, 4] in
+ foreach e2 = [5, 4] in
+ let OverloadedName = "sf_mm_e" # e1 # "m" # !sub(7, e1) # "_e" # e2 # "m" # !sub(7, e2) in
+ defm : RVVSFMatMulFloatBuiltinSet<
+ "sf_mm_e" # e1 # "m" # !sub(7, e1) # "_e" # e2 # "m" # !sub(7, e2),
+ "0KzUvUvzzz", "UvUv", ["c"], 4>;
+
+let RequiredFeatures = ["Xsfmm64a64f"] in
+ defm : RVVSFMatMulFloatBuiltinSet<"sf_mm_f_f", "0Kzvvzzz", "v", ["d"], 1>;
diff --git a/clang/include/clang/Basic/riscv_vector_common.td b/clang/include/clang/Basic/riscv_vector_common.td
index 5a81376208f70..cd593b2b01969 100644
--- a/clang/include/clang/Basic/riscv_vector_common.td
+++ b/clang/include/clang/Basic/riscv_vector_common.td
@@ -245,6 +245,9 @@ class RVVBuiltin<string suffix, string prototype, string type_range,
// Set to true if the builtin has a parameter that models floating-point
// rounding mode control
bit HasFRMRoundModeOp = false;
+
+ // TWiden for XSfmm.
+ int TWiden = 0;
}
// This is the code emitted in the header.
diff --git a/clang/include/clang/Support/RISCVVIntrinsicUtils.h b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
index ddb527597c71c..8e281d5352052 100644
--- a/clang/include/clang/Support/RISCVVIntrinsicUtils.h
+++ b/clang/include/clang/Support/RISCVVIntrinsicUtils.h
@@ -403,6 +403,7 @@ class RVVIntrinsic {
std::vector<int64_t> IntrinsicTypes;
unsigned NF = 1;
Policy PolicyAttrs;
+ unsigned TWiden = 0;
public:
RVVIntrinsic(llvm::StringRef Name, llvm::StringRef Suffix,
@@ -411,8 +412,8 @@ class RVVIntrinsic {
bool HasVL, PolicyScheme Scheme, bool SupportOverloading,
bool HasBuiltinAlias, llvm::StringRef ManualCodegen,
const RVVTypes &Types,
- const std::vector<int64_t> &IntrinsicTypes,
- unsigned NF, Policy PolicyAttrs, bool HasFRMRoundModeOp);
+ const std::vector<int64_t> &IntrinsicTypes, unsigned NF,
+ Policy PolicyAttrs, bool HasFRMRoundModeOp, unsigned TWiden);
~RVVIntrinsic() = default;
RVVTypePtr getOutputType() const { return OutputType; }
@@ -436,6 +437,7 @@ class RVVIntrinsic {
llvm::StringRef getManualCodegen() const { return ManualCodegen; }
PolicyScheme getPolicyScheme() const { return Scheme; }
unsigned getNF() const { return NF; }
+ unsigned getTWiden() const { return TWiden; }
const std::vector<int64_t> &getIntrinsicTypes() const {
return IntrinsicTypes;
}
@@ -508,6 +510,12 @@ enum RVVRequire {
RVV_REQ_Zvfbfwma,
RVV_REQ_Zvfbfmin,
RVV_REQ_Zvfh,
+ RVV_REQ_Xsfmmbase,
+ RVV_REQ_Xsfmm32a8f,
+ RVV_REQ_Xsfmm32a16f,
+ RVV_REQ_Xsfmm32a32f,
+ RVV_REQ_Xsfmm64a64f,
+ RVV_REQ_Xsfmm32a8i,
RVV_REQ_Experimental,
RVV_REQ_NUM,
};
diff --git a/clang/lib/CodeGen/TargetBuiltins/RISCV.cpp b/clang/lib/CodeGen/TargetBuiltins/RISCV.cpp
index 89e3f6f203df3..68ce56c2a78bf 100644
--- a/clang/lib/CodeGen/TargetBuiltins/RISCV.cpp
+++ b/clang/lib/CodeGen/TargetBuiltins/RISCV.cpp
@@ -188,6 +188,8 @@ Value *CodeGenFunction::EmitRISCVBuiltinExpr(unsigned BuiltinID,
bool IsMasked = false;
// This is used by segment load/store to determine it's llvm type.
unsigned SegInstSEW = 8;
+ // This is used by XSfmm.
+ unsigned TWiden = 0;
// Required for overloaded intrinsics.
llvm::SmallVector<llvm::Type *, 2> IntrinsicTypes;
diff --git a/clang/lib/Headers/sifive_vector.h b/clang/lib/Headers/sifive_vector.h
index 4e67ad6fca262..ae01627de2d7d 100644
--- a/clang/lib/Headers/sifive_vector.h
+++ b/clang/lib/Headers/sifive_vector.h
@@ -115,4 +115,60 @@
#endif
#endif
+#define __riscv_sf_vsettnt_e8w1(atn) __riscv_sf_vsettnt(atn, 0, 1);
+#define __riscv_sf_vsettnt_e8w2(atn) __riscv_sf_vsettnt(atn, 0, 2);
+#define __riscv_sf_vsettnt_e8w4(atn) __riscv_sf_vsettnt(atn, 0, 3);
+#define __riscv_sf_vsettnt_e16w1(atn) __riscv_sf_vsettnt(atn, 1, 1);
+#define __riscv_sf_vsettnt_e16w2(atn) __riscv_sf_vsettnt(atn, 1, 2);
+#define __riscv_sf_vsettnt_e16w4(atn) __riscv_sf_vsettnt(atn, 1, 3);
+#define __riscv_sf_vsettnt_e32w1(atn) __riscv_sf_vsettnt(atn, 2, 1);
+#define __riscv_sf_vsettnt_e32w2(atn) __riscv_sf_vsettnt(atn, 2, 2);
+#define __riscv_sf_vsettm_e8w1(atm) __riscv_sf_vsettm(atm, 0, 1);
+#define __riscv_sf_vsettm_e8w2(atm) __riscv_sf_vsettm(atm, 0, 2);
+#define __riscv_sf_vsettm_e8w4(atm) __riscv_sf_vsettm(atm, 0, 3);
+#define __riscv_sf_vsettm_e16w1(atm) __riscv_sf_vsettm(atm, 1, 1);
+#define __riscv_sf_vsettm_e16w2(atm) __riscv_sf_vsettm(atm, 1, 2);
+#define __riscv_sf_vsettm_e16w4(atm) __riscv_sf_vsettm(atm, 1, 3);
+#define __riscv_sf_vsettm_e32w1(atm) __riscv_sf_vsettm(atm, 2, 1);
+#define __riscv_sf_vsettm_e32w2(atm) __riscv_sf_vsettm(atm, 2, 2);
+#define __riscv_sf_vsettn_e8w1(atn) __riscv_sf_vsettn(atn, 0, 1);
+#define __riscv_sf_vsettn_e8w2(atn) __riscv_sf_vsettn(atn, 0, 2);
+#define __riscv_sf_vsettn_e8w4(atn) __riscv_sf_vsettn(atn, 0, 3);
+#define __riscv_sf_vsettn_e16w1(atn) __riscv_sf_vsettn(atn, 1, 1);
+#define __riscv_sf_vsettn_e16w2(atn) __riscv_sf_vsettn(atn, 1, 2);
+#define __riscv_sf_vsettn_e16w4(atn) __riscv_sf_vsettn(atn, 1, 3);
+#define __riscv_sf_vsettn_e32w1(atn) __riscv_sf_vsettn(atn, 2, 1);
+#define __riscv_sf_vsettn_e32w2(atn) __riscv_sf_vsettn(atn, 2, 2);
+#define __riscv_sf_vsettk_e8w1(atk) __riscv_sf_vsettk(atk, 0, 1);
+#define __riscv_sf_vsettk_e8w2(atk) __riscv_sf_vsettk(atk, 0, 2);
+#define __riscv_sf_vsettk_e8w4(atk) __riscv_sf_vsettk(atk, 0, 3);
+#define __riscv_sf_vsettk_e16w1(atk) __riscv_sf_vsettk(atk, 1, 1);
+#define __riscv_sf_vsettk_e16w2(atk) __riscv_sf_vsettk(atk, 1, 2);
+#define __riscv_sf_vsettk_e16w4(atk) __riscv_sf_vsettk(atk, 1, 3);
+#define __riscv_sf_vsettk_e32w1(atk) __riscv_sf_vsettk(atk, 2, 1);
+#define __riscv_sf_vsettk_e32w2(atk) __riscv_sf_vsettk(atk, 2, 2);
+#define __riscv_sf_vtzero_t_e8w1(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 3, 1);
+#define __riscv_sf_vtzero_t_e8w2(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 3, 2);
+#define __riscv_sf_vtzero_t_e8w4(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 3, 4);
+#define __riscv_sf_vtzero_t_e16w1(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 4, 1);
+#define __riscv_sf_vtzero_t_e16w2(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 4, 2);
+#define __riscv_sf_vtzero_t_e16w4(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 4, 4);
+#define __riscv_sf_vtzero_t_e32w1(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 5, 1);
+#define __riscv_sf_vtzero_t_e32w2(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 5, 2);
+#if __riscv_v_elen >= 64
+#define __riscv_sf_vsettnt_e64w1(atn) __riscv_sf_vsettnt(atn, 3, 1);
+#define __riscv_sf_vsettm_e64w1(atm) __riscv_sf_vsettm(atm, 3, 1);
+#define __riscv_sf_vsettn_e64w1(atn) __riscv_sf_vsettn(atn, 3, 1);
+#define __riscv_sf_vsettk_e64w1(atk) __riscv_sf_vsettk(atk, 3, 1);
+#define __riscv_sf_vtzero_t_e64w1(tile, atm, atn) \
+ __riscv_sf_vtzero_t(tile, atm, atn, 6, 1);
+#endif
#endif //_SIFIVE_VECTOR_H_
diff --git a/clang/lib/Sema/SemaRISCV.cpp b/clang/lib/Sema/SemaRISCV.cpp
index ac88f5e059b7b..311bc25ca6851 100644
--- a/clang/lib/Sema/SemaRISCV.cpp
+++ b/clang/lib/Sema/SemaRISCV.cpp
@@ -244,6 +244,12 @@ void RISCVIntrinsicManagerImpl::ConstructRVVIntrinsics(
{"zvfbfwma", RVV_REQ_Zvfbfwma},
{"zvfbfmin", RVV_REQ_Zvfbfmin},
{"zvfh", RVV_REQ_Zvfh},
+ {"xsfmmbase", RVV_REQ_Xsfmmbase},
+ {"xsfmm32a8f", RVV_REQ_Xsfmm32a8f},
+ {"xsfmm32a16f", RVV_REQ_Xsfmm32a16f},
+ {"xsfmm32a32f", RVV_REQ_Xsfmm32a32f},
+ {"xsfmm64a64f", RVV_REQ_Xsfmm64a64f},
+ {"xsfmm32a8i", RVV_REQ_Xsfmm32a8i},
{"experimental", RVV_REQ_Experimental}};
// Construction of RVVIntrinsicRecords need to sync with createRVVIntrinsics
@@ -679,6 +685,50 @@ bool SemaRISCV::CheckBuiltinFunctionCall(const TargetInfo &TI,
return CheckVSetVL(1, 2);
case RISCVVector::BI__builtin_rvv_vsetvlimax:
return CheckVSetVL(0, 1);
+ case RISCVVector::BI__builtin_rvv_sf_vsettnt:
+ case RISCVVector::BI__builtin_rvv_sf_vsettm:
+ case RISCVVector::BI__builtin_rvv_sf_vsettn:
+ case RISCVVector::BI__builtin_rvv_sf_vsettk:
+ return SemaRef.BuiltinConstantArgRange(TheCall, 1, 0, 3) ||
+ SemaRef.BuiltinConstantArgRange(TheCall, 2, 1, 3);
+ case RISCVVector::BI__builtin_rvv_sf_mm_f_f_w1:
+ case RISCVVector::BI__builtin_rvv_sf_mm_f_f_w2:
+ case RISCVVector::BI__builtin_rvv_sf_mm_e5m2_e4m3_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_e5m2_e5m2_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_e4m3_e4m3_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_e4m3_e5m2_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_u_u_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_u_s_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_s_u_w4:
+ case RISCVVector::BI__builtin_rvv_sf_mm_s_s_w4: {
+ QualType Arg1Type = TheCall->getArg(1)->getType();
+ ASTContext::BuiltinVectorTypeInfo Info =
+ SemaRef.Context.getBuiltinVectorTypeInfo(
+ Arg1Type->castAs<BuiltinType>());
+ unsigned EltSize = SemaRef.Context.getTypeSize(Info.ElementType);
+ llvm::APSInt Result;
+
+ // We can't check the value of a dependent argument.
+ Expr *Arg = TheCall->getArg(0);
+ if (Arg->isTypeDependent() || Arg->isValueDependent())
+ return false;
+
+ // Check constant-ness first.
+ if (SemaRef.BuiltinConstantArg(TheCall, 0, Result))
+ return true;
+
+ // For TEW = 32, mtd can only be 0, 4, 8, 12.
+ // For TEW = 64, mtd can only be 0, 2, 4, 6, 8, 10, 12, 14.
+ // Only `sf_mm_f_f_w1` and `sf_mm_f_f_w2` might have TEW = 64.
+ if ((BuiltinID == RISCVVector::BI__builtin_rvv_sf_mm_f_f_w1 &&
+ EltSize == 64) ||
+ (BuiltinID == RISCVVector::BI__builtin_rvv_sf_mm_f_f_w2 &&
+ EltSize == 32))
+ return SemaRef.BuiltinConstantArgRange(TheCall, 0, 0, 15) ||
+ SemaRef.BuiltinConstantArgMultiple(TheCall, 0, 2);
+ return SemaRef.BuiltinConstantArgRange(TheCall, 0, 0, 15) ||
+ SemaRef.BuiltinConstantArgMultiple(TheCall, 0, 4);
+ }
case RISCVVector::BI__builtin_rvv_vget_v: {
ASTContext::BuiltinVectorTypeInfo ResVecInfo =
Context.getBuiltinVectorTypeInfo(cast<BuiltinType>(
diff --git a/clang/lib/Support/RISCVVIntrinsicUtils.cpp b/clang/lib/Support/RISCVVIntrinsicUtils.cpp
index 37f95411af195..2adc6e2470d0a 100644
--- a/clang/lib/Support/RISCVVIntrinsicUtils.cpp
+++ b/clang/lib/Support/RISCVVIntrinsicUtils.cpp
@@ -978,11 +978,12 @@ RVVIntrinsic::RVVIntrinsic(
bool HasMaskedOffOperand, bool HasVL, PolicyScheme Scheme,
bool SupportOverloading, bool HasBuiltinAlias, StringRef ManualCodegen,
const RVVTypes &OutInTypes, const std::vector<int64_t> &NewIntrinsicTypes,
- unsigned NF, Policy NewPolicyAttrs, bool HasFRMRoundModeOp)
+ unsigned NF, Policy NewPolicyAttrs, bool HasFRMRoundModeOp, unsigned TWiden)
: IRName(IRName), IsMasked(IsMasked),
HasMaskedOffOperand(HasMaskedOffOperand), HasVL(HasVL), Scheme(Scheme),
SupportOverloading(SupportOverloading), HasBuiltinAlias(HasBuiltinAlias),
- ManualCodegen(ManualCodegen.str()), NF(NF), PolicyAttrs(NewPolicyAttrs) {
+ ManualCodegen(ManualCodegen.str()), NF(NF), PolicyAttrs(NewPolicyAttrs),
+ TWiden(TWiden) {
// Init BuiltinName, Name and OverloadedName
BuiltinName = NewName.str();
@@ -1233,6 +1234,12 @@ llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, enum RVVRequire Require) {
STRINGIFY(RVV_REQ_Zvfbfwma)
STRINGIFY(RVV_REQ_Zvfbfmin)
STRINGIFY(RVV_REQ_Zvfh)
+ STRINGIFY(RVV_REQ_Xsfmmbase)
+ STRINGIFY(RVV_REQ_Xsfmm32a8f)
+ STRINGIFY(RVV_REQ_Xsfmm32a16f)
+ STRINGIFY(RVV_REQ_Xsfmm32a32f)
+ STRINGIFY(RVV_REQ_Xsfmm64a64f)
+ STRINGIFY(RVV_REQ_Xsfmm32a8i)
STRINGIFY(RVV_REQ_Experimental)
default:
llvm_unreachable("Unsupported RVVRequire!");
diff --git a/clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e4m3_e4m3.c b/clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e4m3_e4m3.c
new file mode 100644
index 0000000000000..3da3ba04d942d
--- /dev/null
+++ b/clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e4m3_e4m3.c
@@ -0,0 +1,18 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py UTC_ARGS: --version 4
+// REQUIRES: riscv-registered-target
+// RUN: %clang_cc1 -triple riscv64 -target-feature +xsfmm32a8f \
+// RUN: -disable-O0-optnone -emit-llvm %s -o - | \
+// RUN: opt -S -passes=mem2reg | FileCheck --check-prefix=CHECK-RV64 %s
+
+#include <sifive_vector.h>
+
+// CHECK-RV64-LABEL: define dso_local void @test_sf_mm_e4m3_e4m3_w4_u8m8_u8m8(
+// CHECK-RV64-SAME: <vscale x 64 x i8> [[V1:%.*]], <vscale x 64 x i8> [[V2:%.*]], i64 noundef [[TM:%.*]], i64 noundef [[TN:%.*]], i64 noundef [[TK:%.*]]) #[[ATTR0:[0-9]+]] {
+// CHECK-RV64-NEXT: entry:
+// CHECK-RV64-NEXT: call void @llvm.riscv.sf.mm.e4m3.e4m3.i64.nxv64i8(i64 0, <vscale x 64 x i8> [[V1]], <vscale x 64 x i8> [[V2]], i64 [[TM]], i64 [[TN]], i64 [[TK]], i64 4)
+// CHECK-RV64-NEXT: ret void
+//
+void test_sf_mm_e4m3_e4m3_w4_u8m8_u8m8(vuint8m8_t v1...
[truncated]
|
You can test this locally with the following command:git-clang-format --diff HEAD~1 HEAD --extensions c,h,cpp -- clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e4m3_e4m3.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e4m3_e5m2.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e5m2_e4m3.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_e5m2_e5m2.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_f_f.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_s_s.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_s_u.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_u_s.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_mm_u_u.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vlte16.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vlte32.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vlte64.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vlte8.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vsettk.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vsettm.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vsettn.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vsettnt.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vste16.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vste32.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vste64.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vste8.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vtdiscard.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vtmv_t_v.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vtmv_v_t.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/non-overloaded/sf_vtzero_t.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_e4m3_e4m3.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_e4m3_e5m2.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_e5m2_e4m3.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_e5m2_e5m2.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_f_f.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_s_s.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_s_u.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_u_s.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_mm_u_u.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vlte16.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vlte32.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vlte64.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vlte8.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vste16.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vste32.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vste64.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vste8.c clang/test/CodeGen/RISCV/rvv-intrinsics-sifive/non-policy/overloaded/sf_vtmv_t_v.c clang/test/Sema/sifive-xsfmm.c clang/test/Sema/sifive_sf_vset_invalid.c clang/include/clang/Support/RISCVVIntrinsicUtils.h clang/lib/CodeGen/TargetBuiltins/RISCV.cpp clang/lib/Headers/sifive_vector.h clang/lib/Sema/SemaRISCV.cpp clang/lib/Support/RISCVVIntrinsicUtils.cpp clang/utils/TableGen/RISCVVEmitter.cpp llvm/lib/Target/RISCV/AsmParser/RISCVAsmParser.cpp llvm/lib/Target/RISCV/MCTargetDesc/RISCVBaseInfo.h llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp llvm/lib/Target/RISCV/RISCVISelDAGToDAG.cpp llvm/lib/Target/RISCV/RISCVISelDAGToDAG.h llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp llvm/lib/Target/RISCV/RISCVInstrInfo.cpp llvm/lib/Target/RISCV/RISCVRegisterInfo.cppView the diff from clang-format here.diff --git a/llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp b/llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp
index 88d1178eb..45c8e040a 100644
--- a/llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp
+++ b/llvm/lib/Target/RISCV/RISCVInsertVSETVLI.cpp
@@ -1128,8 +1128,9 @@ RISCVInsertVSETVLI::computeInfoForInstr(const MachineInstr &MI) const {
}
void RISCVInsertVSETVLI::insertVSETVLI(MachineBasicBlock &MBB,
- MachineBasicBlock::iterator InsertPt, DebugLoc DL,
- const VSETVLIInfo &Info, const VSETVLIInfo &PrevInfo) {
+ MachineBasicBlock::iterator InsertPt,
+ DebugLoc DL, const VSETVLIInfo &Info,
+ const VSETVLIInfo &PrevInfo) {
++NumInsertedVSETVL;
if (Info.getTWiden()) {
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this version of intrinsics, users need to manage the life time of
tiles on their own, compiler doesn't have tile type for variables not
only for design simplicity but also preventing users to write bad
performance code that could potentially having tile spills which are
quite expensive in terms of cycles.
stack on: #143068 and #143069