Skip to content

[Clang] Optimize tok::isLiteral with range-based condition #153228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 13, 2025
Merged

[Clang] Optimize tok::isLiteral with range-based condition #153228

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
73b3419
Optimize tok::isLiteral with range-based condition
Thibault-Monnier Aug 12, 2025
6ce02ca
Assert that TokenKind is ordered right
Thibault-Monnier Aug 12, 2025
342c864
Extend preprocessor `if`
Thibault-Monnier Aug 13, 2025
afe00a3
Reformat
Thibault-Monnier Aug 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 14 additions & 4 deletions clang/include/clang/Basic/TokenKinds.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,10 +95,20 @@ inline bool isStringLiteral(TokenKind K) {
/// Return true if this is a "literal" kind, like a numeric
/// constant, string, etc.
inline bool isLiteral(TokenKind K) {
return K == tok::numeric_constant || K == tok::char_constant ||
K == tok::wide_char_constant || K == tok::utf8_char_constant ||
K == tok::utf16_char_constant || K == tok::utf32_char_constant ||
isStringLiteral(K) || K == tok::header_name || K == tok::binary_data;
const bool isInLiteralRange =
K >= tok::numeric_constant && K <= tok::utf32_string_literal;

#if !NDEBUG
const bool isLiteralExplicit =
K == tok::numeric_constant || K == tok::char_constant ||
K == tok::wide_char_constant || K == tok::utf8_char_constant ||
K == tok::utf16_char_constant || K == tok::utf32_char_constant ||
isStringLiteral(K) || K == tok::header_name || K == tok::binary_data;
assert(isInLiteralRange == isLiteralExplicit &&
"TokenKind literals should be contiguous");
#endif

return isInLiteralRange;
}

/// Return true if this is any of tok::annot_* kinds.
Expand Down