Skip to content

Conversation

kuilpd
Copy link
Collaborator

@kuilpd kuilpd commented Feb 13, 2025

Change current token kind in the token list instead of a current copy, because if a TentativeParsingRollback happens, it discards the changed copy.

@kuilpd kuilpd requested a review from cmtice February 13, 2025 17:49
@cmtice
Copy link
Collaborator

cmtice commented Feb 18, 2025

This fix is probably not ok; I originally had a Token::SetKind method in the Token class, and the reviewer wanted me to remove it, in favor of just creating new tokens when needed instead.

I'm not sure I fully understand the problem that you are trying to solve here; could you elaborate a bit more please?

@cmtice
Copy link
Collaborator

cmtice commented Feb 18, 2025

More info (after discussing this in the meeting -- just noting down what we said): If splitting the token occurs inside a tentative parsing block, we need to do something like make a tentative copy of the lexer's m_lexed_tokens vector, and do the splitting/inserting there. If we decide to keep the tentative parsing, then we would replace the m_lexed_tokens vector in the lexer with our tentative copy. If we don't do something like this, then we could end up with m_lexed_tokens in the lexer containing an extra token (for each tentative parsing attempt that got rolled back), which could lead to some very confusing errors.
The original lldb-eval didn't need to do this because it didn't pre-lex all the tokens ahead of time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants