Add commentValue property to Trivia for cleaned comments

[zyadtaha]

This PR continues the work from #2578, addressing the review comments and refining the behavior of Trivia.commentValue.

[ahoppen]

Thanks for addressing the issue, @zyadtaha. I think the PR still needs some polishing regarding indentation handling. I left a few comments inline.

[zyadtaha]

Thanks for the feedback, @ahoppen! I've addressed the comments—let me know if anything else needs attention.

[ahoppen]

Thanks for the adjustments. I left a few more edge cases to consider as comments.

[zyadtaha]

Hey @ahoppen, just wanted to check if there's anything else needed from my side on this PR. Let me know if you have any concerns. Thanks!

[ahoppen]

Thanks for the updates. I think there are still a few more issues that need to be addressed.

[zyadtaha]

Hey @ahoppen, I've pushed my updates. For your last two comments, I've added responses—let me know your feedback when you get a chance. Thanks!

[ahoppen]

I addressed my own review comments in a 2197103, which I pushed to your branch. Could you check if these changes make sense to you?

[zyadtaha]

I addressed my own review comments in a 2197103, which I pushed to your branch. Could you check if these changes make sense to you?

Thank you! I pulled the changes and reviewed the new logic and tests, and everything looks great to me. I really appreciated the detailed doc comments; they helped me understand your approach much better. I also liked how you split the tests into separate functions since it makes things much clearer.

I just had one question: why did you move the commentValue property to a separate file, and similarly, the test cases to a separate file?"

Also, if you have any feedback on my code, communication, or anything I could improve, I’d be very grateful. I want to make sure I’m not wasting your time and that I'm learning as much as possible through this process.

[ahoppen]

Thank you! I pulled the changes and reviewed the new logic and tests, and everything looks great to me. I really appreciated the detailed doc comments; they helped me understand your approach much better. I also liked how you split the tests into separate functions since it makes things much clearer.

Thanks for checking that the changes look good to you.

I just had one question: why did you move the commentValue property to a separate file, and similarly, the test cases to a separate file?"

The implementation of commentValue ended up being non-trivial and since it isn’t all that related to the other Trivia functionality (which is a lot more low-level), I decided to put it into its own file. This way we can also communicate that the extensions defined in that file are only intended for the commentValue property.

Also, if you have any feedback on my code, communication, or anything I could improve, I’d be very grateful. I want to make sure I’m not wasting your time and that I'm learning as much as possible through this process.

You did great work to get things started. I just figured that it was easier to fix the edge cases instead of repeatedly pointing out edge cases and then discovering more after you addressed them. Thinking of all possible edge cases is a big part of developing compiler-related tooling and it’s hard to teach that general way of thinking in pull request comments.

[rintaro]

I'm not sure about concatenating the pieces. The current behavior sounds fragile and not predictable.

TriviaPiece.commentValue: String? (as #1890 requested) would make more sense and we should implement that at least. But it's also debatable what to do with the space after //.

Trivia.commentValue: String? should be IMO a high-level API based on TriviaPiece.commentValue, but the concatenating behavior mixing trivia kinds should be discussed more.

Maybe Trivia.docCommentValue: String? that would only handle .docLineComment/.docBlockComment makes more sense.

[rintaro]

Here we allow

/// Foo Bar
/// This is a doc-comment.
// @_spi(Testing)
public fooBar() {}

resulting "Foo Bar\nThis is a doc-comment\n@_spi(Testing)", but not

/**
 * Foo Bar
 * This is a doc-comment.
 */
// @_spi(Testing)
public fooBar() {}

Not sure about this case.

[zyadtaha]

TriviaPiece.commentValue: String? (as #1890 requested) would make more sense and we should implement that at least. But it's also debatable what to do with the space after //.

I appreciate the thoughtful feedback! I agree that TriviaPiece.commentValue: String? (per #1890) is a great starting point. On the space after //, I'm not sure we should preserve it. In my view, something like TriviaPiece.lineComment("// Hello") could simply result in commentValue = "Hello", not " Hello". Keeping the space feels unnecessary with the likely intent of accessing the actual comment content. What’s your take on this?

Trivia.commentValue: String? should be IMO a high-level API based on TriviaPiece.commentValue, but the concatenating behavior mixing trivia kinds should be discussed more.

Maybe Trivia.docCommentValue: String? that would only handle .docLineComment/.docBlockComment makes more sense.

Your suggestion of Trivia.docCommentValue: String? for .docLineComment and .docBlockComment makes sense to me. As Alex noted (#2966 (comment)), it could return nil for multiple block comments or mixed comment types. Similarly, Trivia.commentValue: String? could focus on .lineComment and .blockComment, returning nil in those same cases. What do you think?