Inference with the checkpoint

[mathmax12]

Though there are some related tickets, it would be great to see the steps to use the raw checkpoints for inference. https://github.com/pytorch/torchtitan/blob/main/docs/checkpoint.md
This helps to verify the training works well.

[tianyu-l]

torchtitan is place to do training. For inference maybe you can try torchchat?

[jaysonfrancis]

@tianyu-l Some basic generation may help discover/test issues in checkpoint conversion, tokenization, validation before continued pre training etc.. I am working on adapting a minimal version from torchtune to support basic train-time evals during checkpointing

[tianyu-l]

@jaysonfrancis Covering the very basic generation/inference sounds good, as long as the code can be kept simple. Please feel free to submit a PR, we can help review.