ENH: Add Functionality to extract embeddings

[StatMixedML]

Summary

This issue is a suggestion for adding functionality to Chronos that allows users to extract embeddings directly. This feature would greatly enhance the model's utility by enabling more detailed analysis and application in various forecasting tasks.

[abdulfatir]

@StatMixedML Thanks for opening. I am already working on this in #24. Please feel free to leave your thoughts there.

[abdulfatir]

Done in #24.

[StatMixedML]

@abdulfatir Thanks for the quick release!

I am running the example as shown in the README.md. The data has shape df.shape = (144, 2), whereas the embeddings have shape embeddings.shape = torch.Size([1, 145, 768]). Is there a reason why the second dimension, i.e., the time dimension of the embeddings has one more observation than the original data?

[abdulfatir]

@StatMixedML it's described in the docstring:

chronos-forecasting/src/chronos/chronos.py

Lines 359 to 369 in b4a6c0c

	Returns
	-------
	embeddings, tokenizer_state
	A tuple of two tensors: the encoder embeddings and the tokenizer_state,
	e.g., the scale of the time series in the case of mean scaling.
	The encoder embeddings are shaped (batch_size, context_length, d_model)
	or (batch_size, context_length + 1, d_model), where context_length
	is the size of the context along the time axis if a 2D tensor was provided
	or the length of the longest time series, if a list of 1D tensors was
	provided, and the extra 1 is for EOS.
	"""

TL;DR: it's for the EOS (end of sequence) token.

[StatMixedML]

Ok thanks, so this means we should disregard the last embedding embeddings[:,:-1,:] for downstream tasks?

[abdulfatir]

I would say that it's for the end user to test and decide. I am not sure how information rich the last embedding is.