ggml: allow casting between f32 and i32

[ngxson]

Motivation:

• When working with kyutai-mimi, at some points, I realized that I cannot do further calculation from the output of ggml_argmax or ggml_top_k due to missing an op to convert i32 --> f32
• Allowing Llama 4 attn_scale to be calculated on cgraph (just pointing this out, but probably we don't need to change this code)
• This discussion: model : add GroveMoE support #15510 (comment) (cc @CISC)
• Maybe useful for future models

Note: casting from f32 --> i32 will discard the fractional part

Planned to implement it on these backends:

• CPU
• Metal
• CUDA
• Vulkan

test-backend-ops:

  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  11841/11841 tests passed
  Backend Metal: OK

  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  11841/11841 tests passed
  Backend CUDA0: OK

  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=f32,type_dst=i32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[0,0,0,0],permute_dst=[0,0,0,0]): OK
  CPY(type_src=i32,type_dst=f32,ne=[256,2,3,4],permute_src=[1,0,2,3],permute_dst=[0,0,0,0]): OK
  11841/11841 tests passed
  Backend Vulkan0: OK

[slaren]

Note: casting from f32 --> i32 will be equivalent to floor()

For C/C++, the behavior of float to int cast is to discard the fractional part, truncating the value towards zero. For negative values, this is not the same as floor().

[ngxson]

@slaren thanks, I've updated the test and comment to reflect this. according to the test, the behavior is currently the same on all backends

[CISC]

A test that actually verifies that the cast produces the intended values (ie, 1.5->1, 1->1.0. etc) would be nice I guess.

[ngxson]

Yes that would be nice, and also be useful for many other ops. It can also act as examples for how to use certain ops. However, we need to adapt the code of test-backend-ops to support this, which can be quite complicated.

[CISC]

Yes that would be nice, and also be useful for many other ops. It can also act as examples for how to use certain ops. However, we need to adapt the code of test-backend-ops to support this, which can be quite complicated.

Yeah, figured as much, another thing on the collective consciousness might-TODO-list. :)

[ggerganov]

Yes that would be nice, and also be useful for many other ops. It can also act as examples for how to use certain ops. However, we need to adapt the code of test-backend-ops to support this, which can be quite complicated.

I think we can implement this by setting the suitable values in initialize_tensors. For example, setting setting values of -0.5 will cover that all backends are truncating towards zero.

[ngxson]

I think we can implement this by setting the suitable values in initialize_tensors. For example, setting setting values of -0.5 will cover that all backends are truncating towards zero.

Yes that's kinda what I'm doing, I set the range to [-150.0, 150.0] (basically copy the same code from test_gelu). So, we should randomly have negative values, which confirms the consistent behavior on all backends.

[0cc4m]

The Vulkan change looks good,

[ggerganov]

Should we merge this into the F32 branch above?

[ngxson]

Yes and indeed I also want to migrate some of these codes into template function. WDYT?

[ggerganov]

Yes, refactoring this code is welcome.

[ngxson]

I'll merge this PR as-is and will open another PR to refactor this code