Skip to content

Fix marshalling of bool flags in MF #3210

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 5, 2019
Merged

Fix marshalling of bool flags in MF #3210

merged 3 commits into from
Apr 5, 2019

Conversation

wschin
Copy link
Member

@wschin wschin commented Apr 5, 2019

Fix #3003. The marshalled Boolean size from C# to C should be 1 byte, not 4. Also, the change from LIBMF is for supporting quiet option in new implicit-feedback MF.

@wschin wschin requested review from Ivanidzo4ka and eerhardt April 5, 2019 03:17
@wschin wschin self-assigned this Apr 5, 2019
@wschin wschin added the bug Something isn't working label Apr 5, 2019
@codecov
Copy link

codecov bot commented Apr 5, 2019

Codecov Report

Merging #3210 into master will increase coverage by 0.01%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #3210      +/-   ##
==========================================
+ Coverage   72.61%   72.62%   +0.01%     
==========================================
  Files         807      807              
  Lines      145077   145080       +3     
  Branches    16213    16213              
==========================================
+ Hits       105345   105365      +20     
+ Misses      35316    35297      -19     
- Partials     4416     4418       +2
Flag Coverage Δ
#Debug 72.62% <0%> (+0.01%) ⬆️
#production 68.17% <0%> (+0.01%) ⬆️
#test 88.92% <ø> (ø) ⬆️
Impacted Files Coverage Δ
...osoft.ML.Recommender/SafeTrainingAndModelBuffer.cs 78.87% <0%> (ø) ⬆️
...soft.ML.Data/DataLoadSave/Text/TextLoaderCursor.cs 84.7% <0%> (-0.41%) ⬇️
...ML.Transforms/Text/StopWordsRemovingTransformer.cs 86.1% <0%> (-0.16%) ⬇️
...Microsoft.ML.Transforms/FeatureSelectionCatalog.cs 60% <0%> (ø) ⬆️
...osoft.ML.Recommender/MatrixFactorizationTrainer.cs 70.39% <0%> (ø) ⬆️
.../Microsoft.ML.Tests/TrainerEstimators/SdcaTests.cs 97.31% <0%> (+0.05%) ⬆️
...oft.ML.StandardTrainers/Standard/SdcaMulticlass.cs 91.12% <0%> (+1.02%) ⬆️
...LogisticRegression/MulticlassLogisticRegression.cs 67.61% <0%> (+1.74%) ⬆️
...oft.ML.StandardTrainers/StandardTrainersCatalog.cs 92.34% <0%> (+3.27%) ⬆️

@@ -133,19 +133,19 @@ private struct MFParameter
/// Specify if the factor matrices should be non-negative.
/// </summary>
[FieldOffset(48)]
public int DoNmf;
public byte DoNmf;
Copy link
Member

@eerhardt eerhardt Apr 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this is better than what we had before, this is still not correct.

https://docs.microsoft.com/en-us/cpp/cpp/fundamental-types-cpp?view=vs-2019

bool Type bool is an integral type that can have one of the two values true or false. Its size is unspecified.

https://en.cppreference.com/w/cpp/language/types

bool - type, capable of holding one of the two values: true or false. The value of sizeof(bool) is implementation defined and might differ from 1.

Instead, to make interop/marshalling work correctly on all computers - the technique we use in .NET is to ensure the ABI (application binary interface) is stable no matter the compiler/processor/etc. This means only stable-sized types can be used in the ABI that C# code P/Invokes into.

So to fix this, the libmf C code should change to have a stable ABI. I see it uses float, int, and bool. For sure int and bool can change sizes depending on the compiler/processor. Instead, we should be using int32_t and int8_t on the C side. (Note: you can just have a wrapper export function in the C library that exposes the stable ABI and calls into your current code. So the main C/C++ code can remain unchanged.) #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing this out. Another iteration is pushed. Hopefully, it can address this comment.

@wschin wschin requested a review from eerhardt April 5, 2019 18:16
@@ -133,19 +133,19 @@ private struct MFParameter
/// Specify if the factor matrices should be non-negative.
/// </summary>
[FieldOffset(48)]
public int DoNmf;
public sbyte DoNmf;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why sbyte? Why not just byte?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's int8 in C side.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make it uint8_t on the C side then?

Signed-ness doesn't really matter here. So using byte is more appropriate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I will change other place to match this decision.

@@ -9,25 +9,48 @@

using namespace mf;

EXPORT_API(mf_parameter) make_param(const mf_parameter_bridge *param_bridge)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this needs EXPORT_API does it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes!

param.eta = param_bridge->eta;
param.alpha = param_bridge->alpha;
param.c = param_bridge->c;
param.do_nmf = static_cast<bool>(param_bridge->do_nmf);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of static cast, would it be better to use param_bridge->do_nmf != 0 ? true : false?

Copy link
Member Author

@wschin wschin Apr 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Copy link
Member

@eerhardt eerhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@wschin
Copy link
Member Author

wschin commented Apr 5, 2019

I will make change to mf_problem and mf_model in other PRs because they incur more complicated changes.

@wschin wschin merged commit b2ac8e0 into dotnet:master Apr 5, 2019
@wschin wschin deleted the quiet-mf branch April 5, 2019 22:05
wschin added a commit to wschin/machinelearning that referenced this pull request Apr 11, 2019
@ghost ghost locked as resolved and limited conversation to collaborators Mar 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants