Skip to content

Commit 1dcae10

Browse files
LittleLittleCloudJordi Ramosjordi1215
authored
Fairlearn (#6539)
* same as last commit message * using xiaoyun\'s .rows.gropy() implementation method for the ByGroup() function * adding the regression catalog * added the fairlearn extension * Fixed the dataframe column adding bug; added unit testing Added `inPlace:true` to the append to fix the bug that no columns are added; Added unit testing to test the basic funcitonality of Metrc.Regression.ByGroup() * Added a Metric Test; Added difference between groups in regression * Added more metrics to the regression Added MSE and MAE (MeanAbsoluteError), and included tests for MSE and RMS * RMS and MSE support RMS and MSE fully supported by Fairlearn.Metric.Regression now accross all three functions * Moment and UtilityParity created the moment class and utilityParity class. Initial commit * Demographic Parity initial unit tests passed Passed the initial unit tests for Demographic Parity. Every class is made public which needs to be changed in the future. Utility Parity still needs to be changed for other parities to work * update * Added the rest of the binary metrics at byGroup() * Fixed a typo in the comment * Started on GridSearch, finished implementing the first approach to calculating the signed weights * Turned the moment class to an abstract class There is still a lot of development needed to be done with the moment class * Added signed weights, fixed ordering in Gamma * Added assembly reference for the AutoML to gain access to different models * fix sensitive feature column name bug, change default value for grid search getting sensitive feature column names directly from a getter function, adjusted default value for the option to a random value * Created a trail runner for grid search built the first prototype for the gridSearchTrialRunner. We have enabled a seperate training set for training and testing set for validating the result, which is a different approach from the original implementation. * Added AutoMLExperimentExtension to enable AutoML methods on gridsearch * Added row item lookup by columnname Added a new feature so that users can look up row item by the name of the column instead of the raw row index * Revert "Added AutoMLExperimentExtension to enable AutoML methods on gridsearch" This reverts commit a9f07a9. * made serviceCollection internal to be accessed by AutoML extension in fairlearn In fairlearn AutoML, we have to add in a singleton to the serviceCollection called moment, which will be later extracted to calculate the fairness parity. * Updated IMonitor for fairlearn Added an if statement to output fairlearn metric if using fairlearn * added FairnessTrialResulst for fairnessMetric * Revert "made serviceCollection internal to be accessed by AutoML extension in fairlearn" This reverts commit 328f718. * Added an extension for AutoML experiment the experiment is able to add a moment to its serviceCollection which is later used to calculate fairlearn parity. * Added a Fairlearn AutoMLTuner Created a tuner so that we can go through the search space through the gridsearch algorithm * updated Project reference * added a gridSearch test * Update BranchInfo.props * Fixed documentation Added documentation, deleted unneeded code * Update abstract methods Updating two methods and making them abstract * Updated documentation and added the Bound method * Moved metric related files into the metric folder * Added user's gridLimit support Allows users to provide customized gridLimit input for Grid Search * Delete GridSearch.cs Since we are leveraging AutoML, having a gridSearch file will not be necessary * added include Fairlearn.metric * added binary classification metric tests * fix tests * update * update * update * fix build error * re-add fsharp test * disable fairlearn test in automl as it's too costy * fix build issue * fix test * Update GridSearchTest.cs * fix comment * fix build error * capitalize folder name * fix builderror * revert change in AutoFit --------- Co-authored-by: Jordi Ramos <[email protected]> Co-authored-by: Jordi Ramos <[email protected]>
1 parent 3d705bf commit 1dcae10

File tree

25 files changed

+1263
-27
lines changed

25 files changed

+1263
-27
lines changed

Microsoft.ML.sln

Lines changed: 49 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
Microsoft Visual Studio Solution File, Format Version 12.00
1+
2+
Microsoft Visual Studio Solution File, Format Version 12.00
23
# Visual Studio Version 17
34
VisualStudioVersion = 17.1.32120.378
45
MinimumVisualStudioVersion = 10.0.40219.1
@@ -65,8 +66,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.CpuMath.Perfor
6566
EndProject
6667
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.CpuMath.UnitTests", "test\Microsoft.ML.CpuMath.UnitTests\Microsoft.ML.CpuMath.UnitTests.csproj", "{E97D8B5A-3035-4D41-9B0D-77FF8FB8D132}"
6768
EndProject
68-
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Microsoft.ML.FSharp.Tests", "test\Microsoft.ML.FSharp.Tests\Microsoft.ML.FSharp.Tests.fsproj", "{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}"
69-
EndProject
7069
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.ImageAnalytics", "src\Microsoft.ML.ImageAnalytics\Microsoft.ML.ImageAnalytics.csproj", "{00E38F77-1E61-4CDF-8F97-1417D4E85053}"
7170
EndProject
7271
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Mkl.Components", "src\Microsoft.ML.Mkl.Components\Microsoft.ML.Mkl.Components.csproj", "{A7222F41-1CF0-47D9-B80C-B4D77B027A61}"
@@ -92,8 +91,8 @@ EndProject
9291
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Samples", "docs\samples\Microsoft.ML.Samples\Microsoft.ML.Samples.csproj", "{ECB71297-9DF1-48CE-B93A-CD969221F9B6}"
9392
EndProject
9493
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.SamplesUtils", "src\Microsoft.ML.SamplesUtils\Microsoft.ML.SamplesUtils.csproj", "{11A5210E-2EA7-42F1-80DB-827762E9C781}"
95-
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Samples.OneDal", "docs\samples\Microsoft.ML.Samples.OneDal\Microsoft.ML.Samples.OneDal.csproj", "{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}"
9694
EndProject
95+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Samples.OneDal", "docs\samples\Microsoft.ML.Samples.OneDal\Microsoft.ML.Samples.OneDal.csproj", "{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}"
9796
EndProject
9897
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Recommender", "src\Microsoft.ML.Recommender\Microsoft.ML.Recommender.csproj", "{C8E1772B-DFD9-4A4D-830D-6AAB1C668BB3}"
9998
EndProject
@@ -157,12 +156,18 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.SearchSpace.Te
157156
EndProject
158157
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.AutoML.SourceGenerator", "tools-local\Microsoft.ML.AutoML.SourceGenerator\Microsoft.ML.AutoML.SourceGenerator.csproj", "{C804B990-390E-41D7-8FF1-6774495D70E2}"
159158
EndProject
159+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Fairlearn", "src\Microsoft.ML.Fairlearn\Microsoft.ML.Fairlearn.csproj", "{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}"
160+
EndProject
161+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Fairlearn.Tests", "test\Microsoft.ML.Fairlearn.Tests\Microsoft.ML.Fairlearn.Tests.csproj", "{416E682A-3958-49B9-8693-14EA96967AD3}"
162+
EndProject
160163
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.TorchSharp", "src\Microsoft.ML.TorchSharp\Microsoft.ML.TorchSharp.csproj", "{FF0BD187-4451-4A3B-934B-2AE3454896E2}"
161164
EndProject
162165
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Tokenizers", "src\Microsoft.ML.Tokenizers\Microsoft.ML.Tokenizers.csproj", "{BBC3A950-BD68-45AC-9DBD-A8F4D8847745}"
163166
EndProject
164167
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Microsoft.ML.Tokenizers.Tests", "test\Microsoft.ML.Tokenizers.Tests\Microsoft.ML.Tokenizers.Tests.csproj", "{C3D82402-F207-4F19-8C57-5AF0FBAF9682}"
165168
EndProject
169+
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Microsoft.ML.FSharp.Tests", "test\Microsoft.ML.FSharp.Tests\Microsoft.ML.FSharp.Tests.fsproj", "{041CB5CD-5832-413E-A894-D9DBED210B16}"
170+
EndProject
166171
Global
167172
GlobalSection(SolutionConfigurationPlatforms) = preSolution
168173
Debug|Any CPU = Debug|Any CPU
@@ -387,14 +392,6 @@ Global
387392
{E97D8B5A-3035-4D41-9B0D-77FF8FB8D132}.Release|Any CPU.Build.0 = Release|Any CPU
388393
{E97D8B5A-3035-4D41-9B0D-77FF8FB8D132}.Release|x64.ActiveCfg = Release|Any CPU
389394
{E97D8B5A-3035-4D41-9B0D-77FF8FB8D132}.Release|x64.Build.0 = Release|Any CPU
390-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
391-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Debug|Any CPU.Build.0 = Debug|Any CPU
392-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Debug|x64.ActiveCfg = Debug|Any CPU
393-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Debug|x64.Build.0 = Debug|Any CPU
394-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Release|Any CPU.ActiveCfg = Release|Any CPU
395-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Release|Any CPU.Build.0 = Release|Any CPU
396-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Release|x64.ActiveCfg = Release|Any CPU
397-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3}.Release|x64.Build.0 = Release|Any CPU
398395
{00E38F77-1E61-4CDF-8F97-1417D4E85053}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
399396
{00E38F77-1E61-4CDF-8F97-1417D4E85053}.Debug|Any CPU.Build.0 = Debug|Any CPU
400397
{00E38F77-1E61-4CDF-8F97-1417D4E85053}.Debug|x64.ActiveCfg = Debug|Any CPU
@@ -407,6 +404,10 @@ Global
407404
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Debug|Any CPU.Build.0 = Debug|Any CPU
408405
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Debug|x64.ActiveCfg = Debug|Any CPU
409406
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Debug|x64.Build.0 = Debug|Any CPU
407+
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|Any CPU.ActiveCfg = Release|Any CPU
408+
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|Any CPU.Build.0 = Release|Any CPU
409+
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|x64.ActiveCfg = Release|Any CPU
410+
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|x64.Build.0 = Release|Any CPU
410411
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
411412
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Debug|Any CPU.Build.0 = Debug|Any CPU
412413
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Debug|x64.ActiveCfg = Debug|Any CPU
@@ -415,10 +416,6 @@ Global
415416
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Release|Any CPU.Build.0 = Release|Any CPU
416417
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Release|x64.ActiveCfg = Release|Any CPU
417418
{A7222F94-2AF1-10C9-A21C-C4D22B137A69}.Release|x64.Build.0 = Release|Any CPU
418-
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|Any CPU.ActiveCfg = Release|Any CPU
419-
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|Any CPU.Build.0 = Release|Any CPU
420-
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|x64.ActiveCfg = Release|Any CPU
421-
{A7222F41-1CF0-47D9-B80C-B4D77B027A61}.Release|x64.Build.0 = Release|Any CPU
422419
{570A0B8A-5463-44D2-8521-54C0CA4CACA9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
423420
{570A0B8A-5463-44D2-8521-54C0CA4CACA9}.Debug|Any CPU.Build.0 = Debug|Any CPU
424421
{570A0B8A-5463-44D2-8521-54C0CA4CACA9}.Debug|x64.ActiveCfg = Debug|Any CPU
@@ -491,6 +488,14 @@ Global
491488
{11A5210E-2EA7-42F1-80DB-827762E9C781}.Release|Any CPU.Build.0 = Release|Any CPU
492489
{11A5210E-2EA7-42F1-80DB-827762E9C781}.Release|x64.ActiveCfg = Release|Any CPU
493490
{11A5210E-2EA7-42F1-80DB-827762E9C781}.Release|x64.Build.0 = Release|Any CPU
491+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
492+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|Any CPU.Build.0 = Debug|Any CPU
493+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|x64.ActiveCfg = Debug|Any CPU
494+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|x64.Build.0 = Debug|Any CPU
495+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|Any CPU.ActiveCfg = Release|Any CPU
496+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|Any CPU.Build.0 = Release|Any CPU
497+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|x64.ActiveCfg = Release|Any CPU
498+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|x64.Build.0 = Release|Any CPU
494499
{C8E1772B-DFD9-4A4D-830D-6AAB1C668BB3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
495500
{C8E1772B-DFD9-4A4D-830D-6AAB1C668BB3}.Debug|Any CPU.Build.0 = Debug|Any CPU
496501
{C8E1772B-DFD9-4A4D-830D-6AAB1C668BB3}.Debug|x64.ActiveCfg = Debug|Any CPU
@@ -735,6 +740,22 @@ Global
735740
{C804B990-390E-41D7-8FF1-6774495D70E2}.Release|Any CPU.Build.0 = Release|Any CPU
736741
{C804B990-390E-41D7-8FF1-6774495D70E2}.Release|x64.ActiveCfg = Release|Any CPU
737742
{C804B990-390E-41D7-8FF1-6774495D70E2}.Release|x64.Build.0 = Release|Any CPU
743+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
744+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Debug|Any CPU.Build.0 = Debug|Any CPU
745+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Debug|x64.ActiveCfg = Debug|Any CPU
746+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Debug|x64.Build.0 = Debug|Any CPU
747+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Release|Any CPU.ActiveCfg = Release|Any CPU
748+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Release|Any CPU.Build.0 = Release|Any CPU
749+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Release|x64.ActiveCfg = Release|Any CPU
750+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99}.Release|x64.Build.0 = Release|Any CPU
751+
{416E682A-3958-49B9-8693-14EA96967AD3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
752+
{416E682A-3958-49B9-8693-14EA96967AD3}.Debug|Any CPU.Build.0 = Debug|Any CPU
753+
{416E682A-3958-49B9-8693-14EA96967AD3}.Debug|x64.ActiveCfg = Debug|Any CPU
754+
{416E682A-3958-49B9-8693-14EA96967AD3}.Debug|x64.Build.0 = Debug|Any CPU
755+
{416E682A-3958-49B9-8693-14EA96967AD3}.Release|Any CPU.ActiveCfg = Release|Any CPU
756+
{416E682A-3958-49B9-8693-14EA96967AD3}.Release|Any CPU.Build.0 = Release|Any CPU
757+
{416E682A-3958-49B9-8693-14EA96967AD3}.Release|x64.ActiveCfg = Release|Any CPU
758+
{416E682A-3958-49B9-8693-14EA96967AD3}.Release|x64.Build.0 = Release|Any CPU
738759
{FF0BD187-4451-4A3B-934B-2AE3454896E2}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
739760
{FF0BD187-4451-4A3B-934B-2AE3454896E2}.Debug|Any CPU.Build.0 = Debug|Any CPU
740761
{FF0BD187-4451-4A3B-934B-2AE3454896E2}.Debug|x64.ActiveCfg = Debug|Any CPU
@@ -759,14 +780,14 @@ Global
759780
{C3D82402-F207-4F19-8C57-5AF0FBAF9682}.Release|Any CPU.Build.0 = Release|Any CPU
760781
{C3D82402-F207-4F19-8C57-5AF0FBAF9682}.Release|x64.ActiveCfg = Release|Any CPU
761782
{C3D82402-F207-4F19-8C57-5AF0FBAF9682}.Release|x64.Build.0 = Release|Any CPU
762-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
763-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|Any CPU.Build.0 = Debug|Any CPU
764-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|x64.ActiveCfg = Debug|Any CPU
765-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Debug|x64.Build.0 = Debug|Any CPU
766-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|Any CPU.ActiveCfg = Release|Any CPU
767-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|Any CPU.Build.0 = Release|Any CPU
768-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|x64.ActiveCfg = Release|Any CPU
769-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31}.Release|x64.Build.0 = Release|Any CPU
783+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
784+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Debug|Any CPU.Build.0 = Debug|Any CPU
785+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Debug|x64.ActiveCfg = Debug|Any CPU
786+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Debug|x64.Build.0 = Debug|Any CPU
787+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Release|Any CPU.ActiveCfg = Release|Any CPU
788+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Release|Any CPU.Build.0 = Release|Any CPU
789+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Release|x64.ActiveCfg = Release|Any CPU
790+
{041CB5CD-5832-413E-A894-D9DBED210B16}.Release|x64.Build.0 = Release|Any CPU
770791
EndGlobalSection
771792
GlobalSection(SolutionProperties) = preSolution
772793
HideSolutionNode = FALSE
@@ -799,7 +820,6 @@ Global
799820
{3E4ABF07-7970-4BE6-B45B-A13D3C397545} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
800821
{7333EDEF-4144-405C-A5EC-6F42201857D8} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
801822
{E97D8B5A-3035-4D41-9B0D-77FF8FB8D132} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
802-
{802233D6-8CC0-46AD-9F23-FEE1E9AED9B3} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
803823
{00E38F77-1E61-4CDF-8F97-1417D4E85053} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
804824
{A7222F41-1CF0-47D9-B80C-B4D77B027A61} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
805825
{A7222F94-2AF1-10C9-A21C-C4D22B137A69} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
@@ -812,6 +832,7 @@ Global
812832
{4B101D58-E7E4-4877-A536-A9B41E2E82A3} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
813833
{ECB71297-9DF1-48CE-B93A-CD969221F9B6} = {DA452A53-2E94-4433-B08C-041EDEC729E6}
814834
{11A5210E-2EA7-42F1-80DB-827762E9C781} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
835+
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31} = {DA452A53-2E94-4433-B08C-041EDEC729E6}
815836
{C8E1772B-DFD9-4A4D-830D-6AAB1C668BB3} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
816837
{9222FC9D-599A-49A5-B685-08CC9A5C81D7} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
817838
{6C29AA9B-054B-4762-BEA5-D305B932AA80} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
@@ -843,10 +864,12 @@ Global
843864
{A8F4F08F-1F9D-4AAE-8C8D-502CDBBDE7D3} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
844865
{A3E9F25F-2718-4FF9-A35A-54C232A847AB} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
845866
{C804B990-390E-41D7-8FF1-6774495D70E2} = {7F13E156-3EBA-4021-84A5-CD56BA72F99E}
867+
{2FB7A5CB-B41A-4A89-9B81-AFF24576DE99} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
868+
{416E682A-3958-49B9-8693-14EA96967AD3} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
846869
{FF0BD187-4451-4A3B-934B-2AE3454896E2} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
847870
{BBC3A950-BD68-45AC-9DBD-A8F4D8847745} = {09EADF06-BE25-4228-AB53-95AE3E15B530}
848871
{C3D82402-F207-4F19-8C57-5AF0FBAF9682} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
849-
{38ED61F4-FA22-4DE9-B0C4-91F327F4EE31} = {DA452A53-2E94-4433-B08C-041EDEC729E6}
872+
{041CB5CD-5832-413E-A894-D9DBED210B16} = {AED9C836-31E3-4F3F-8ABC-929555D3F3C4}
850873
EndGlobalSection
851874
GlobalSection(ExtensibilityGlobals) = postSolution
852875
SolutionGuid = {41165AF1-35BB-4832-A189-73060F82B01D}

src/Microsoft.Data.Analysis/DataFrameRow.cs

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,23 @@ public object this[int index]
5252
}
5353
}
5454

55+
/// <summary>
56+
/// An indexer to return the value at <paramref name="columnName"/>.
57+
/// </summary>
58+
/// <param name="columnName">The name of the column that corresponds to the return value</param>
59+
/// <returns>The value at this <paramref name="columnName"/>.</returns>
60+
public object this[string columnName]
61+
{
62+
get
63+
{
64+
return _dataFrame[columnName][_rowIndex];
65+
}
66+
set
67+
{
68+
_dataFrame[columnName][_rowIndex] = value;
69+
}
70+
}
71+
5572
/// <summary>
5673
/// A simple string representation of the values in this row
5774
/// </summary>

src/Microsoft.ML.AutoML/Assembly.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
[assembly: InternalsVisibleTo("mlnet.Tests, PublicKey=00240000048000009400000006020000002400005253413100040000010001004b86c4cb78549b34bab61a3b1800e23bfeb5b3ec390074041536a7e3cbd97f5f04cf0f857155a8928eaa29ebfd11cfbbad3ba70efea7bda3226c6a8d370a4cd303f714486b6ebc225985a638471e6ef571cc92a4613c00b8fa65d61ccee0cbe5f36330c9a01f4183559f1bef24cc2917c6d913e3a541333a1d05d9bed22b38cb")]
1010
[assembly: InternalsVisibleTo("Benchmark, PublicKey=00240000048000009400000006020000002400005253413100040000010001004b86c4cb78549b34bab61a3b1800e23bfeb5b3ec390074041536a7e3cbd97f5f04cf0f857155a8928eaa29ebfd11cfbbad3ba70efea7bda3226c6a8d370a4cd303f714486b6ebc225985a638471e6ef571cc92a4613c00b8fa65d61ccee0cbe5f36330c9a01f4183559f1bef24cc2917c6d913e3a541333a1d05d9bed22b38cb")]
1111
[assembly: InternalsVisibleTo("Microsoft.ML.CodeGenerator, PublicKey=00240000048000009400000006020000002400005253413100040000010001004b86c4cb78549b34bab61a3b1800e23bfeb5b3ec390074041536a7e3cbd97f5f04cf0f857155a8928eaa29ebfd11cfbbad3ba70efea7bda3226c6a8d370a4cd303f714486b6ebc225985a638471e6ef571cc92a4613c00b8fa65d61ccee0cbe5f36330c9a01f4183559f1bef24cc2917c6d913e3a541333a1d05d9bed22b38cb")]
12+
[assembly: InternalsVisibleTo("Microsoft.ML.Fairlearn, PublicKey=00240000048000009400000006020000002400005253413100040000010001004b86c4cb78549b34bab61a3b1800e23bfeb5b3ec390074041536a7e3cbd97f5f04cf0f857155a8928eaa29ebfd11cfbbad3ba70efea7bda3226c6a8d370a4cd303f714486b6ebc225985a638471e6ef571cc92a4613c00b8fa65d61ccee0cbe5f36330c9a01f4183559f1bef24cc2917c6d913e3a541333a1d05d9bed22b38cb")]
1213
[assembly: InternalsVisibleTo("Microsoft.ML.CodeGenerator.Tests, PublicKey=00240000048000009400000006020000002400005253413100040000010001004b86c4cb78549b34bab61a3b1800e23bfeb5b3ec390074041536a7e3cbd97f5f04cf0f857155a8928eaa29ebfd11cfbbad3ba70efea7bda3226c6a8d370a4cd303f714486b6ebc225985a638471e6ef571cc92a4613c00b8fa65d61ccee0cbe5f36330c9a01f4183559f1bef24cc2917c6d913e3a541333a1d05d9bed22b38cb")]
1314
[assembly: InternalsVisibleTo("Microsoft.ML.ModelBuilder, PublicKey=002400000480000094000000060200000024000052534131000400000100010007d1fa57c4aed9f0a32e84aa0faefd0de9e8fd6aec8f87fb03766c834c99921eb23be79ad9d5dcc1dd9ad236132102900b723cf980957fc4e177108fc607774f29e8320e92ea05ece4e821c0a5efe8f1645c4c0c93c1ab99285d622caa652c1dfad63d745d6f2de5f17e5eaf0fc4963d261c8a12436518206dc093344d5ad293")]
1415
[assembly: InternalsVisibleTo("Microsoft.ML.ModelBuilder.AutoMLService, PublicKey=002400000480000094000000060200000024000052534131000400000100010007d1fa57c4aed9f0a32e84aa0faefd0de9e8fd6aec8f87fb03766c834c99921eb23be79ad9d5dcc1dd9ad236132102900b723cf980957fc4e177108fc607774f29e8320e92ea05ece4e821c0a5efe8f1645c4c0c93c1ab99285d622caa652c1dfad63d745d6f2de5f17e5eaf0fc4963d261c8a12436518206dc093344d5ad293")]

src/Microsoft.ML.AutoML/AutoMLExperiment/AutoMLExperiment.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,7 @@ public async Task<TrialResult> RunAsync(CancellationToken ct = default)
234234
var monitor = serviceProvider.GetService<IMonitor>();
235235
var trialResultManager = serviceProvider.GetService<ITrialResultManager>();
236236
var trialNum = trialResultManager?.GetAllTrialResults().Max(t => t.TrialSettings?.TrialId) + 1 ?? 0;
237+
serviceProvider.GetService<ITrialRunner>();
237238
var tuner = serviceProvider.GetService<ITuner>();
238239
Contracts.Assert(tuner != null, "tuner can't be null");
239240

src/Microsoft.ML.AutoML/AutoMLExperiment/IMonitor.cs

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,14 @@ public virtual void ReportBestTrial(TrialResult result)
4545

4646
public virtual void ReportCompletedTrial(TrialResult result)
4747
{
48-
_logger.Info($"Update Completed Trial - Id: {result.TrialSettings.TrialId} - Metric: {result.Metric} - Pipeline: {_pipeline.ToString(result.TrialSettings.Parameter)} - Duration: {result.DurationInMilliseconds} - Peak CPU: {result.PeakCpu?.ToString("p")} - Peak Memory in MB: {result.PeakMemoryInMegaByte?.ToString("F")}");
48+
if (result is FairnessTrialResult fResult)
49+
{ //TODO: now we are assuming the higher the raw metric the better and the lower the fairness metric the better. If we have a raw metric that needs to be minimized then this should change
50+
_logger.Info($"Update Completed Trial - Id: {result.TrialSettings.TrialId} - Raw Metric: {result.Metric + fResult.FairnessMetric} - Fairness Metric: {-fResult.FairnessMetric} - Total Metric: {result.Metric} - Pipeline: {this._pipeline} - Duration: {result.DurationInMilliseconds}");
51+
}
52+
else
53+
{
54+
_logger.Info($"Update Completed Trial - Id: {result.TrialSettings.TrialId} - Metric: {result.Metric} - Pipeline: {this._pipeline} - Duration: {result.DurationInMilliseconds}");
55+
}
4956
_completedTrials.Add(result);
5057
}
5158

src/Microsoft.ML.AutoML/AutoMLExperiment/TrialResult.cs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,9 @@ internal class TrialResult<TMetric> : TrialResult
6666

6767
public EstimatorChain<ITransformer> Pipeline { get; set; }
6868
}
69+
70+
public class FairnessTrialResult : TrialResult
71+
{
72+
public double FairnessMetric { get; set; }
73+
}
6974
}

src/Microsoft.ML.Core/Properties/AssemblyInfo.cs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@
4343
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.TimeSeries" + PublicKey.Value)]
4444
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.Transforms" + PublicKey.Value)]
4545
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.AutoML" + PublicKey.Value)]
46+
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.Fairlearn" + PublicKey.Value)]
4647
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.TorchSharp" + PublicKey.Value)]
4748
[assembly: InternalsVisibleTo(assemblyName: "Microsoft.ML.Internal.MetaLinearLearner" + InternalPublicKey.Value)]
4849
[assembly: InternalsVisibleTo(assemblyName: "TreeVisualizer" + InternalPublicKey.Value)]

0 commit comments

Comments
 (0)