focal loss for categorical (one-hot) encodings

[jadevaibhav]

Describe the feature and the current behavior/state.
Currently, sigmoid focal crossentropy provided is implemented for single value labels for binary labels, as it internally uses binary crossentropy for its calculations.

Relevant information

• Are you willing to contribute it (yes/no): Yes
• Are you willing to maintain it going forward? (yes/no): Yes
• Is there a relevant academic paper? (if so, where):
• Is there already an implementation in another framework? (if so, where):
• Was it part of tf.contrib? (if so, where): No

Which API type would this fall under (layer, metric, optimizer, etc.)
losses
Who will benefit with this feature?
If you want to implement the the binary classifier in one-hot encoding way, it would be useful.
Any other info.
I am attaching a [colab notebook]:https://colab.research.google.com/drive/1ORlgu9zfntzjsv6oxUgvSBvN4cSxmBEJ?usp=sharing . I would love to discuss this.

[AakashKumarNain]

@jadevaibhav Looks good. Can you open a PR? Also, add support for sparse labels as well.

[AakashKumarNain]

@jadevaibhav actually it works for OHE encoded labels as well, though it doesn't support sparse_labels as of now. You only need to submit a PR where it works with sparse labels as well when dealing with multi-class case. Here is an example:

oss = fl(
  [[0.8, 0.2], [0.9, 0.1], [0.3, 0.7]],
  [[1.0, 0.0], [1.0, 0.0], [0.0, 1.0]])
print('Loss: ', loss.numpy()) 

Loss:  [0.12314279 0.01539744 0.41548318]

Sorry for the lack of clarification in the earlier comment. When the focal loss version revised, it was made sure that it works for OHE labels and multi-class case.

[jadevaibhav]

@AakashKumarNain the current version uses binary crossentropy internally. I think it was because the paper also defines it in binary imbalanced class setting. Although passing OHE labels to binary crossentropy works, but it does not make sense. Hence I proposed the above implementation.

I haven't extrapolated the idea for multi-class classification, but I think it would just require per-class weight alpha. I think same goes for sparse-label. I haven't added for sparse label because it is same as single value for binary classification.

If you could clarify these things, I can look into extending to multi-class classification.

[AakashKumarNain]

I will provide you the details by tomorrow, so that you have a better understanding of what all needs to be done before you submit a PR.

[AakashKumarNain]

@jadevaibhav TF model zoo also have a focal loss implementation here. It is exactly the same as we have here and the plan is to merge the two in the near future. The only think that TF-addons implementations doesn't support is sparse_labels when doing multi-class. Don't over look the term binary_crossentropy, check the calculations.

If you wanna add a PR, please add support for sparse labels. If you have any doubt regarding that, you can tag me in this thread. Thank you

[bhack]

@AakashKumarNain check #914 (comment)

[AakashKumarNain]

Thanks @bhack for pointing it out.

[seanpmorgan]

TensorFlow Addons is transitioning to a minimal maintenance and release mode. New features will not be added to this repository. For more information, please see our public messaging on this decision:
TensorFlow Addons Wind Down

Please consider sending feature requests / contributions to other repositories in the TF community with a similar charters to TFA:
Keras
Keras-CV
Keras-NLP