Add Esm #2244
Open
Add Esm #2244
+1,494
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ruff.....................................................................Passed
ruff-format..............................................................Passed
Error: Process completed with exit code 1.Please help me figure out how to solve this problem. |
|
Probably an issue with generating the API symbols. Looks like you need to sync with the latest changes on master, then you could try running |
You can rebase it to latest master code |
keras_hub/src/layers/modeling/reversible_embedding_test.py::ReversibleEmbeddingTest::test_quantize_dtype_argument_tie_weights - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/layers/modeling/reversible_embedding_test.py::ReversibleEmbeddingTest::test_quantize_dtype_argument_untie_weights - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/layers/modeling/reversible_embedding_test.py::ReversibleEmbeddingTest::test_quantize_int8_tie_weights - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/layers/modeling/reversible_embedding_test.py::ReversibleEmbeddingTest::test_quantize_int8_untie_weights - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/albert/albert_backbone_test.py::AlbertBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/bart/bart_backbone_test.py::BartBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/bert/bert_backbone_test.py::BertBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/bloom/bloom_backbone_test.py::BloomBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/clip/clip_backbone_test.py::CLIPBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/deberta_v3/deberta_v3_backbone_test.py::DebertaV3BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/distil_bert/distil_bert_backbone_test.py::DistilBertBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/electra/electra_backbone_test.py::ElectraBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/f_net/f_net_backbone_test.py::FNetBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/falcon/falcon_backbone_test.py::FalconBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/gemma/gemma_backbone_test.py::GemmaBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/gemma/gemma_backbone_test.py::Gemma2BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/gpt2/gpt2_backbone_test.py::GPT2BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/gpt_neo_x/gpt_neo_x_backbone_test.py::GPTNeoXBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/llama/llama_backbone_test.py::LlamaTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/mistral/mistral_backbone_test.py::MistralBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/opt/opt_backbone_test.py::OPTBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/pali_gemma/pali_gemma_backbone_test.py::PaliGemmaBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/pali_gemma/pali_gemma_backbone_test.py::PaliGemma2BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/phi3/phi3_backbone_test.py::Phi3Test::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/phi3/phi3_backbone_test.py::Phi3Test::test_backbone_basics_with_su_rotary - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/roberta/roberta_backbone_test.py::RobertaBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/siglip/siglip_backbone_test.py::SigLIPBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/siglip/siglip_backbone_test.py::SigLIP2BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/t5/t5_backbone_test.py::T5BackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/whisper/whisper_backbone_test.py::WhisperBackboneTest::test_backbone_basics - TypeError: _int8_build() takes 2 positional arguments but 3 were given
FAILED keras_hub/src/models/xlm_roberta/xlm_roberta_backbone_test.py@mattdangerw @sachinprasadhs |
|
It's not related to your code, looks like some issue with the JAX backend, we will look into it. |
Comment on lines
+27
to
+29
| Disclaimer: Pre-trained models are provided on an "as is" basis, without | ||
| warranties or conditions of any kind. | ||
|
|
There was a problem hiding this comment.
You can remove this, we can have this mentioned in the Kaggle or HuggingFace if needed.
| intermediate_dim: int. The output dimension of the first Dense layer in | ||
| a two-layer feedforward network for each transformer. | ||
| dropout: float. Dropout probability for the Transformer encoder. | ||
| layer_norm_eps:bool.Should we use ln after embedding? |
There was a problem hiding this comment.
Didn't get the point here, are you asking our input or it's the arg detail, if it is the arg details, it needs to be repharsed, avoid question marks and the argument name is emb_layer_norm_before
layer_norm_eps discription needs to be updated.
| hidden_dim: int. The size of the transformer encoding and pooler layers. | ||
| intermediate_dim: int. The output dimension of the first Dense layer in | ||
| a two-layer feedforward network for each transformer. | ||
| dropout: float. Dropout probability for the Transformer encoder. |
There was a problem hiding this comment.
add: Defaults to 0.1
Comment on lines
+47
to
+51
| dtype: string or `keras.mixed_precision.DTypePolicy`. The dtype to use | ||
| for model computations and weights. Note that some computations, | ||
| such as softmax and layer normalization, will always be done at | ||
| float32 precision regardless of dtype. | ||
|
|
There was a problem hiding this comment.
None or str or keras.mixed_precision.DTypePolicy.
Comment on lines
+94
to
+98
| support_positon_type = ["rotary", "absolute"] | ||
| if position_embedding_type.lower() not in support_positon_type: | ||
| raise ( | ||
| f"This model only support below position embedding type: {support_positon_type}" # noqa: E501 | ||
| ) |
There was a problem hiding this comment.
Suggested change
| support_positon_type = ["rotary", "absolute"] | |
| if position_embedding_type.lower() not in support_positon_type: | |
| raise ( | |
| f"This model only support below position embedding type: {support_positon_type}" # noqa: E501 | |
| ) | |
| if position_embedding_type not in ( | |
| "rotary", | |
| "absolute",): | |
| raise ValueError( | |
| '`position_embedding_type` must be either `"rotary"`, or ' | |
| f'`"absolute"`. Received position_embedding_type={position_embedding_type}.' | |
| ) |
| init_kwargs=self.init_kwargs, | ||
| train_data=self.train_data, | ||
| expected_output_shape=(2, 2), | ||
| ) |
There was a problem hiding this comment.
saved model test is missing
| init_kwargs=self.init_kwargs, | ||
| train_data=self.train_data, | ||
| expected_output_shape=(2, 5, 10), | ||
| ) |
There was a problem hiding this comment.
saved model test is missing
Comment on lines
+30
to
+34
| Disclaimer: Pre-trained models are provided on an "as is" basis, without | ||
| warranties or conditions of any kind. The underlying model is provided by a | ||
| third party and subject to a separate license, available | ||
| [here](https://github.com/facebookresearch/esm). | ||
|
|
There was a problem hiding this comment.
Remove this, this will be part of the license and in the model card.
Comment on lines
+25
to
+27
| Disclaimer: Pre-trained models are provided on an "as is" basis, without | ||
| warranties or conditions of any kind. | ||
|
|
Comment on lines
+23
to
+36
| ), # 默认值为None | ||
| "layer_norm_eps": transformers_config.get( | ||
| "layer_norm_eps", 1e-12 | ||
| ), # 默认值为1e-12 | ||
| "emb_layer_norm_before": transformers_config.get( | ||
| "emb_layer_norm_before", False | ||
| ), # 默认值为False | ||
| "activation": transformers_config.get( | ||
| "activation", "gelu" | ||
| ), # 默认值为"gelu" | ||
| "max_wavelength": transformers_config.get( | ||
| "max_wavelength", 10000 | ||
| ), # 默认值为10000 | ||
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
from #2177
Achieved a smaller error with hf.