Add get_config method to Loss #586
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mberr
commented
Sep 6, 2021
trigger ci
trigger ci
trigger ci
cthoyt
reviewed
Sep 6, 2021
cthoyt
reviewed
Sep 6, 2021
cthoyt
reviewed
Sep 6, 2021
| def get_config(self) -> MutableMapping[str, Any]: # noqa: D102 | ||
| config = super().get_config() | ||
| config["margin"] = self.margin | ||
| config["margin_activation"] = margin_activation_resolver.normalize_inst(self.margin_activation) |
There was a problem hiding this comment.
what if the activation has kwargs? should this be recursive?
There was a problem hiding this comment.
Yes, this should also consider kwargs 🤔 Since activations usually are PyTorch objects, this is may be problematic - so maybe we need the second alternative described above, where the resolver gets the get_config method?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a
get_configmethod toLossand its subclasses. The method is inspired by https://keras.io/api/layers/base_layer/#getconfig-method and can be used to obtain all parameters necessary to re-instantiate an equivalent instance. It prefers "simple" forms, e.g., converts the reduction method of margin activation to string representation.This method can be used in the future for logging hyperparameters, or creating JSONified configurations.
There is also a test which checks whether the re-instantiated loss instance produces the same loss values for a random batch of scores.
Alternative
Instead of making this a class method, we could also extend the class resolver to also provide the "inverse way" of extracting the configuration from an instance.