@mberr can you elaborate on how the token representation might be useful for the chemical fingerprints (cf #1509 (comment))?

We don't seem to have any examples and the documentation in those components is pretty specific towards usage in NodePiece

The chemical representations are 2048-bit Morgan Fingerprints, where (more or less) each bit corresponds to the existence of a substructure in the molecule.

@mberr can you elaborate on how the token representation might be useful for the chemical fingerprints (cf #1509 (comment))?

We don't seem to have any examples and the documentation in those components is pretty specific towards usage in NodePiece

The chemical representations are 2048-bit Morgan Fingerprints, where (more or less) each bit corresponds to the existence of a substructure in the molecule.

In TokenizationRepresentation you have each entity assigned to a fixed number of tokens. Each token is an ID that is used to obtain token representations. There is also a padding token index to allow for less than num_tokens of actual tokens.

In your case, if the number of possible ones in these fingerprints is reasonably small, you could treat the indices of the 1s as the token indices for a given molecule. This would give you an explicit representation for each of the substructures (in the token representation), and the representation of the molecule would be composed of the representations of its substructures. Overall, this requires more computation than a simple lookup, but you may get some additional benefit from the inductive bias of shared representations for shared chemical substructures.

P.S.: By default, you get an extra dimension by using the TokenizationRepresentation, i.e. if your substructure token representation is of the shape shape, the output of the tokenization representation will be of the shape (num_tokens, *shape); you will probably want to reduce this again by a transformation. For simple aggregations, PyTorch actually has a built-in torch.nn.EmbeddingBag that may be useful to add.

Edit: I added the embedding bag here: #1512

so at this point, I am still thinking I like the static representations rather than trying to learn an embedding for each token, at least for demo purposes. I think that I'd have to develop this further to have a proper evaluation where the choice between a feature-enriched static representation vs and embedding bag would make a difference