Skip to content

Change modality one-hot encoding to a learnt encoding #26

@jacobbieker

Description

@jacobbieker

Detailed Description

AS @JackKelly pointed out, the PerceiverIO paper uses a learnt embedding for the modality, while this uses a one-hot vector instead. Switching to a learnt embedding might be better?

Context

Possible Implementation

A torch.nn.Embedding with a vocab size set to the number of data sources

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions