Cell Systems 2025

two modalities, one latent space

i am responsible for DECIPHAER's modeling development: the autoencoders, the adversarial training that merges transcriptional and morphological data into one latent space, and the work that continues to build on it. two encoders learn to hide which modality a point came from, while a label classifier keeps drug or condition classes distinct. this shows the objective at work.

  1. different inputs need different encoders. RNA-seq features and morphological features each enter a fully connected variational autoencoder, with hidden width 1024 and a 128-dimensional latent representation. reparameterisation samples a latent point from each encoder’s learned distribution. the display uses only two schematic dimensions.
  2. the modality discriminator tries to tell the two inputs apart. the autoencoders receive swapped modality targets, encouraging overlap instead. updates alternate: first the autoencoders and label classifier, with discriminator weights held fixed; then the discriminator. the falling accuracy here illustrates the aim of approaching chance, not a measured result.
  3. merging is not the same as collapsing. the label classifier encourages drug or condition classes to remain separable; colour shows invented toy classes, while filled circles and outlined diamonds distinguish modalities. the deposited defaults weight reconstruction MSE by 0.1 for each modality, label cross-entropy by 1000, adversarial cross-entropy by 1, and KL divergence by 1e-8. the indices shown are illustrative changes relative to their own starting values, not weighted loss magnitudes.
  4. a shared latent representation enables cross-modal translation: encode in one modality, then decode through the other modality’s decoder. overlap is the objective; this diagram does not establish translation accuracy or guarantee alignment.
  5. this is a schematic simulation of the training objective, not a recorded run. every moving point and metric comes from a deterministic, seeded toy dynamic stored in the accompanying JSON. epoch counts, curves, and toy classes are illustrative; architecture and default loss weights come from the deposited code.

[paper][code][back]

source: wjohnsonTufts/deciphaer @ 714fd8e · code/model.py + code/train_bi_modal.py · architecture and weights verified against source; dynamic simulated · generated by scripts/work/latent-merging.mjs · [data + checksums]