Researchers from EPFL have introduced MODUS — a unified multimodal model with a decoder-only architecture capable of performing any-to-any transformation tasks. Built upon the BAGEL-7B base, the model utilizes a hybrid approach with two experts to process different data types within a single structure.

image
image

What Happened

The MODUS architecture has been developed to support 16 different modalities. The model uses a 1D expert to handle discrete tokens (text, boxes) via autoregressive prediction, and a 2D expert for continuous spatial data (RGB, depth, segmentation) using flow matching technology. This allows for the construction of transformation chains where the output of one stage serves as the input for the next.

Context

Unlike traditional multimodal systems that rely on complex cascades of multiple separate encoders and decoders, MODUS proposes a shift toward a single unified decoder. The strong language model BAGEL-7B was used as a foundation, ensuring high-quality understanding of textual instructions.

Why It Matters for the Industry

For the industry, this is a significant step toward architectural unification. Moving away from fragmented modules in favor of a single decoder-only block simplifies the development of complex AI systems and reduces the complexity of integrating new modalities. The application of flow matching within the transformer opens new possibilities for high-precision multimodal data synthesis.

Why It Matters for Users

For end users, this means increased AI flexibility. It is now possible to create complex workflows within a single logical structure — for example, asking the model to turn text into a depth map, and then using that map to generate a realistic image while maintaining high consistency across all details.

What Is Not Yet Known / Limitations

At this stage, the project is research-oriented. To become production-ready, more data is needed regarding latency, throughput, and inference costs for supporting all 16 modalities.

Sources

Author

Look at AI, Editorial Team