In the field of biological sequence modeling, deep learning has made remarkable progress, but the high computational demands and reliance on large datasets have hindered many researchers. Recently, a team from MIT, Harvard University, and Carnegie Mellon University introduced Lyra, a novel biological sequence modeling method. This method significantly reduces the number of parameters to just 1/120,000th of traditional models and can be trained in just two hours using two GPUs, greatly improving model efficiency.
Lyra's design is inspired by epistasis (the interaction between mutations within a sequence). It uses a sub-quadratic architecture to effectively understand the relationship between biological sequences and their functions. This new model demonstrates excellent performance across over 100 biological tasks, including protein fitness prediction, RNA function analysis, and CRISPR design, even achieving state-of-the-art (SOTA) performance in some key applications.
Compared to traditional Convolutional Neural Networks (CNNs) and Transformer models, Lyra's inference speed is 64.18 times faster, while significantly reducing parameter requirements. This is due to its innovative hybrid model architecture. Lyra combines a state-space model (SSM) and projected gated convolution (PGC) to capture both local and global dependencies in biological sequences. The SSM efficiently models global relationships using Fast Fourier Transform (FFT), while the PGC focuses on extracting local features. This combination allows Lyra to achieve a good balance between computational efficiency and interpretability.
Lyra's efficiency can not only advance fundamental biological research but also play a significant role in practical applications such as therapeutic development, pathogen monitoring, and biomanufacturing. The research team hopes that Lyra will enable more researchers to perform complex biological sequence modeling with limited resources, thereby accelerating the exploration of biological science.