AIbase
Product LibraryTool Navigation

Small-lenguaje-Model-Hybrid-Norm-Furier-Formers

Public

A compact language model implementing HybridNorm and Fourier-based attention. Combines CoLA (low-rank projections), FANformer, and hybrid normalization to create an efficient decoder-only transformer. Leverages periodicity modeling and gated residuals to enhance performance while maintaining a small parameter footprint.

Creat2025-03-13T09:06:24
Update2025-03-19T09:44:22
2
Stars
0
Stars Increase

Related projects