Analysis and Detection of Sections of English Pop Song Lyrics Using Transfer Learning from The Longformer Model

Abstract

Understanding the structural composition of song lyrics is essential for various applications, including music recommendation, summarization, and computational creativity. In this study, we explore automated section classification of song lyrics—specifically identifying parts such as verse, chorus, bridge, and others—using a transformer-based model. We fine-tuned a Longformer model on a dataset of 10,000 English pop lyrics with annotated section labels. The model was trained as a token classification task without the use of global attention, relying solely on local context to capture structural cues. Despite working with a significantly reduced dataset and limited training resources, the model achieved strong performance on the dominant structural classes, reaching F1-scores of 0.78 for verse and 0.77 for chorus. Secondary and infrequent sections such as bridge and prechorus showed moderate performance, while more ambiguous categories like postchorus and other were less accurately predicted. Analysis of the confusion matrix revealed that most misclassifications occurred between semantically overlapping sections, particularly among chorus-adjacent types. The results demonstrate that transformer models can effectively learn lyric structure from text alone, even with constrained data and without musical input. Our findings suggest that such models can serve as a strong foundation for future lyric analysis systems and that performance can be further improved through dataset expansion, label refinement, and multimodal integration.

Keywords: Lyrics Segmentation, Section Classification, Transformer, Longformer, Natural Language Processing, Music Structure Analysis, Chorus Detection, Lyric Modeling.