Author, Institution: Lukas Stankevičius, Kaunas University of Technology
Science area, field of science: Natural Sciences, Informatics, N009
Research supervisor: Assoc. Prof. Dr. Mantas Lukoševičius (Kaunas University of Technology, Natural Sciences, Informatics, N009)
Dissertation Defence Board of Informatics Science Field:
Prof. Dr. Hab. Rimantas Barauskas (Kaunas University of Technology, Natural Sciences, Informatics, N009) – chairperson
Prof. Dr. Andrius Kriščiūnas (Kaunas University of Technology, Natural Sciences, Informatics, N009)
Prof. Dr. Sanda Martincic-Ipsic (University of Rijeka, Croatia, Natural Sciences, Informatics, N009)
Prof. Dr. Agnė Paulauskaitė-Tarasevičienė (Kaunas University of Technology, Natural Sciences, Informatics, N009)
Prof. Dr. Simona Ramanauskaitė (Vilnius Gediminas Technical University, Natural Sciences, Informatics, N009)
Dissertation defence meeting will be at Senate Hall of Kaunas University of Technology (K. Donelaičio 73-302, Kaunas)
The doctoral dissertation is available at the library of Kaunas University of Technology (Gedimino 50, Kaunas) and on the internet: L. Stankevičius el. dissertation.pdf
© L. Stankevičius, 2026 “The text of the thesis may not be copied, distributed, published, made public, including by making it publicly available on computer networks (Internet), reproduced in any form or by any means, including, but not limited to, electronic, mechanical or other means. Pursuant to Article 25(1) of the Law on Copyright and Related Rights of the Republic of Lithuania, a person with a disability who has difficulties in reading a document of a thesis published on the Internet, and insofar as this is justified by a particular disability, shall request that the document be made available in an alternative form by e-mail to doktorantura@ktu.lt.”
Annotation: Digital textual information is growing at a rapid pace. Effective automatic methods for understanding and processing such information are required, typically relying on the transformation of text into numerical representations. The emergence of Transformer-based models has created new challenges and opportunities for obtaining such representations. This dissertation investigates how Transformer models can be utilized for extracting text representations. For the English language, unsupervised representation extraction methods from pretrained Transformer models are explored for three groups of representation evaluation tasks: semantic textual similarity, short text clustering, and classification. For the Lithuanian language, the fine-tuning of Transformer models for grammatical error correction and abstractive summarization tasks is investigated, improving the original text itself. The dissertation presents methods that enable the extraction of improved representations from pretrained Transformer models. In addition, the use of a random vector model as a baseline method for future research is proposed. The first Transformer-based models for Lithuanian grammatical error correction and abstractive summarization are presented. The dissertation demonstrates that pretrained Transformer model weights can be effectively utilized to obtain high-quality representations without additional training, and how these models can be applied to lower-resource languages.