RSGPT: a generative transformer model for retrosynthesis planning pre-trained on ten billion datapoints by Yafeng Deng, Xinda Zhao, Hanyu Sun, Yu Chen, Xiaorui Wang, Xi Xue, Liangning Li, Jianfei Song, Chang-Yu Hsieh, Tingjun Hou, Xiandao Pan, Taghrid Saad Alomar, Xiangyang Ji, Xiaojian Wang

“…A generative pretrained transformer model is subsequently developed for template-free retrosynthesis planning by pre-training on 10 billion generated data. …”
Get full text

ReactionT5: a pre-trained transformer model for accurate chemical reaction prediction with limited data by Tatsuya Sagawa, Ryosuke Kojima

“…This study introduces ReactionT5, a transformer-based chemical reaction foundation model pre-trained on the Open Reaction Database—a large publicly available reaction dataset. …”
Get full text

Unsupervised evaluation of pre-trained DNA language model embeddings by Raghav Awasthi, Gayan Samuditha Mend Mend Arachchige, Xiaofeng Zhu

Get full text

CGM: Copy Mechanism GPT with Mask for Ellipsis and Anaphora Resolution in Dialogue by Ji-Won Cho, Jinyoung Oh, Jeong-Won Cha

“…GPT (Generative Pre-trained Transformer) is a generative language model that demonstrates outstanding performance in the field of text generation. …”
Get full text

Octascope: A Lightweight Pre-Trained Model for Optical Coherence Tomography by Haoyang Cui, Chen Wang, Paul Calle, Yunlong Liu, Qinghao Zhang, Sinaro Ly, Justin Reynolds, Feng Yan, Ke Zhang, Ronghao Liu, Junyuan Liu, Kar-Ming Fung, Zhongxin Yu, Ajay Jain, Qinggong Tang, Chongle Pan

“…This multi-domain pre-training enables Octascope to generalize across varied tissue types. …”
Get full text

Generative AI-powered multilingual ASR for seamless language-mixing transcriptions by Puspita Dash, Sruthi Babu, Logeswari Singaravel, Devadarshini Balasubramanian

Subjects: Get full text

Ensemble Approach for Image Recompression-Based Forgery Detection by Se-Jun Ham, Van-Ha Hoang, Chun-Su Park

Get full text

Impact of retrieval augmented generation and large language model complexity on undergraduate exams created and taken by AI agents by Erick Tyndall, Colleen Gayheart, Alexandre Some, Joseph Genz, Torrey Wagner, Brent Langhals

Subjects: Get full text

Vision Transformer untuk Klasifikasi Kematangan Pisang by Arya Pangestu, Bedy Purnama, Risnandar Risnandar

“…Penelitian dilakukan dengan menggunakan lima model ViT yang sudah dilatih sebelumnya atau pre-trained, yaitu ViT-B/16, ViT-B/32, ViT-L/16, ViT-L/32, and ViT-H/14 pada ImageNet-21k dan ImageNet-1k. …”
Get full text

Perbandingan Pretrained Model Transformer pada Deteksi Ulasan Palsu by Aisyah Awalina, Fitra Abdurrachman Bachtiar, Fitri Utaminingrum

“…Ada dua pendekatan yang dapat dilakukan dalam model Transformer yaitu pre-training dan fine-tuning. Penelitian sebelumnya telah banyak menggunakan fine-tuning dari model Transformer dikarenakan adanya kemudahan dalam pelatihan, waktu yang lebih sedikit, biaya dan kebutuhan lingkungan yang lebih rendah dibanding proses pre-training. …”
Get full text

CLFormer: a cross-lingual transformer framework for temporal forgery localization by Haonan Cheng, Hanyue Liu, Juanjuan Cai, Long Ye

“…Additionally, we introduced a cross-lingual transformer framework (CLFormer), which prioritizes audio features and utilizes a pre-trained multi-lingual Wav2Vec2 to enhance cross-lingual generalization, while incorporating visual features to further refine TFL. …”
Get full text

Lexicon-enhanced transformer with spatial-aware integration for Chinese named entity recognition by Jiachen Huang, Shuo Liu

“…Abstract Chinese Named Entity Recognition (CNER) is a fundamental and crucial task in information extraction. In recent years, pre-trained language and lexicon-based models have proven more powerful than the previous character-based models in CNER tasks. …”
Get full text

Visual language transformer framework for multimodal dance performance evaluation and progression monitoring by Lei Chen

“…We propose a novel transformer-based visual-language framework for multi-modal dance performance evaluation and progression monitoring. …”
Get full text

Herbify: an ensemble deep learning framework integrating convolutional neural networks and vision transformers for precise herb identification by Farhan Sheth, Ishika Chatter, Manvendra Jasra, Gireesh Kumar, Richa Sharma

“…Utilizing transfer learning, the research harnessed pre-trained Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs), then integrated these models into an ensemble framework that leverages the unique strengths of each architecture. …”
Get full text

Enhancing semantic segmentation of Ecuadorian shrimp ponds through fine-tuned vision transformers and U-Net architectures utilizing open-source remote sensing data by Daniel Jacome, Jianghao Wang, Yong Ge

Get full text

FSID: a novel approach to human activity recognition using few-shot weight imprinting by Mohammad Belal, Taimur Hassan, Abdelfatah Hassan, Divya Velayudhan, Noureldin Elhendawi, Ahmad Aljarah, Irfan Hussain

“…This paper proposes Few-Shot Imprinted DINO (FSID), a novel framework for HAR in low-data regimes, combining Few-Shot learning with weight imprinting and a self-supervised vision transformer, DINO (Distillation with No Labels). The FSID pipeline begins by converting raw time-series sensor data (e.g., EMG and IMU signals) into spectrogram images using the Short-Time Fourier Transform. …”
Get full text

MATSFT: User query-based multilingual abstractive text summarization for low resource Indian languages by fine-tuning mT5 by Siginamsetty Phani, Ashu Abdul, M. Krishna Siva Prasad, V. Dinesh Reddy

“…Experimental results show that MATSFT outperforms the monolingual transformer model, pre-trained MTM, mT5 model, NLI model, IndicBART, mBART25, and mBART50 on the IL dataset. …”
Get full text

Rumor detection using dual embeddings and text-based graph convolutional network by Barsha Pattanaik, Sourav Mandal, Rudra M. Tripathy, Arif Ahmed Sekh

“…This model uses dual embedding from two pre-trained transformer models: generative pre-trained transformers (GPT) and bidirectional encoder representations from transformers (BERT). …”
Get full text

Unveiling the spectrum of Arabic offensive language: Taxonomy and insights. by Chaya Liebeskind, Yossef Haim Shrem, Marina Litvak, Natalia Vanetik

“…We have also analyzed the performance of pre-trained and fine-tuned Arabic transformer offensive language detection models on these datasets. …”
Get full text

High-accuracy physical property prediction for pure organics via molecular representation learning: bridging data to discovery by Qi Ou, Hongshuai Wang, Minyang Zhuang, Shangqian Chen, Lele Liu, Ning Wang, Zhifeng Gao

“…Traditional trial-and-error methods for discovering highly functional organic compounds are expensive and time-consuming. We employed a 3D transformer-based molecular representation learning algorithm to create the Org-Mol pre-trained model, using 60 million semi-empirically optimized small organic molecule structures. …”
Get full text