Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

Retrieval-Pretrained Transformer: Long-range Language Modeling with Self-retrieval

Saved in:

Bibliographic Details
Main Authors:	Ohad Rubin, Jonathan Berant
Format:	Article
Language:	English
Published:	The MIT Press 2024-10-01
Series:	Transactions of the Association for Computational Linguistics
Online Access:	http://dx.doi.org/10.1162/tacl_a_00693
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

In-Context Retrieval-Augmented Language Models
by: Ori Ram, et al.
Published: (2023-11-01)

Layered Query Retrieval: An Adaptive Framework for Retrieval-Augmented Generation in Complex Question Answering for Large Language Models
by: Jie Huang, et al.
Published: (2024-11-01)

Geographic Adaptation of Pretrained Language Models
by: Valentin Hofmann, et al.
Published: (2024-04-01)

MVR: Synergizing Large and Vision Transformer for Multimodal Natural Language-Driven Vehicle Retrieval
by: Tareq Mahmod AlZubi, et al.
Published: (2025-01-01)

Opportunities for retrieval and tool augmented large language models in scientific facilities
by: Michael H. Prince, et al.
Published: (2024-11-01)

Identification of LSA Data Retrieval Method and Temporal Graph for Document Retrieval
by: Shahla Rezvani, et al.
Published: (2025-01-01)

Embedding Web-Based Statistical Translation Models in Cross-Language Information Retrieval
by: Wessel Kraaij, et al.
Published: (2021-03-01)

Compounds in dictionary-based Cross-language information retrieval_revised
Published: (2002-01-01)

Leveraging Retrieval-Augmented Generation for Swahili Language Conversation Systems
by: Edmund V. Ndimbo, et al.
Published: (2025-01-01)

A Critique of the Book: Natural Language Processing and Information Retrieval
by: Mortaza Kokabi
Published: (2020-12-01)

A predictive human model of language challenges traditional views in linguistics and pretrained transformer research
by: Torres-Martínez Sergio
Published: (2024-12-01)

Precipitation Science: Observations, Retrievals, and Modeling
by: Francisco J. Tapiador, et al.
Published: (2015-01-01)

Erratum: Measuring and Improving Consistency in Pretrained Language Models
by: Yanai Elazar, et al.
Published: (2022-01-01)

Information Retrieval Systems
by: Yaşar Tonta
Published: (1995-09-01)

A nomogram model to predict non-retrieval of short-term retrievable inferior vena cava filters
by: Lihao Qin, et al.
Published: (2024-12-01)

Effectiveness of a large language model for clinical information retrieval regarding shoulder arthroplasty
by: Jacob F. Oeding, et al.
Published: (2024-10-01)

EpiGePT: a pretrained transformer-based language model for context-specific human epigenomics
by: Zijing Gao, et al.
Published: (2024-12-01)

FaN-REMs: Fair and Normalized Retrieval Evaluation Metrics for Learning Retrieval Systems
by: Amar Jaiswal, et al.
Published: (2024-01-01)

Retrieve Fast, Rerank Smart: Cooperative and Joint Approaches for Improved Cross-Modal Retrieval
by: Gregor Geigle, et al.
Published: (2022-05-01)

English Audio Language Retrieval Based on Adaptive Speech-Adjusting Algorithm
by: Xiaoyan Feng, et al.
Published: (2021-01-01)

On conceptual models for information seeking and retrieval research
by: Kalervo Jarvelin, et al.
Published: (2003-01-01)

Perbandingan Pretrained Model Transformer pada Deteksi Ulasan Palsu
by: Aisyah Awalina, et al.
Published: (2022-06-01)

Norm Retrievable Wavelet Systems
by: Mahdieh sadat Aghaei, et al.
Published: (2024-10-01)

Introduction to Modern Information Retrieval /
by: Chowdhury, G. G. (Gobinda G.)
Published: (2010)

Mathematical Foundations of Information Retrieval
by: Sándor Dominich
Published: (2021-03-01)

Technologies of network audio retrieval
by: ZHANG Wei-qiang, et al.
Published: (2007-01-01)

Information Retrieval Systems in Information Chain, Library Institution and Other Information Centers in Information Retrieval
by: Oya Gürdal Tamdoğan
Published: (2013-11-01)

Synchronised Object Retrieval: the enhancement of information retrieval performance in multimedia environments using synchronisation protocols
by: Brophy P.
Published: (2003-01-01)

Using Large Language Models to Retrieve Critical Data from Clinical Processes and Business Rules
by: Yunguo Yu, et al.
Published: (2024-12-01)

Combined query embroidery image retrieval based on enhanced CNN and blend transformer
by: Xinzhen Zhuo, et al.
Published: (2024-11-01)

Retrieving the Lost Self: The Greening of America, Easy Rider and the Politics of Countercultural Whiteness
by: Till Kadritzke
Published: (2020-06-01)

Evaluation of the integration of retrieval-augmented generation in large language model for breast cancer nursing care responses
by: Ruiyu Xu, et al.
Published: (2024-12-01)

Fine-Tuning Retrieval-Augmented Generation with an Auto-Regressive Language Model for Sentiment Analysis in Financial Reviews
by: Miehleketo Mathebula, et al.
Published: (2024-11-01)

Retrieval‐augmented generation versus document‐grounded generation: a key distinction in large language models
by: Shunsuke Koga, et al.
Published: (2025-01-01)

Enhancement of the Performance of Large Language Models in Diabetes Education through Retrieval-Augmented Generation: Comparative Study
by: Dingqiao Wang, et al.
Published: (2024-11-01)

Unfolded Algorithms for Deep Phase Retrieval
by: Naveed Naimipour, et al.
Published: (2024-12-01)

Evaluation criteria for information retrieval systems.
by: Julian Warner
Published: (1999-01-01)

Laparoscopic Retrieval of a Peritoneal Mouse
by: Dara O. Kavanagh, et al.
Published: (2010-01-01)

Image information measurement for video retrieval
by: Qing-sheng YUAN, et al.
Published: (2016-02-01)

A Sequence-to-Sequence Transformer Model for Satellite Retrieval of Aerosol Optical and Microphysical Parameters from Space
by: Luo Zhang, et al.
Published: (2024-12-01)