Development of local knowledge base application using retrieval augmented generation technology
Retrieval Augmented Generation (RAG) technology can enable large language models to access external knowledge bases by introducing external documents, thereby large language models can generate more authentic and reliable answers, and effectively solve the problems of outdated data and insufficient...
Saved in:
Main Authors: | , |
---|---|
Format: | Article |
Language: | zho |
Published: |
Editorial Department of Journal on Communications
2024-11-01
|
Series: | Tongxin xuebao |
Subjects: | |
Online Access: | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024227/ |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
_version_ | 1841537100999032832 |
---|---|
author | ZHU Junyi ZHU Shangming |
author_facet | ZHU Junyi ZHU Shangming |
author_sort | ZHU Junyi |
collection | DOAJ |
description | Retrieval Augmented Generation (RAG) technology can enable large language models to access external knowledge bases by introducing external documents, thereby large language models can generate more authentic and reliable answers, and effectively solve the problems of outdated data and insufficient corpus. On the basis of introducing the basic architecture and fine-tuning techniques of large language models, the application framework of using retrieval enhanced generation technology to build a local knowledge base system was discussed. The application framework consisted of six parts: loading local documents, splitting documents, embedding splitting fragments, matching text based on questions, constructing prompts, and generating responses. Finally, based on the ERNIE-4.0 model and the AppBuilder development platform, an intelligent question answering system for campus information services was designed and developed, and a specific implementation was provided. |
format | Article |
id | doaj-art-9123383857df4e53baec03aa1ad19fa6 |
institution | Kabale University |
issn | 1000-436X |
language | zho |
publishDate | 2024-11-01 |
publisher | Editorial Department of Journal on Communications |
record_format | Article |
series | Tongxin xuebao |
spelling | doaj-art-9123383857df4e53baec03aa1ad19fa62025-01-14T08:46:45ZzhoEditorial Department of Journal on CommunicationsTongxin xuebao1000-436X2024-11-014524224779661895Development of local knowledge base application using retrieval augmented generation technologyZHU JunyiZHU ShangmingRetrieval Augmented Generation (RAG) technology can enable large language models to access external knowledge bases by introducing external documents, thereby large language models can generate more authentic and reliable answers, and effectively solve the problems of outdated data and insufficient corpus. On the basis of introducing the basic architecture and fine-tuning techniques of large language models, the application framework of using retrieval enhanced generation technology to build a local knowledge base system was discussed. The application framework consisted of six parts: loading local documents, splitting documents, embedding splitting fragments, matching text based on questions, constructing prompts, and generating responses. Finally, based on the ERNIE-4.0 model and the AppBuilder development platform, an intelligent question answering system for campus information services was designed and developed, and a specific implementation was provided.http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024227/large language modelretrieval augmented generationpromptlocal knowledge baseintelligent question answering system |
spellingShingle | ZHU Junyi ZHU Shangming Development of local knowledge base application using retrieval augmented generation technology Tongxin xuebao large language model retrieval augmented generation prompt local knowledge base intelligent question answering system |
title | Development of local knowledge base application using retrieval augmented generation technology |
title_full | Development of local knowledge base application using retrieval augmented generation technology |
title_fullStr | Development of local knowledge base application using retrieval augmented generation technology |
title_full_unstemmed | Development of local knowledge base application using retrieval augmented generation technology |
title_short | Development of local knowledge base application using retrieval augmented generation technology |
title_sort | development of local knowledge base application using retrieval augmented generation technology |
topic | large language model retrieval augmented generation prompt local knowledge base intelligent question answering system |
url | http://www.joconline.com.cn/zh/article/doi/10.11959/j.issn.1000-436x.2024227/ |
work_keys_str_mv | AT zhujunyi developmentoflocalknowledgebaseapplicationusingretrievalaugmentedgenerationtechnology AT zhushangming developmentoflocalknowledgebaseapplicationusingretrievalaugmentedgenerationtechnology |