A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation
Abstract Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Po...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-12-01
|
| Series: | Engineering Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/eng2.12952 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846121502712266752 |
|---|---|
| author | Zengkun Liu Yuqian Lu |
| author_facet | Zengkun Liu Yuqian Lu |
| author_sort | Zengkun Liu |
| collection | DOAJ |
| description | Abstract Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals. |
| format | Article |
| id | doaj-art-651deeba15cd4619ad47b918d840e0e5 |
| institution | Kabale University |
| issn | 2577-8196 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | Engineering Reports |
| spelling | doaj-art-651deeba15cd4619ad47b918d840e0e52024-12-16T03:09:12ZengWileyEngineering Reports2577-81962024-12-01612n/an/a10.1002/eng2.12952A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automationZengkun Liu0Yuqian Lu1Department of Mechanical Engineering The University of Auckland Auckland New ZealandDepartment of Mechanical Engineering The University of Auckland Auckland New ZealandAbstract Maintenance manuals are crucial information sources for maintenance and repair. Prior studies explored factual knowledge extraction from textual documents. However, maintenance knowledge in manuals is more task‐centric rather than factual knowledge and often documented in an unstructured Portable Document Format (PDF), posing challenges for knowledge extraction. Addressing this, this research develops effective methods to extract task‐centric maintenance knowledge from unstructured PDF manuals. A new Task‐centric Knowledge Graph (TCKG) schema centralized on maintenance task components (MTCs) is proposed to address the need for structured knowledge representation. A method (Heterogeneous Graph‐based Method, HGM) for knowledge extraction is then proposed, which is enhanced by incorporating visual and spatial information. In the experiments, the proposed HGM exhibits robust performance in the knowledge extraction process, surpassing the baseline Graph‐based Interaction Model with a Tracker (GIT) method in MTCs extraction by 13.3%, and the baseline Translate Embedding (TransE) method in MTCs' relation extraction by 3.8%. A series of ablation studies also prove that including visual and spatial information through the proposed method can improve the relation extraction performance by over 10%. This research supplies valuable insights for future developments in information extraction from maintenance manuals.https://doi.org/10.1002/eng2.12952industrial maintenancemaintenance manualmulti‐modal representation learningtask‐centric knowledge graph |
| spellingShingle | Zengkun Liu Yuqian Lu A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation Engineering Reports industrial maintenance maintenance manual multi‐modal representation learning task‐centric knowledge graph |
| title | A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation |
| title_full | A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation |
| title_fullStr | A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation |
| title_full_unstemmed | A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation |
| title_short | A task‐centric knowledge graph construction method based on multi‐modal representation learning for industrial maintenance automation |
| title_sort | task centric knowledge graph construction method based on multi modal representation learning for industrial maintenance automation |
| topic | industrial maintenance maintenance manual multi‐modal representation learning task‐centric knowledge graph |
| url | https://doi.org/10.1002/eng2.12952 |
| work_keys_str_mv | AT zengkunliu ataskcentricknowledgegraphconstructionmethodbasedonmultimodalrepresentationlearningforindustrialmaintenanceautomation AT yuqianlu ataskcentricknowledgegraphconstructionmethodbasedonmultimodalrepresentationlearningforindustrialmaintenanceautomation AT zengkunliu taskcentricknowledgegraphconstructionmethodbasedonmultimodalrepresentationlearningforindustrialmaintenanceautomation AT yuqianlu taskcentricknowledgegraphconstructionmethodbasedonmultimodalrepresentationlearningforindustrialmaintenanceautomation |