Potential of Large Language Models for Ontology Development in Construction Domain

Abstract

In any domain, effective knowledge management is crucial for informed, rational, and explainable decision-making. Ontologies help in structuring and organizing knowledge, making it easier to find, apply, and justify the decisions based on it. However, purely manual ontology development presents challenges due to its time-consuming nature and the broad expertise it requires. Therefore, this paper explores the integration of Large Language Models (LLMs) in ontology development. Our work was motivated by the assumption that LLMs can automate the extraction of ontology elements such as concepts, relations, and constraints, reducing the burden on domain experts when creating a shared conceptualization. This can be achieved either by focusing on the entire corpus of text the LLMs were trained on or by concentrating on just a few documents, such as a scientific paper or a standard. Through case studies and examples, we investigate and evaluate the potential of LLMs in optimizing ontology development for the construction building safety domain. The optimized prompts significantly improved the performance metrics of GPT-3.5. Self-evaluations by GPT showed an F1 score of 0.78, while expert evaluations recorded a lower score of 0.47, indicating some overestimation by the model. These findings confirm LLM utility in ontology development and show the potential for improvement of GPT. Additionally, we discuss challenges, considerations, and future directions in this evolving field, highlighting the significance of integrating LLMs for improved knowledge management in the construction industry.