Development of an ontology-based method for scalable generation of Partially Observable Markov Decision Process (POMDP) models
DOI:
https://doi.org/10.15587/2706-5448.2026.361385Keywords:
ontology, knowledge graphs, automated model generation, probabilistic planning, POMDPAbstract
The object of research is the automated construction of partially observable Markov decision process (POMDP) models from a semantic knowledge base, in which probabilistic parameters are formalized as an ontology or knowledge graph.
The problem addressed in this research is that the traditional approach requires manual construction of transition, observation, reward, and cost tables, which results in low data traceability. This significantly complicates model auditing and updating, and hinders reconciliation in cases of conflicting information sources or changes in domain knowledge.
A domain-independent schema has been developed that represents each numeric POMDP parameter as a provenance-aware claim, including the information source and timestamp.
A deterministic compilation method has been implemented that selects claims via SPARQL queries, resolves conflicts according to a policy, normalizes parameters, and forms the output tabular values for transition T, observation O, and reward R matrices.
A benchmark evaluation was conducted using a diagnostic task, confirming the model’s high generation speed. In a series of 10 experiments, generation time did not exceed 3.6 s, even for a state space of |S| = 3957, and solution time by the SARSOP solver ranged from a few seconds to a set timeout of 600s, depending on the scaling mode.
Treating parameters as graph entities allows transforming model updates into an explicit claim-editing procedure that is accessible to audit.
The method can be used when the domain knowledge is expressed as ontology with identifiable states/actions/observations, and when parameter provenance matters. It supports maintainable decision-support deployments with auditable parameter histories and handling of conflicting claims, generation of model variants for sensitivity analysis, and controlled extension of the model via ontology learning.
References
- Musen, M. A., Noy, N. F., Shah, N. H., Whetzel, P. L., Chute, C. G., Story, M.-A., Smith, B. et al. (2012). The National Center for Biomedical Ontology. Journal of the American Medical Informatics Association, 19 (2), 190–195. https://doi.org/10.1136/amiajnl-2011-000523
- Hoehndorf, R., Dumontier, M., Gkoutos, G. V. (2012). Evaluation of research in biomedical ontologies. Briefings in Bioinformatics, 14 (6), 696–712. https://doi.org/10.1093/bib/bbs053
- Beard, R. W. (2018). Decision Making Under Uncertainty: Theory and Application. IEEE Control Systems, 38 (6), 114–115. https://doi.org/10.1109/mcs.2018.2866656
- Iocchi, L., Lukasiewicz, T., Nardi, D., Rosati, R. (2009). Reasoning about actions with sensing under qualitative and probabilistic uncertainty. ACM Transactions on Computational Logic, 10 (1), 1–41. https://doi.org/10.1145/1459010.1459015
- Zhang, S., Stone, P. (2015). CORPP: Commonsense Reasoning and Probabilistic Planning, as Applied to Dialog with a Mobile Robot. Proceedings of the AAAI Conference on Artificial Intelligence, 29 (1). https://doi.org/10.1609/aaai.v29i1.9385
- Amiri, S., Shokrolah Shirazi, M., Zhang, S. (2020). Learning and Reasoning for Robot Sequential Decision Making under Uncertainty. Proceedings of the AAAI Conference on Artificial Intelligence, 34 (3), 2726–2733. https://doi.org/10.1609/aaai.v34i03.5659
- Moulouel, K., Chibani, A., Amirat, Y. (2023). Ontology-based hybrid commonsense reasoning framework for handling context abnormalities in uncertain and partially observable environments. Information Sciences, 631, 468–486. https://doi.org/10.1016/j.ins.2023.02.078
- Ghanadbashi, S., Zarchini, A., Golpayegani, F. (2023). An Ontology-Based Augmented Observation for Decision-Making in Partially Observable Environments. Proceedings of the 15th International Conference on Agents and Artificial Intelligence. Lisbon, 343–354. https://doi.org/10.5220/0011793200003393
- Golpayegani, F., Ghanadbashi, S., Zarchini, A. (2024). Advancing Sustainable Manufacturing: Reinforcement Learning with Adaptive Reward Machine Using an Ontology-Based Approach. Sustainability, 16 (14), 5873. https://doi.org/10.3390/su16145873
- Ding, Z., Peng, Y., Pan, R. (2006). BayesOWL: Uncertainty Modeling in Semantic Web Ontologies. Soft Computing in Ontologies and Semantic Web. Springer, 3–29. https://doi.org/10.1007/978-3-540-33473-6_1
- Carvalho, R. N., Laskey, K. B., Costa, P. C. G. (2017). PR-OWL – a language for defining probabilistic ontologies. International Journal of Approximate Reasoning, 91, 56–79. https://doi.org/10.1016/j.ijar.2017.08.011
- Bellodi, E., Lamma, E., Riguzzi, F., Albani, S. (2011). A distribution semantics for probabilistic ontologies. URSW, 778, 75–86. Available at: https://dl.acm.org/doi/10.5555/2887702.2887709
- Lukasiewicz, T. (2008). Probabilistic description logic programs under inheritance with overriding for the Semantic Web. International Journal of Approximate Reasoning, 49 (1), 18–34. https://doi.org/10.1016/j.ijar.2007.08.005
- Belhajjame, K., Cheney, J., Corsar, D., Garijo, D., Soiland-Reyes, S., Zednik, S., Zhao, J.; Lebo, T., Sahoo, S., McGuinness, D. (Eds.) (2013). PROV-O: The PROV ontology. Available at: https://www.w3.org/TR/prov-o/
- Kuhn, T., Chichester, C., Krauthammer, M., Queralt-Rosinach, N., Verborgh, R., Giannakopoulos, G. et al. (2016). Decentralized provenance-aware publishing with nanopublications. PeerJ Preprints. https://doi.org/10.7287/peerj.preprints.1760v1
- Kuhn, T., Banda, J. M., Willighagen, E., Ehrhart, F., Evelo, C., Malas, T. B. et al. (2018). Nanopublications: A Growing Resource of Provenance-Centric Scientific Linked Data. 2018 IEEE 14th International Conference on E-Science (E-Science), 83–92. https://doi.org/10.1109/escience.2018.00024
- Klusch, M., Gerber, A., Schmidt, M. (2005). Semantic web service composition planning with OWLS-XPlan. AAAI Fall Symposium: Agents and the Semantic Web, 55–62. Available at: https://www.researchgate.net/publication/228711042_Semantic_Web_service_composition_planning_with_OWLS-XPlan
- John, T., Koopmann, P. (2023). Towards ontology-mediated planning with OWL DL ontologies. CEUR Workshop Proceedings. https://doi.org/10.48550/arXiv.2308.08200
- Malburg, L., Klein, P., Bergmann, R. (2023). Converting semantic web services into formal planning domain descriptions to enable manufacturing process planning and scheduling in industry 4.0. Engineering Applications of Artificial Intelligence, 126, 106727. https://doi.org/10.1016/j.engappai.2023.106727
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Yaroslav Teplyi, Dmytro Dosyn
This work is licensed under a Creative Commons Attribution 4.0 International License.
The consolidation and conditions for the transfer of copyright (identification of authorship) is carried out in the License Agreement. In particular, the authors reserve the right to the authorship of their manuscript and transfer the first publication of this work to the journal under the terms of the Creative Commons CC BY license. At the same time, they have the right to conclude on their own additional agreements concerning the non-exclusive distribution of the work in the form in which it was published by this journal, but provided that the link to the first publication of the article in this journal is preserved.
