Methods of automation and interpretation of the quality educational tests in distributed virtual learning systems
DOI:
https://doi.org/10.30837/ITSSI.2023.25.040Keywords:
software engineering; knowledge bases; algebra of finite predicates; use of knowledge; rulesAbstract
The subject matter of the article is the development of mathematical and algorithmic support for intellectual tools, which allows you to conduct continuous control of the knowledge of students (subjects of study) objectively and comprehensively. The goal of the work is to create methods for assessing the quality of educational tests and automating such processes. The following tasks were solved in the article: formation of a testing model in a distributed virtual learning environment and a validity assessment model based on the content of sets of test tasks. The following methods used are – algebra of finite predicates and operations, methods of mathematical statistics and methods of intellectual data analysis. The following results were obtained – the principles of intellectual analysis of the value of reliability coefficients, validity coefficients, discriminability coefficient, the difficulty index of the task of assessing the knowledge of subjects of training are formulated. Conclusions: the application of methods of formalization of test evaluation, analysis of software requirements, software development confirm the need to introduce quantitative methods of assessing students' knowledge into educational practice. The introduction of quantitative methods involves the correct setting of control goals, the selection of the measurement object and the selection of measurement tools. The use of pedagogical tests contributes to the effective implementation of all control functions and corresponds to its main principles for solving the problem of assessing the quality of tests, based on distributed virtual learning models and analysis methods. A test example of calculations was developed, with the help of which the efficiency of the proposed methods was researched.
References
Список літератури
Соколовський Я. І., Сторожук О. Л., Крошний І. М. Застосування сучасних інформаційно-комунікаційних технологій для організації дистанційного навчання. Науковий вісник НЛТУ України. 2015. Вип. 25.6. С. 243–251. URL: http://www.irbis-nbuv.gov.ua/cgi-bin/irbis_nbuv/cgiirbis_64.exe?I21DBN=LINK&P21DBN=UJRN&Z21ID=&S21REF=10&S21CNR=20&S21STN=1&S21FMT=ASP_meta&C21COM=S&2_S21P03=FILA=&2_S21STR=nvnltu_2015_25
Learning management system. URL: https://en.wikipedia.org/wiki/Learning_management_system (дата звернення: 26.08.2023).
Backer R. Educational data mining and learning analytics. The Cambridge handbook of the learning sciences. 2019. 274 р. DOI: 10.1017/CBO9781139519526.016
A Guide to the Scrum Body of Knowledge (SBOK™ GUIDE) URL: https://www.scrumstudy.com/SBOK/SCRUMstudy-SBOK-Guide-3rd.pdf (дата звернення: 26.08.2023).
Gruzdo I., Kyrychenko I., Tereshchenko G., Shanidze O. Analysis of Models Usability Methods Used on Design Stage to Increase Site Optimization Proceedings of the 7th International Conference on Computational Linguistics and Intelligent Systems. (COLINS), Volume III: Intelligent Systems Workshop, 2023. In CEUR Workshop Proceedings, Vol-3403, Р. 387–4093. URL: https://ceur-ws.org/Vol-3403/paper31.pdf
Shubin І. Development of conjunctive decomposition tools. CEUR Workshop Proceedings, 2021. Р. 890–900. URL: https://ceur-ws.org/Vol-2870/
Karataiev O., Sitnikov D., Sharonova N. A Method for Investigating Links between Discrete Data Features in Knowledge Bases in the Form of Predicate Equations, CEUR Workshop Proceedings, 2023. Р. 224–235. URL: https://ceur-ws.org/Vol-3387/paper17.pdf
Тест як інструмент педагогічного моніторингу. URL: http://opentest.com.ua/test-kak-instrument-izmereniya-urovnya-znanij (дата звернення: 20.08.2023).
MyTestXPro – система програм для створення та проведення комп’ютерного тестування, збору та аналізу їх результатів. URL: http://mytest.net (дата звернення: 28.08.2023).
Комп’ютерна програма тестування OpenTEST2. URL: http://opentest.com. ua/kompyuternaya-programma-testirovaniya-znanij-opentest-2 (дата звернення: 21.08.2023).
Конструктор тестів Keepsoft. URL: http://www.keepsoft.ru/simulator.htm. (дата звернення: 20.08.2023.)
Безкоштовна програма для тестування знань та онлайн підготовки. URL: http://xtls.org.ua/ test.html (дата звернення: 20.08.2023).
Sharonova N. et al. Issues of Fact-based Information Analysis. International Conference on Computational Linguistics and Intelligent Systems. 2018. 178 р. URL: http://web.kpi.kharkov.ua/iks/wp-content/uploads/sites/113/2021/10/preface_colins_volume2_2018.pdf
Williams P. E-learning: what the literature tells us about distance education. An overview. Aslib Proceedings. Vol. 57. 2005. P 109–122. DOI: https://doi.org/10.1108/00012530510589083
Omran P. G., Wang K., Wang Z. An Embedding-based Approach to Rule Learning in Knowledge Graphs, IEEE Transactions on Knowledge and Data Engineering. Vol. 33(4). 2021. Р. 1348–1359. DOI: 10.1109/TKDE.2019.2941685
Pellissier-Tanon T., Weikum G., Suchanek F. YAGO 4: A Reasonable Knowledge Base, 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31–June 4, 2020. P. 583–596. DOI:10.1007/978-3-030-49461-2_34
Kyrychenko I.,Malikin D. Research of Methods for Practical Educational Tasks Generation Based on Various Difficulty Levels 6th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2022), May 12–13, 2022, Gliwice, Poland. CEUR Workshop Proceedings 3171, Volume I: Main, Р. 1030–1042. URL: https://ceur-ws.org/Vol-3171/paper74.pdf
Omran P. G., Wang Z., Wang K. Scalable rule learning via learning representation, Twenty-Seventh International Joint Conference on Artificial Intelligence. IJCAI-18. 2018. Р. 2149–2155. DOI:10.24963/ijcai.2018/297
Svato M., Schockaert S., Davis J. STRiKE: Rule-Driven Relational Learning Using Stratified k-Entailment, in: ECAI, 2020. URL: https://ida.fel.cvut.cz/~kuzelka/pubs/ecai2020.pdf
Sharonova N., Gruzdo I., Tereshchenko G. Generalized Semantic Analysis Algorithm of Natural Language Texts for Various Functional Style Types. 6th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2022), May 12–13, 2022, Gliwice, Poland. CEUR Workshop Proceedings 3171, Volume I: Main, Р. 16–26. URL: https://ceur-ws.org/Vol-3171/paper4.pdf
Barkovska O. Research into Speech-to-text Transformation Module in the Proposed Model of a Speaker’s Automatic Speech Annotation. Innovative Technologies and Scientific Solutions for Industries. 2022. № 4 (22). Р. 5–13. DOI: https://doi.org/10.30837/ITSSI.2022.22.005
References
Sokolovskyi, Ya, Storozhuk, O., Kroshnyi, I. "The use of modern information and communication technologies for the organization of distance learning". Scientific Bulletin of the National Technical University of Ukraine. 2015. Issue 25.6. P. 243–251. available at: http://www.irbis-nbuv.gov.ua/cgi-bin/irbis_nbuv/cgiirbis_64.exe?I21DBN=LINK&P21DBN=UJRN&Z21ID=&S21REF=10&S21CNR=20&S21STN=1&S21FMT=ASP_meta&C21COM=S&2_S21P03=FILA=&2_S21STR=nvnltu_2015_25
"Learning management system". available at: https://en.wikipedia.org/wiki/Learning_management_system (last accessed 26.08.2023).
Backer, R. (2019), Educational data mining and learning analytics. The Cambridge handbook of the learning sciences. 274 р. DOI: 10.1017/CBO9781139519526.016
"A Guide to the Scrum Body of Knowledge (SBOK™ GUIDE)", available at: https://www.scrumstudy.com/SBOK/SCRUMstudy-SBOK-Guide-3rd.pdf (last accessed 26.08.2023).
Gruzdo, I., Kyrychenko, I., Tereshchenko, G., Shanidze, O. "Analysis of Models Usability Methods Used on Design Stage to Increase Site Optimization" Proceedings of the 7th International Conference on Computational Linguistics and Intelligent Systems. (COLINS), Volume III: Intelligent Systems Workshop, 2023. In CEUR Workshop Proceedings, Vol-3403, Р. 387–4093. available at: https://ceur-ws.org/Vol-3403/paper31.pdf
Shubin, І. "Development of conjunctive decomposition tools". CEUR Workshop Proceedings, 2021. Р. 890–900. available at: https://ceur-ws.org/Vol-2870/
Karataiev, O., Sitnikov, D., Sharonova, N. "A Method for Investigating Links between Discrete Data Features in Knowledge Bases in the Form of Predicate Equations", CEUR Workshop Proceedings, 2023, Р. 224–235. available at: https://ceur-ws.org/Vol-3387/paper17.pdf
"The test as a tool of pedagogical monitoring" ["Test yak instrument pedahohichnoho monitorynhu"] available at: http://opentest.com.ua/test-kak-instrument-izmereniya-urovnya-znanij (last accessed 20.08.2023.)
"MyTestXPro – system of programs for creating and conducting computer testing, collecting and analyzing their results". ["Systema prohram dlia stvorennia ta provedennia kompiuternoho testuvannia, zboru ta analizu yikh rezultativ"] available at: http://mytest.net (last accessed 28.08.2023).
"Computer testing program OpenTEST2". ["Kompiuterna prohrama testuvannia OpenTEST2"]. available at: http://opentest.com. ua/kompyuternaya-programma-testirovaniya-znanij-opentest-2. (last accessed 21.08.2023).
"Test designer Keepsoft". ["Konstruktor testiv Keepsoft"]. available at: http://www.keepsoft.ru/simulator.htm. (last accessed 20.08.2023).
"Free program for knowledge testing and online preparation". ["Bezkoshtovna prohrama dlia testuvannia znan ta onlain pidhotovky"]. available at: http://xtls.org.ua/ test.html (last accessed 20.08.2023).
Sharonova, N. et al. "Issues of Fact-based Information Analysis". International Conference on Computational Linguistics and Intelligent Systems. 2018. 178 р. available at: http://web.kpi.kharkov.ua/iks/wp-content/uploads/sites/113/2021/10/preface_colins_volume2_2018.pdf
Williams, P. (2005), "E-learning: what the literature tells us about distance education". An overview. Aslib Proceedings. Vol. 57. P 109–122. DOI: https://doi.org/10.1108/00012530510589083
Omran, P. G., Wang, K., Wang, Z. (2021), "An Embedding-based Approach to Rule Learning in Knowledge Graphs", IEEE Transactions on Knowledge and Data Engineering. Vol. 33(4). Р. 1348–1359. DOI: 10.1109/TKDE.2019.2941685
Pellissier-Tanon, T., Weikum, G., Suchanek, F. (2020), "F. YAGO 4: A Reasonable Knowledge Base", 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31–June 4. P. 583–596. DOI:10.1007/978-3-030-49461-2_34
Kyrychenko, I., Malikin, D. "Research of Methods for Practical Educational Tasks Generation Based on Various Difficulty Levels" 6th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2022), May 12–13, 2022, Gliwice, Poland. CEUR Workshop Proceedings 3171, Volume I: Main, Р. 1030–1042. available at: https://ceur-ws.org/Vol-3171/paper74.pdf
Omran, P. G., Wang, Z., Wang, K. (2018), "Scalable rule learning via learning representation", Twenty-Seventh International Joint Conference on Artificial Intelligence. IJCAI-18. Р. 2149-2155. DOI:10.24963/ijcai.2018/297
Svato, M., Schockaert, S., Davis, J. "STRiKE: Rule-Driven Relational Learning Using Stratified k-Entailment", in: ECAI, 2020. available at: https://ida.fel.cvut.cz/~kuzelka/pubs/ecai2020.pdf
Sharonova, N., Gruzdo, I., Tereshchenko, G. "Generalized Semantic Analysis Algorithm of Natural Language Texts for Various Functional Style Types". 6th International Conference on Computational Linguistics and Intelligent Systems (COLINS-2022), May 12–13, 2022, Gliwice, Poland. CEUR Workshop Proceedings 3171, Volume I: Main, Р. 16–26. available at: https://ceur-ws.org/Vol-3171/paper4.pdf
Barkovska, O. (2022), Research into Speech-to-text Transformation Module in the Proposed Model of a Speaker’s Automatic Speech Annotation. Innovative Technologies and Scientific Solutions for Industries. № 4 (22). Р. 5–13. DOI: https://doi.org/10.30837/ITSSI.2022.22.005
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Our journal abides by the Creative Commons copyright rights and permissions for open access journals.
Authors who publish with this journal agree to the following terms:
Authors hold the copyright without restrictions and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
Authors are able to enter into separate, additional contractual arrangements for the non-commercial and non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
Authors are permitted and encouraged to post their published work online (e.g., in institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.
