Development of a computer system for generating semantic template of a group of documents by using latent semantic analysis - Taranenko, Yuriy; Kabanova, Maryna (Eastern-European Journal of Enterprise Technologies, 2016)
The CS was developed by means of the Python programming language to generate a semantic template of a group of documents by the LSA method. The system contains eight software modules, each performs one stage of the LSA. The control module of the frequency word-document matrix and the measuring module of semantic distance between the template documents are unique. Adjustment of CS to the contents and structure of the document templates is performed by changing a set of modules. According to the research, the frequency matrix normalization enhances the resolution of the semantic template generated by using the LSA. It is proved that the removal of individual words improves the resolution of the generated semantic template and does not affect the semantic content. Application of semantic proximity of documents, the cosine of the difference of angles between the vector of a group of basic words and vectors of documents for evaluation allows increasing the resolution of the generated semantic template. To ensure the continuity of the LSA, the module of the frequency matrix analysis for compliance of excess (or equality) of the number of words over the number of documents was introduced in the CS. In the event of a mismatch, the module starts over the LSA process with a new set of words and documents after removal of the inappropriate document and related words.