Using Hidden Markov Model to improve the accuracy of Punjabi POS tagger

POS tagger is the process of assigning a correct tag to each word of the sentence. Accuracy of all NLP tasks like grammar checker, phrase chunker, machine translation etc. depends upon the accuracy of the POS tagger. We attempted to improve the accuracy of existing Punjabi POS tagger. This POS tagge...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:2011 IEEE International Conference on Computer Science and Automation Engineering Ročník 2; s. 697 - 701
Hlavní autoři: Sharma, S. K., Lehal, G. S.
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 01.06.2011
Témata:
Accuracy
Hidden Markov models
HMM
Natural language processing
POS
Probability
Punjabi
Speech
Tagging
Training
ISBN:9781424487271, 1424487277
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:POS tagger is the process of assigning a correct tag to each word of the sentence. Accuracy of all NLP tasks like grammar checker, phrase chunker, machine translation etc. depends upon the accuracy of the POS tagger. We attempted to improve the accuracy of existing Punjabi POS tagger. This POS tagger lacks in resolving the ambiguity of compound and complex sentences. A Bi-gram Hidden Markov Model has been used to solve the part of speech tagging problem. An annotated corpus of 20,000 words was used for training and estimating of HMM parameter. Maximum likelihood method has been used to estimate the parameter. This HMM approach has been implemented by using Viterby algorithm. A module has been developed that takes the existing POS tagger output as input and assign the correct tag to the words having more than one tag. Our module was tested on the corpus containing 26,479 words. The accuracy of 90.11% was evaluated using manual approach.
ISBN:9781424487271
1424487277
DOI:10.1109/CSAE.2011.5952600