Using Hidden Markov Model to improve the accuracy of Punjabi POS tagger

POS tagger is the process of assigning a correct tag to each word of the sentence. Accuracy of all NLP tasks like grammar checker, phrase chunker, machine translation etc. depends upon the accuracy of the POS tagger. We attempted to improve the accuracy of existing Punjabi POS tagger. This POS tagge...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2011 IEEE International Conference on Computer Science and Automation Engineering Ročník 2; s. 697 - 701
Hlavní autori: Sharma, S. K., Lehal, G. S.
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 01.06.2011
Predmet:
Accuracy
Hidden Markov models
HMM
Natural language processing
POS
Probability
Punjabi
Speech
Tagging
Training
ISBN:9781424487271, 1424487277
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:POS tagger is the process of assigning a correct tag to each word of the sentence. Accuracy of all NLP tasks like grammar checker, phrase chunker, machine translation etc. depends upon the accuracy of the POS tagger. We attempted to improve the accuracy of existing Punjabi POS tagger. This POS tagger lacks in resolving the ambiguity of compound and complex sentences. A Bi-gram Hidden Markov Model has been used to solve the part of speech tagging problem. An annotated corpus of 20,000 words was used for training and estimating of HMM parameter. Maximum likelihood method has been used to estimate the parameter. This HMM approach has been implemented by using Viterby algorithm. A module has been developed that takes the existing POS tagger output as input and assign the correct tag to the words having more than one tag. Our module was tested on the corpus containing 26,479 words. The accuracy of 90.11% was evaluated using manual approach.
ISBN:9781424487271
1424487277
DOI:10.1109/CSAE.2011.5952600