Speech understanding for spoken dialogue systems: from corpus harvesting to grammar rule inductionSpeech understanding for spoken dialogue systems: from corpus harvesting to grammar rule induction Peer-Reviewed Journal Publication Δημοσίευση σε Περιοδικό με Κριτές 2019-11-142018enWe investigate algorithms and tools for the semi-automatic authoring of grammars for spoken dialogue systems (SDS) proposing a framework that spans from corpora creation to grammar induction algorithms. A realistic human-in-the-loop approach is followed balancing automation and human intervention to optimize cost to performance ratio for grammar development. Web harvesting is the main approach investigated for eliciting spoken dialogue textual data, while crowdsourcing is also proposed as an alternative method. Several techniques are presented for constructing web queries and filtering the acquired corpora. We also investigate how the harvested corpora can be used for the automatic and semi-automatic (human-in-the-loop) induction of grammar rules. SDS grammar rules and induction algorithms are grouped into two types, namely, low- and high-level. Two families of algorithms are investigated for rule induction: one based on semantic similarity and distributional semantic models, and the other using more traditional statistical modeling approaches (e.g., slot-filling algorithms using Conditional Random Fields). Evaluation results are presented for two domains and languages. High-level induction precision scores up to 60% are obtained. Results advocate the portability of the proposed features and algorithms across languages and domains.http://creativecommons.org/licenses/by/4.0/Computer Speech and Language47January 2018272-297 Iosif Ilias Ιωσηφ Ηλιας Klasinas Ioannis Κλασινας Ιωαννης Athanasopoulou Georgia Αθανασοπουλου Γεωργια Palogiannidi Elisavet Παλογιαννιδη Ελισαβετ Georgiladakis Spyridon Γεωργιλαδακης Σπυριδων Louka Katerina Potamianos Alexandros Ποταμιανος Αλεξανδρος Elsevier Corpora creation Crowdsourcing Grammar induction Semantic similarity Spoken dialogue systems Web mining