Question Answering is a type of information retrieval. Then, Speech Question Answering is a kind of information retrieval which requires complex natural language processing techniques together with the speech recognition techniques. Most probably, QA system is used at web site by giving certain questions as the input and the web server will try to retrieve the answer from the database. Besides, there is another type of informational retrieve system called telephony IVR (Interactive Voice Recognition) system which is very commonly used. Most of the question answering system which is probably used is somehow a unidirectional communication with the user, that is, the system is not able enough to have interact ion with the users, however, there is a trend of developing a bidirectional question answering system with some interactive dialogues as the importance of artificial intelligence is running up. Here, our research is following this trend by using speech. Here, our focus will be on researching this field and let the East Asian Games (EAG) be the domain of our research. That is, the spoken question will be recognized by our speech engine and our system will extract a correct answer, or gives some other communicative dialogues if the question is out of our domain, in response to the questions spoken by the user. There are two main parts in our research: one is QA system and the other one is the speech recognition development.