【Metrics Seminar】Estimation of Dynamic Discrete Choice Models with Unobserved State Variables Using Reinforcement Learning
2024/07/10
研討會日期 : 2024-07-10
時間 : 14:30
主講人 : Professor Yingyao Hu
地點 : B110會議室
主持人 : Professor Yu-Chin Hsu
演講者簡介 : Professor Yingyao Hu received his PhD from Johns Hopkins University in 2003. He is Krieger-Eisenhower Professor of Economics and Vice Dean for Social Sciences at Johns Hopkins University. His research interests include micro-econometrics, empirical industrial organization, and labor economics.
演講摘要 : Dynamic discrete choice models (DDCs) pose significant computational challenges, especially when dealing with high-dimensional state spaces and unobserved heterogeneity. To address these challenges, this paper introduces a unified estimation framework that integrates policy gradient methods from reinforcement learning with the established indirect inference approach. By directly parameterizing and estimating the policy function using the policy gradient method in the inner loop of the full model solution, this method builds a mapping from deep structural parameters to optimal policy function parameters, significantly reducing the computational burden of the estimation. Notably, the framework is adaptable to models with partially observed state variables, demonstrating efficacy in estimating models with various types of unobserved state variables under specific conditions. Using this method, discretizing the unobserved state variable is no longer necessary, making the estimation of DDCs with continuous, time-varying unobserved state variables tractable. Empirical validation across various model specifications confirms the effectiveness of the proposed framework, with estimates closely aligning with true parameter values. By enhancing computational efficiency and accuracy, the proposed framework facilitates more comprehensive analyses of dynamic decision-making processes.
