A Simulation-based Optimization Approach for Integrated Outpatient Flow and Medication Management

In this research, first, a cancer outpatient and medication management system is studied, which reflects realistic features of patient and medication planning process and focuses on maximizing profitability and improving scheduling efficiency in a cancer clinic. Decision-making under uncertain environments, that can be effectively handled by simulation-based optimization is considered, and a learning-based patient and medication management (LPMM) algorithm is developed, that can digest a complex decision-making structure of cancer outpatient and medication management, and improve resulting schedules effectively. A Markovian model is then developed to optimize patient admission and assignment problems in a general outpatient setting with different treatment types and considering common appointment cancellations. The aim of this model is to admit walk-in patients and assign patients to treatment slots, considering patient's wait time and common appointment cancellations, doctor's idle and overtime, and dissatisfaction of not admitted walk-in patients. The optimal policy of patient admission and assignment is explored, applying a learning-based general outpatient management system (LGOM). The decision-making core of both LPMM and LGOM is a hierarchical deep Q-network (HDQN).  LPMM and LGOM are trained with simulation data, to obtain an intermediate off-line policy, then implemented to the live situation, to update the policy further, and to reflect the real feedback. The performance of HDQN-based LPMM is assessed in a cancer outpatient and medication problem with 576 states and 243 possible actions in each state. The performance of HDQN-based LGOM is also analyzed in a problem with 98,304 states and 6 possible actions in each state. Both HDQN-based LPMM and LGOM algorithms converged to an optimal policy, with a higher max reward per state compared to the flat Q-learning and DQN. The optimal policy of both algorithms provides insightful points regarding patient and medication management strategies that should be followed by healthcare facilities.