Projects

iLQR given a learned cost map seeded with an initial high-cost trajectory (red) generates an optimized trajectory that avoids high-cost regions (yellow).iLQR using Learned Cost Maps was my group's final project for CMU 16-745 Optimal Control and Reinforcement Learning (with teammates Matthew Sivaprakasam and Nayana Suvarna). We focused on exploring alternate control methods for off-road autonomy. Model Predictive Path Integral Control (MPPI) is often the standard for off-road autonomy because it can be accelerated with a GPU and can handle non-differentiable cost maps often generated by neural networks. MPPI comes with drawbacks in that it requires GPU to run in real-time, and is highly stochastic. Instead, we focused on applying iLQR to non-differentiable cost-maps running on the CPU alone.

We performed our optimization on cost maps generated via inverse reinforcement learning trained on the TartanDrive 2.0 dataset. This cost map accounts for geometric features such as ditches, slopes, and uneven terrain. The gradients and hessians of the cost map are pre-computed and re-used in each optimization step. Without any processing, the gradients and hessians would be very noisy, so a Gaussian blur is applied before differentiation. The optimization is seeded with the lowest cost path from a small trajectory library using the kinematic bicycle model. At this point, the iLQR algorithm is performed as normal to generate a low-cost trajectory.

We found that this process generates a lower cost trajectory than the initial input. To demonstrate this, we tested this by optimizing given a poor initial trajectory with high cost. As seen in the image, the generated trajectory avoids high-cost regions. We also used Numba to accelerate the Python code so that the method performs comparably on the CPU to MPPI.