The objective of this project is to explore knowledge distillation and fine-tuning techniques to reduce the model parameter count in diffusion models. Diffusion models are known to demand significant GPU resources during inference, which hinders their effectiveness on edge devices. By adopting knowledge distillation and fine-tuning, we aim to mitigate this issue and make diffusion models more feasible for deployment on edge devices.