Federated Learning (FL) is a machine learning method that trains a different model on decentralized devices or servers, which can protect private data and information. In medical image analysis, to protect data privacy across different clinics, FL is a useful method. However, the distributed nature of FL gives rise to new threats caused by potentially malicious participants. In this project, we are aimed to build up a federated learning framework for medical image segmentation in CT/MRI and poison some of the data to simulate the attack from hackers. Finally, we propose a defense strategy for CT/MRI data that can help identify malicious participants in FL to circumvent poisoning attacks and demonstrate its effectiveness.