Layer 일부 Freeze, fine-tuning

for name, param in model.named_parameters():
        if [LAYER_NAME] in name:
            param.requires_grad = True
        else:
            param.requires_grad = False

ex)
for name, param in model.named_parameters():
        if 'rec_head' in name:
            param.requires_grad = True
        else:
            param.requires_grad = False

 

https://soyoung97.github.io/pytorch-model-%EC%9D%BC%EB%B6%80-layer%EB%A7%8C-freeze-%ED%95%98%EA%B8%B0/

Pytorch Model 일부 Layer만 Freeze 하기

 

optimizer에서 grad 에러 뜰 경우,

optimizer = torch.optim.Adam([param for param in model.parameters() if param.requires_grad == True], lr=cfg.train_cfg.lr)

https://gist.github.com/L0SG/2f6d81e4ad119c4f798ab81fa8d62d3f

PyTorch example: freezing a part of the net (including fine-tuning)

https://github.com/pytorch/pytorch/issues/679