Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

16 votes1.5K views6 answers
Django forms
Python
-1 vote164 views1 answer
Hue seaborn
Python
0 vote171 views1 answer
How to use the dot lower function
Python
44 votes172 views3 answers
Python string cut last character
Python
0 vote150 views1 answer
How to write flow of execution in python
Python
-1 vote152 views1 answer
Pandas subtract days from date
Python
0 vote140 views1 answer
Python send sigint to subprocess
Python
0 vote149 views1 answer
Pandas if value present in df index
Python
1 vote133 views1 answer
Make lists for each 2 items in a list
Python
4 votes315 views1 answer
Delete a row in pandas dataframe
Python