Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

0 vote208 views1 answer
Stack error: command failed: import sys; print "%s.%s.%s" % sys.version_info[:3];
Python
0 vote152 views1 answer
Facet Grid for Bar Plot with non-shared y axes (CatPlot)
Python
0 vote197 views1 answer
Flask sqlalchemy case insensitive like
Python
1 vote251 views1 answer
Install postgres for python mac
Python
1 vote190 views2 answers
Python not null
Python
1 vote183 views1 answer
Using-len-for-text-but-discarding-spaces-in-the-count
Python
0 vote159 views1 answer
How to use spacy
Python
12 votes241 views4 answers
How to add array in python
Python
0 vote205 views1 answer
Python Hewwo wowwd
Python
7 votes1.2K views1 answer
What is r strip function in python
Python