Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

44 votes212 views6 answers
Python random string
Python
0 vote210 views1 answer
How to get the current line number in python
Python
0 vote196 views1 answer
List comprehension python one line
Python
0 vote252 views1 answer
If list is null python apply any function site:stackoverflow.com
Python
1 vote220 views1 answer
How to sort in greatest to least python
Python
7 votes189 views5 answers
How to multiply two lists in python
Python
1 vote247 views2 answers
Flask socketio send
Python Flask
92 votes203 views6 answers
Turn python script into exe
Python
11 votes204 views3 answers
Pandas random sample
Python
0 vote179 views1 answer
Enormous input test codechef solution
Python