/Python/How to add special token to bert tokenizer

How to add special token to bert tokenizer

special_tokens_dict = {'additional_special_tokens': ['[C1]','[C2]','[C3]','[C4]']}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
model.resize_token_embeddings(len(tokenizer))

Similar codes

1 vote224 views1 answer
Generate valid sudoku board python
Python
2 votes211 views1 answer
Split string in the middle python
Python
-2 votes210 views3 answers
How to insert a banner in html
Python
0 vote194 views1 answer
Python no mathcing key method found
Python
0 vote209 views1 answer
Decimal characters python
Python
81 votes346 views10 answers
Python get current directory
Python
2 votes191 views1 answer
Private class method python
Python
4 votes277 views3 answers
** in python
Python
0 vote201 views1 answer
Mechanize python LE #3
Python
1 vote211 views6 answers
Skit learn decision
Python