/Python/How to add special token to bert tokenizer

How to add special token to bert tokenizer

special_tokens_dict = {'additional_special_tokens': ['[C1]','[C2]','[C3]','[C4]']}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
model.resize_token_embeddings(len(tokenizer))

Similar codes

0 vote323 views1 answer
Np array blurring
Python
-1 vote149 views1 answer
How to delete django superuser
Python
6 votes91 views1 answer
Read from yaml file python
Python
-1 vote133 views2 answers
How to find a square root of a number using python
Python
4 votes122 views1 answer
Django objects.create()
Python
18 votes130 views4 answers
Python get all combinations of list
Python
23 votes188 views6 answers
How to combine two lists in python
Python
3 votes122 views1 answer
Update windows wallpaper python
Python
6 votes127 views2 answers
Python check if folder is empty
Python
0 vote131 views2 answers
Not using first row as index pandas
Python