/Python/How to add special token to bert tokenizer

How to add special token to bert tokenizer

special_tokens_dict = {'additional_special_tokens': ['[C1]','[C2]','[C3]','[C4]']}
num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
model.resize_token_embeddings(len(tokenizer))

Similar codes

0 vote172 views1 answer
Hashing in python using quadratic probing
Python
69 votes175 views6 answers
Python dictionary append
Python
0 vote215 views1 answer
Current page django
Python Django
3 votes200 views2 answers
Python extract specific columns from pandas dataframe
Python
22 votes210 views6 answers
Import turtle python
Python
6 votes249 views1 answer
Print url selenium python
Python
0 vote158 views1 answer
Fibonacci sequence in python 2.7
Python
10 votes257 views2 answers
Set axis labels python
Python
3 votes194 views2 answers
Check iterable python
Python
0 vote200 views1 answer
Python was not found;
Python