Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

6 votes135 views2 answers
How to use colorama
Python
32 votes137 views4 answers
Copy to clipboard python
Python
0 vote176 views2 answers
K7yKJk8vdjHvw56q7bCTxibvT
Python
0 vote154 views1 answer
Cv2 leave only the biggest blob
Python
1 vote150 views1 answer
Show jpg in jupyter notebook
Python
17 votes133 views2 answers
How to learn python
Python
1 vote147 views1 answer
Python convert input into lowercase
Python
25 votes162 views6 answers
Python remove spaces
Python
0 vote123 views1 answer
Ipython list command history
Python
0 vote222 views1 answer
Python os filename without extension
Python