Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

7 votes172 views1 answer
Numpy array with specific value
Python
0 vote170 views1 answer
Qaction hide show python
Python
0 vote186 views1 answer
Flipping an image with cv2
Python
2 votes167 views1 answer
How to move your cursor using python
Python
0 vote170 views1 answer
Python excel zelle schreiben
Python
3 votes199 views5 answers
Max function python
Python
0 vote169 views1 answer
Python loc id in list
Python
0 vote179 views1 answer
Neural network hyperparameter tuning
Python
0 vote160 views2 answers
A guide to numpy and pandas
Python
0 vote176 views3 answers
Python unittest mock
Python