Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote96 views1 answer
How to scale an array between two values python
Python
1 vote107 views1 answer
String to tuple python
Python
0 vote113 views1 answer
Request.build_absolute_uri django
Python
1 vote133 views1 answer
Python get line number of error
Python
0 vote121 views3 answers
Python count
Python
0 vote96 views1 answer
Download viper for python ubutunu
Python
1 vote107 views2 answers
Np.argsort reverse
Python
1 vote120 views2 answers
Add a value to an existing field in pandas dataframe after checking conditions
Python
1 vote115 views1 answer
The python libraries to master for machine learning
Python
0 vote110 views1 answer
Django array of dates
Python