Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote84 views1 answer
Django cache
Python Django
0 vote122 views1 answer
Python spearman correlation
Python
0 vote117 views1 answer
Add option in python script
Python
1 vote117 views2 answers
How to make a random question generator in python
Python
4 votes122 views1 answer
Change default option optionmenu tkinter
Python
6 votes128 views1 answer
Python append a file and read
Python
0 vote156 views1 answer
Scipy solve equation numerically
Python
0 vote173 views1 answer
Inspect first 5 rows of dataframe
Python
10 votes131 views3 answers
Python tostring
Python
1 vote130 views1 answer
Vector in python
Python