Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

1 vote180 views1 answer
How to change kay bindings in pycharm
Python
13 votes213 views6 answers
Django orm
Python Django
0 vote145 views1 answer
Limiting a for each loop python
Python
1 vote145 views1 answer
Python making player equipment
Python
2 votes162 views2 answers
Install virtual environment python mac
Python
1 vote142 views1 answer
Python string format_map
Python
0 vote149 views1 answer
Google-api-python-client python 3
Python
1 vote149 views1 answer
Como unir dos listas python
Python
1 vote195 views1 answer
How to multiply in django template
Python Django
1 vote159 views1 answer
Python hide input
Python