Python nltk tokenize

>>> import nltk
>>> sentence = """At eight o'clock on Thursday morning
... Arthur didn't feel very good."""
>>> tokens = nltk.word_tokenize(sentence)
>>> tokens
['At', 'eight', "o'clock", 'on', 'Thursday', 'morning',
'Arthur', 'did', "n't", 'feel', 'very', 'good', '.']

Source: stackoverflow.com

Import word_tokenize

import nltk
from nltk import word_tokenize

Similar codes

0 vote154 views1 answer
Print elements without print function in python
Python
0 vote137 views1 answer
Lsit append in python
Python
4 votes136 views2 answers
Pip install colab
Python
92 votes142 views6 answers
String to datetime
Python
0 vote135 views1 answer
Concatenate directories python
Python
10 votes141 views4 answers
How to check if Python is 32 or 64 bit
Python
2 votes142 views2 answers
Exclude index column pandas
Python
1 vote99 views1 answer
Write a python program to split a given dictionary of lists into list of dictionaries.
Python
0 vote143 views3 answers
How to print a text in python
Python
0 vote136 views1 answer
How to convert each string to a category or int in python dataframe
Python