Python – How to split a text file to its words in python

python

I am very new to python and also didn't work with text before…I have 100 text files, each has around 100 to 150 lines of unstructured text describing patient's condition. I read one file in python using:

with open("C:\\...\\...\\...\\record-13.txt") as f:
    content = f.readlines()
    print (content)

Now I can split each line of this file to its words using for example:

a = content[0].split()
print (a)

but I don't know how to split whole file to words?
do loops (while or for) help with that?

Thank you for your help guys. Your answers help me to write this (in my file, words are split by space so that's delimiter I think!):

with open ("C:\\...\\...\\...\\record-13.txt") as f:
  lines = f.readlines()
  for line in lines:
      words = line.split()
      for word in words:
          print (word)

that simply splits words by line (one word in one line).

Best Answer

It depends on how you define words, or what you regard as the delimiters.
Notice string.split in Python receives an optional parameter delimiter, so you could pass it as this:

for lines in content[0].split():
    for word in lines.split(','):
        print(word)

Unfortunately, string.split receives a single delimiter only, so you may need multi-level splitting like this:

for lines in content[0].split():
    for split0 in lines.split(' '):
        for split1 in split0.split(','):
            for split2 in split1.split('.'):
                for split3 in split2.split('?'):
                    for split4 in split3.split('!'):
                        for word in split4.split(':'): 
                            if word != "":
                                print(word)

Looks ugly, right? Luckily we can use iteration instead:

delimiters = ['\n', ' ', ',', '.', '?', '!', ':', 'and_what_else_you_need']
words = content
for delimiter in delimiters:
    new_words = []
    for word in words:
        new_words += word.split(delimiter)
    words = new_words

EDITED: Or simply we could use the regular expression package:

import re
delimiters = ['\n', ' ', ',', '.', '?', '!', ':', 'and_what_else_you_need']
words = re.split('|'.join(delimiters), content)
Related Topic