ChatGPT解决这个技术问题 Extra ChatGPT

How do I list all files of a directory?

This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.

How can I list all files of a directory in Python and add them to a list?


M
Mateen Ulhaq

os.listdir() returns everything inside a directory -- including both files and directories.

os.path's isfile() can be used to only list files:

from os import listdir
from os.path import isfile, join
onlyfiles = [f for f in listdir(mypath) if isfile(join(mypath, f))]

Alternatively, os.walk() yields two lists for each directory it visits -- one for files and one for dirs. If you only want the top directory you can break the first time it yields:

from os import walk

f = []
for (dirpath, dirnames, filenames) in walk(mypath):
    f.extend(filenames)
    break

or, shorter:

from os import walk

filenames = next(walk(mypath), (None, None, []))[2]  # [] if no file

A bit simpler: (_, _, filenames) = walk(mypath).next() (if you are confident that the walk will return at least one value, which it should.)
Slight modification to store full paths: for (dirpath, dirnames, filenames) in os.walk(mypath): checksum_files.extend(os.path.join(dirpath, filename) for filename in filenames) break
f.extend(filenames) is not actually equivalent to f = f + filenames. extend will modify f in-place, whereas adding creates a new list in a new memory location. This means extend is generally more efficient than +, but it can sometimes lead to confusion if multiple objects hold references to the list. Lastly, it's worth noting that f += filenames is equivalent to f.extend(filenames), not f = f + filenames.
@misterbee, your solution is the best, just one small improvement: _, _, filenames = next(walk(mypath), (None, None, []))
f += filenames is equivalent to extend and not the other way around??? Jeez.
F
Flimm

I prefer using the glob module, as it does pattern matching and expansion.

import glob
print(glob.glob("/home/adam/*"))

It does pattern matching intuitively

import glob
# All files and directories ending with .txt and that don't begin with a dot:
print(glob.glob("/home/adam/*.txt")) 
# All files and directories ending with .txt with depth of 2 folders, ignoring names beginning with a dot:
print(glob.glob("/home/adam/*/*.txt")) 

It will return a list with the queried files and directories:

['/home/adam/file1.txt', '/home/adam/file2.txt', .... ]

Note that glob ignores files and directories that begin with a dot ., as those are considered hidden files and directories, unless the pattern is something like .*.

Use glob.escape to escape strings that are not meant to be patterns:

print(glob.glob(glob.escape(directory_name) + "/*.txt"))

that's a shortcut for listdir+fnmatch docs.python.org/library/fnmatch.html#fnmatch.fnmatch
to clarify, this does not return the "full path"; it simply returns the expansion of the glob, whatever it may be. E.g., given /home/user/foo/bar/hello.txt, then, if running in directory foo, the glob("bar/*.txt") will return bar/hello.txt. There are cases when you do in fact want the full (i.e., absolute) path; for those cases, see stackoverflow.com/questions/51520/…
Related: find files recursively with glob: stackoverflow.com/a/2186565/4561887
doesn't answer this question. glob.glob("*") would.
Is there a way to ensure the items returned from glob are only files? I ask regarding the instance where files could exist without extensions (or other scenarios where files and folders are indistinguishable purely from their path strings). I note this is an appropriate clarification here as the OP did not specify whether or not their files have extensions on them.
x
xxx

list in the current directory

With listdir in os module you get the files and the folders in the current dir

import os

arr = os.listdir()

Looking in a directory

arr = os.listdir('c:\\files')

with glob you can specify a type of file to list like this

import glob

txtfiles = []
for file in glob.glob("*.txt"):
    txtfiles.append(file)

or

mylist = [f for f in glob.glob("*.txt")]

get the full path of only files in the current directory

import os
from os import listdir
from os.path import isfile, join

cwd = os.getcwd()
onlyfiles = [os.path.join(cwd, f) for f in os.listdir(cwd) if 
os.path.isfile(os.path.join(cwd, f))]
print(onlyfiles) 

['G:\\getfilesname\\getfilesname.py', 'G:\\getfilesname\\example.txt']

Getting the full path name with os.path.abspath

You get the full path in return

 import os
 files_path = [os.path.abspath(x) for x in os.listdir()]
 print(files_path)
 
 ['F:\\documenti\applications.txt', 'F:\\documenti\collections.txt']

Walk: going through sub directories

os.walk returns the root, the directories list and the files list, that is why I unpacked them in r, d, f in the for loop; it, then, looks for other files and directories in the subfolders of the root and so on until there are no subfolders.

import os

# Getting the current work directory (cwd)
thisdir = os.getcwd()

# r=root, d=directories, f = files
for r, d, f in os.walk(thisdir):
    for file in f:
        if file.endswith(".docx"):
            print(os.path.join(r, file))

To go up in the directory tree

# Method 1
x = os.listdir('..')

# Method 2
x= os.listdir('/')

Get files of a particular subdirectory with os.listdir()

import os

x = os.listdir("./content")

os.walk('.') - current directory

 import os
 arr = next(os.walk('.'))[2]
 print(arr)
 
 >>> ['5bs_Turismo1.pdf', '5bs_Turismo1.pptx', 'esperienza.txt']

next(os.walk('.')) and os.path.join('dir', 'file')

 import os
 arr = []
 for d,r,f in next(os.walk("F:\\_python")):
     for file in f:
         arr.append(os.path.join(r,file))

 for f in arr:
     print(files)

>>> F:\\_python\\dict_class.py
>>> F:\\_python\\programmi.txt

next... walk

 [os.path.join(r,file) for r,d,f in next(os.walk("F:\\_python")) for file in f]
 
 >>> ['F:\\_python\\dict_class.py', 'F:\\_python\\programmi.txt']

os.walk

x = [os.path.join(r,file) for r,d,f in os.walk("F:\\_python") for file in f]
print(x)

>>> ['F:\\_python\\dict.py', 'F:\\_python\\progr.txt', 'F:\\_python\\readl.py']

os.listdir() - get only txt files

 arr_txt = [x for x in os.listdir() if x.endswith(".txt")]
 

Using glob to get the full path of the files

from path import path
from glob import glob

x = [path(f).abspath() for f in glob("F:\\*.txt")]

Using os.path.isfile to avoid directories in the list

import os.path
listOfFiles = [f for f in os.listdir() if os.path.isfile(f)]

Using pathlib from Python 3.4

import pathlib

flist = []
for p in pathlib.Path('.').iterdir():
    if p.is_file():
        print(p)
        flist.append(p)

With list comprehension:

flist = [p for p in pathlib.Path('.').iterdir() if p.is_file()]

Use glob method in pathlib.Path()

import pathlib

py = pathlib.Path().glob("*.py")

Get all and only files with os.walk: checks only in the third element returned, i.e. the list of the files

import os
x = [i[2] for i in os.walk('.')]
y=[]
for t in x:
    for f in t:
        y.append(f)

Get only files with next in a directory: returns only the file in the root folder

 import os
 x = next(os.walk('F://python'))[2]

Get only directories with next and walk in a directory, because in the [1] element there are the folders only

 import os
 next(os.walk('F://python'))[1] # for the current dir use ('.')
 
 >>> ['python3','others']

Get all the subdir names with walk

for r,d,f in os.walk("F:\\_python"):
    for dirs in d:
        print(dirs)

os.scandir() from Python 3.5 and greater

import os
x = [f.name for f in os.scandir() if f.is_file()]

# Another example with `scandir` (a little variation from docs.python.org)
# This one is more efficient than `os.listdir`.
# In this case, it shows the files only in the current directory
# where the script is executed.

import os
with os.scandir() as i:
    for entry in i:
        if entry.is_file():
            print(entry.name)

This is a mish-mash of too many answers to questions not asked here. It may also be worth explaining what the caveats or recommended approaches are. I'm no better off knowing one way versus 20 ways to do the same thing unless I also know which is more appropriate to use when.
Ok, ASAP I will take a look at my answer and try to make it more clean and with more useful informations about the difference among the methods etc.
You should not determine file's extension by checking if the filename contains a substring. That might cause many troubles. I recommend to always check if the filename ends with the particular substring.
Such compilations can be helpful, but this answer in particular adds no value to the existing answers. Just to give an example, [f for f in glob.glob("*.txt")] is equivalent to glob.glob("*.txt") and warrants no extra section in this write up. It is also very wordy and with lots of spacing. An improvement could be made by adding explanations or pointing out differences instead of listing yet another variant.
Thanks for the comment, you are right of course and I will follow your advises asap to make it more useful, maybe in these years I could make some better ansewers. Just give me some days to re-elaborate it.
c
csano
import os
os.listdir("somedirectory")

will return a list of all files and directories in "somedirectory".


This returns the relative path of the files, as compared with the full path returned by glob.glob
@JIXiang: os.listdir() always returns mere filenames (not relative paths). What glob.glob() returns is driven by the path format of the input pattern.
o
ohe

A one-line solution to get only list of files (no subdirectories):

filenames = next(os.walk(path))[2]

or absolute pathnames:

paths = [os.path.join(path, fn) for fn in next(os.walk(path))[2]]

Only a one-liner if you've already import os. Seems less concise than glob() to me.
problem with glob is that a folder called 'something.something' would be returned by glob('/home/adam/*.*')
On OS X, there's something called a bundle. It's a directory which should generally be treated as a file (like a .tar). Would you want those treated as a file or a directory? Using glob() would treat it as a file. Your method would treat it as a directory.
J
Johnny

Getting Full File Paths From a Directory and All Its Subdirectories

import os

def get_filepaths(directory):
    """
    This function will generate the file names in a directory 
    tree by walking the tree either top-down or bottom-up. For each 
    directory in the tree rooted at directory top (including top itself), 
    it yields a 3-tuple (dirpath, dirnames, filenames).
    """
    file_paths = []  # List which will store all of the full filepaths.

    # Walk the tree.
    for root, directories, files in os.walk(directory):
        for filename in files:
            # Join the two strings in order to form the full filepath.
            filepath = os.path.join(root, filename)
            file_paths.append(filepath)  # Add it to the list.

    return file_paths  # Self-explanatory.

# Run the above function and store its results in a variable.   
full_file_paths = get_filepaths("/Users/johnny/Desktop/TEST")

The path I provided in the above function contained 3 files— two of them in the root directory, and another in a subfolder called "SUBFOLDER." You can now do things like:

print full_file_paths which will print the list: ['/Users/johnny/Desktop/TEST/file1.txt', '/Users/johnny/Desktop/TEST/file2.txt', '/Users/johnny/Desktop/TEST/SUBFOLDER/file3.dat']

['/Users/johnny/Desktop/TEST/file1.txt', '/Users/johnny/Desktop/TEST/file2.txt', '/Users/johnny/Desktop/TEST/SUBFOLDER/file3.dat']

If you'd like, you can open and read the contents, or focus only on files with the extension ".dat" like in the code below:

for f in full_file_paths:
  if f.endswith(".dat"):
    print f

/Users/johnny/Desktop/TEST/SUBFOLDER/file3.dat


P
Peter Mortensen

Since version 3.4 there are builtin iterators for this which are a lot more efficient than os.listdir():

pathlib: New in version 3.4.

>>> import pathlib
>>> [p for p in pathlib.Path('.').iterdir() if p.is_file()]

According to PEP 428, the aim of the pathlib library is to provide a simple hierarchy of classes to handle filesystem paths and the common operations users do over them.

os.scandir(): New in version 3.5.

>>> import os
>>> [entry for entry in os.scandir('.') if entry.is_file()]

Note that os.walk() uses os.scandir() instead of os.listdir() from version 3.5, and its speed got increased by 2-20 times according to PEP 471.

Let me also recommend reading ShadowRanger's comment below.


Thanks! I think it is the only solution not returning directly a list. Could use p.name instead of the first p alternatively if preferred.
Welcome! I would prefer generating pathlib.Path() instances since they have many useful methods I would not want to waste waste. You can also call str(p) on them for path names.
Note: The os.scandir solution is going to be more efficient than os.listdir with an os.path.is_file check or the like, even if you need a list (so you don't benefit from lazy iteration), because os.scandir uses OS provided APIs that give you the is_file information for free as it iterates, no per-file round trip to the disk to stat them at all (on Windows, the DirEntrys get you complete stat info for free, on *NIX systems it needs to stat for info beyond is_file, is_dir, etc., but DirEntry caches on first stat for convenience).
You can also use entry.name to get only the file name, or entry.path to get its full path. No more os.path.join() all over the place.
C
CristiFati

Preliminary notes

Although there's a clear differentiation between file and directory terms in the question text, some may argue that directories are actually special files

The statement: "all files of a directory" can be interpreted in two ways: All direct (or level 1) descendants only All descendants in the whole directory tree (including the ones in sub-directories)

All direct (or level 1) descendants only

All descendants in the whole directory tree (including the ones in sub-directories)

When the question was asked, I imagine that Python 2, was the LTS version, however the code samples will be run by Python 3(.5) (I'll keep them as Python 2 compliant as possible; also, any code belonging to Python that I'm going to post, is from v3.5.4 - unless otherwise specified). That has consequences related to another keyword in the question: "add them into a list": In pre Python 2.2 versions, sequences (iterables) were mostly represented by lists (tuples, sets, ...) In Python 2.2, the concept of generator ([Python.Wiki]: Generators) - courtesy of [Python 3]: The yield statement) - was introduced. As time passed, generator counterparts started to appear for functions that returned/worked with lists In Python 3, generator is the default behavior Not sure if returning a list is still mandatory (or a generator would do as well), but passing a generator to the list constructor, will create a list out of it (and also consume it). The example below illustrates the differences on [Python 3]: map(function, iterable, ...) >>> import sys >>> sys.version '2.7.10 (default, Mar 8 2016, 15:02:46) [MSC v.1600 64 bit (AMD64)]' >>> m = map(lambda x: x, [1, 2, 3]) # Just a dummy lambda function >>> m, type(m) ([1, 2, 3], ) >>> len(m) 3 >>> import sys >>> sys.version '3.5.4 (v3.5.4:3f56838, Aug 8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)]' >>> m = map(lambda x: x, [1, 2, 3]) >>> m, type(m) (, ) >>> len(m) Traceback (most recent call last): File "", line 1, in TypeError: object of type 'map' has no len() >>> lm0 = list(m) # Build a list from the generator >>> lm0, type(lm0) ([1, 2, 3], ) >>> >>> lm1 = list(m) # Build a list from the same generator >>> lm1, type(lm1) # Empty list now - generator already consumed ([], )

In pre Python 2.2 versions, sequences (iterables) were mostly represented by lists (tuples, sets, ...)

In Python 2.2, the concept of generator ([Python.Wiki]: Generators) - courtesy of [Python 3]: The yield statement) - was introduced. As time passed, generator counterparts started to appear for functions that returned/worked with lists

In Python 3, generator is the default behavior

Not sure if returning a list is still mandatory (or a generator would do as well), but passing a generator to the list constructor, will create a list out of it (and also consume it). The example below illustrates the differences on [Python 3]: map(function, iterable, ...)

The examples will be based on a directory called root_dir with the following structure (this example is for Win, but I'm using the same tree on Lnx as well): E:\Work\Dev\StackOverflow\q003207219>tree /f "root_dir" Folder PATH listing for volume Work Volume serial number is 00000029 3655:6FED E:\WORK\DEV\STACKOVERFLOW\Q003207219\ROOT_DIR ¦ file0 ¦ file1 ¦ +---dir0 ¦ +---dir00 ¦ ¦ ¦ file000 ¦ ¦ ¦ ¦ ¦ +---dir000 ¦ ¦ file0000 ¦ ¦ ¦ +---dir01 ¦ ¦ file010 ¦ ¦ file011 ¦ ¦ ¦ +---dir02 ¦ +---dir020 ¦ +---dir0200 +---dir1 ¦ file10 ¦ file11 ¦ file12 ¦ +---dir2 ¦ ¦ file20 ¦ ¦ ¦ +---dir20 ¦ file200 ¦ +---dir3

Solutions

Programmatic approaches:

[Python 3]: os.listdir(path='.') Return a list containing the names of the entries in the directory given by path. The list is in arbitrary order, and does not include the special entries '.' and '..' ... >>> import os >>> root_dir = "root_dir" # Path relative to current dir (os.getcwd()) >>> >>> os.listdir(root_dir) # List all the items in root_dir ['dir0', 'dir1', 'dir2', 'dir3', 'file0', 'file1'] >>> >>> [item for item in os.listdir(root_dir) if os.path.isfile(os.path.join(root_dir, item))] # Filter items and only keep files (strip out directories) ['file0', 'file1'] A more elaborate example (code_os_listdir.py): import os from pprint import pformat def _get_dir_content(path, include_folders, recursive): entries = os.listdir(path) for entry in entries: entry_with_path = os.path.join(path, entry) if os.path.isdir(entry_with_path): if include_folders: yield entry_with_path if recursive: for sub_entry in _get_dir_content(entry_with_path, include_folders, recursive): yield sub_entry else: yield entry_with_path def get_dir_content(path, include_folders=True, recursive=True, prepend_folder_name=True): path_len = len(path) + len(os.path.sep) for item in _get_dir_content(path, include_folders, recursive): yield item if prepend_folder_name else item[path_len:] def _get_dir_content_old(path, include_folders, recursive): entries = os.listdir(path) ret = list() for entry in entries: entry_with_path = os.path.join(path, entry) if os.path.isdir(entry_with_path): if include_folders: ret.append(entry_with_path) if recursive: ret.extend(_get_dir_content_old(entry_with_path, include_folders, recursive)) else: ret.append(entry_with_path) return ret def get_dir_content_old(path, include_folders=True, recursive=True, prepend_folder_name=True): path_len = len(path) + len(os.path.sep) return [item if prepend_folder_name else item[path_len:] for item in _get_dir_content_old(path, include_folders, recursive)] def main(): root_dir = "root_dir" ret0 = get_dir_content(root_dir, include_folders=True, recursive=True, prepend_folder_name=True) lret0 = list(ret0) print(ret0, len(lret0), pformat(lret0)) ret1 = get_dir_content_old(root_dir, include_folders=False, recursive=True, prepend_folder_name=False) print(len(ret1), pformat(ret1)) if __name__ == "__main__": main() Notes: There are two implementations: One that uses generators (of course here it seems useless, since I immediately convert the result to a list) The classic one (function names ending in _old) Recursion is used (to get into subdirectories) For each implementation there are two functions: One that starts with an underscore (_): "private" (should not be called directly) - that does all the work The public one (wrapper over previous): it just strips off the initial path (if required) from the returned entries. It's an ugly implementation, but it's the only idea that I could come with at this point In terms of performance, generators are generally a little bit faster (considering both creation and iteration times), but I didn't test them in recursive functions, and also I am iterating inside the function over inner generators - don't know how performance friendly is that Play with the arguments to get different results Output: (py35x64_test) E:\Work\Dev\StackOverflow\q003207219>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" "code_os_listdir.py" 22 ['root_dir\\dir0', 'root_dir\\dir0\\dir00', 'root_dir\\dir0\\dir00\\dir000', 'root_dir\\dir0\\dir00\\dir000\\file0000', 'root_dir\\dir0\\dir00\\file000', 'root_dir\\dir0\\dir01', 'root_dir\\dir0\\dir01\\file010', 'root_dir\\dir0\\dir01\\file011', 'root_dir\\dir0\\dir02', 'root_dir\\dir0\\dir02\\dir020', 'root_dir\\dir0\\dir02\\dir020\\dir0200', 'root_dir\\dir1', 'root_dir\\dir1\\file10', 'root_dir\\dir1\\file11', 'root_dir\\dir1\\file12', 'root_dir\\dir2', 'root_dir\\dir2\\dir20', 'root_dir\\dir2\\dir20\\file200', 'root_dir\\dir2\\file20', 'root_dir\\dir3', 'root_dir\\file0', 'root_dir\\file1'] 11 ['dir0\\dir00\\dir000\\file0000', 'dir0\\dir00\\file000', 'dir0\\dir01\\file010', 'dir0\\dir01\\file011', 'dir1\\file10', 'dir1\\file11', 'dir1\\file12', 'dir2\\dir20\\file200', 'dir2\\file20', 'file0', 'file1']

[Python 3]: os.scandir(path='.') (Python 3.5+, backport: [PyPI]: scandir) Return an iterator of os.DirEntry objects corresponding to the entries in the directory given by path. The entries are yielded in arbitrary order, and the special entries '.' and '..' are not included. Using scandir() instead of listdir() can significantly increase the performance of code that also needs file type or file attribute information, because os.DirEntry objects expose this information if the operating system provides it when scanning a directory. All os.DirEntry methods may perform a system call, but is_dir() and is_file() usually only require a system call for symbolic links; os.DirEntry.stat() always requires a system call on Unix but only requires one for symbolic links on Windows. >>> import os >>> root_dir = os.path.join(".", "root_dir") # Explicitly prepending current directory >>> root_dir '.\\root_dir' >>> >>> scandir_iterator = os.scandir(root_dir) >>> scandir_iterator >>> [item.path for item in scandir_iterator] ['.\\root_dir\\dir0', '.\\root_dir\\dir1', '.\\root_dir\\dir2', '.\\root_dir\\dir3', '.\\root_dir\\file0', '.\\root_dir\\file1'] >>> >>> [item.path for item in scandir_iterator] # Will yield an empty list as it was consumed by previous iteration (automatically performed by the list comprehension) [] >>> >>> scandir_iterator = os.scandir(root_dir) # Reinitialize the generator >>> for item in scandir_iterator : ... if os.path.isfile(item.path): ... print(item.name) ... file0 file1 Notes: It's similar to os.listdir But it's also more flexible (and offers more functionality), more Pythonic (and in some cases, faster)

[Python 3]: os.walk(top, topdown=True, onerror=None, followlinks=False) Generate the file names in a directory tree by walking the tree either top-down or bottom-up. For each directory in the tree rooted at directory top (including top itself), it yields a 3-tuple (dirpath, dirnames, filenames). >>> import os >>> root_dir = os.path.join(os.getcwd(), "root_dir") # Specify the full path >>> root_dir 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir' >>> >>> walk_generator = os.walk(root_dir) >>> root_dir_entry = next(walk_generator) # First entry corresponds to the root dir (passed as an argument) >>> root_dir_entry ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir', ['dir0', 'dir1', 'dir2', 'dir3'], ['file0', 'file1']) >>> >>> root_dir_entry[1] + root_dir_entry[2] # Display dirs and files (direct descendants) in a single list ['dir0', 'dir1', 'dir2', 'dir3', 'file0', 'file1'] >>> >>> [os.path.join(root_dir_entry[0], item) for item in root_dir_entry[1] + root_dir_entry[2]] # Display all the entries in the previous list by their full path ['E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0', 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir1', 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir2', 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir3', 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\file0', 'E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\file1'] >>> >>> for entry in walk_generator: # Display the rest of the elements (corresponding to every subdir) ... print(entry) ... ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0', ['dir00', 'dir01', 'dir02'], []) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir00', ['dir000'], ['file000']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir00\\dir000', [], ['file0000']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir01', [], ['file010', 'file011']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir02', ['dir020'], []) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir02\\dir020', ['dir0200'], []) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir0\\dir02\\dir020\\dir0200', [], []) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir1', [], ['file10', 'file11', 'file12']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir2', ['dir20'], ['file20']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir2\\dir20', [], ['file200']) ('E:\\Work\\Dev\\StackOverflow\\q003207219\\root_dir\\dir3', [], []) Notes: Under the scenes, it uses os.scandir (os.listdir on older versions) It does the heavy lifting by recurring in subfolders

[Python 3]: glob.glob(pathname, *, recursive=False) ([Python 3]: glob.iglob(pathname, *, recursive=False)) Return a possibly-empty list of path names that match pathname, which must be a string containing a path specification. pathname can be either absolute (like /usr/src/Python-1.5/Makefile) or relative (like ../../Tools/*/*.gif), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). ... Changed in version 3.5: Support for recursive globs using “**”. >>> import glob, os >>> wildcard_pattern = "*" >>> root_dir = os.path.join("root_dir", wildcard_pattern) # Match every file/dir name >>> root_dir 'root_dir\\*' >>> >>> glob_list = glob.glob(root_dir) >>> glob_list ['root_dir\\dir0', 'root_dir\\dir1', 'root_dir\\dir2', 'root_dir\\dir3', 'root_dir\\file0', 'root_dir\\file1'] >>> >>> [item.replace("root_dir" + os.path.sep, "") for item in glob_list] # Strip the dir name and the path separator from begining ['dir0', 'dir1', 'dir2', 'dir3', 'file0', 'file1'] >>> >>> for entry in glob.iglob(root_dir + "*", recursive=True): ... print(entry) ... root_dir\ root_dir\dir0 root_dir\dir0\dir00 root_dir\dir0\dir00\dir000 root_dir\dir0\dir00\dir000\file0000 root_dir\dir0\dir00\file000 root_dir\dir0\dir01 root_dir\dir0\dir01\file010 root_dir\dir0\dir01\file011 root_dir\dir0\dir02 root_dir\dir0\dir02\dir020 root_dir\dir0\dir02\dir020\dir0200 root_dir\dir1 root_dir\dir1\file10 root_dir\dir1\file11 root_dir\dir1\file12 root_dir\dir2 root_dir\dir2\dir20 root_dir\dir2\dir20\file200 root_dir\dir2\file20 root_dir\dir3 root_dir\file0 root_dir\file1 Notes: Uses os.listdir For large trees (especially if recursive is on), iglob is preferred Allows advanced filtering based on name (due to the wildcard)

[Python 3]: class pathlib.Path(*pathsegments) (Python 3.4+, backport: [PyPI]: pathlib2) >>> import pathlib >>> root_dir = "root_dir" >>> root_dir_instance = pathlib.Path(root_dir) >>> root_dir_instance WindowsPath('root_dir') >>> root_dir_instance.name 'root_dir' >>> root_dir_instance.is_dir() True >>> >>> [item.name for item in root_dir_instance.glob("*")] # Wildcard searching for all direct descendants ['dir0', 'dir1', 'dir2', 'dir3', 'file0', 'file1'] >>> >>> [os.path.join(item.parent.name, item.name) for item in root_dir_instance.glob("*") if not item.is_dir()] # Display paths (including parent) for files only ['root_dir\\file0', 'root_dir\\file1'] Notes: This is one way of achieving our goal It's the OOP style of handling paths Offers lots of functionalities

[Python 2]: dircache.listdir(path) (Python 2 only) But, according to [GitHub]: python/cpython - (2.7) cpython/Lib/dircache.py, it's just a (thin) wrapper over os.listdir with caching def listdir(path): """List directory contents, using cache.""" try: cached_mtime, list = cache[path] del cache[path] except KeyError: cached_mtime, list = -1, [] mtime = os.stat(path).st_mtime if mtime != cached_mtime: list = os.listdir(path) list.sort() cache[path] = mtime, list return list

[man7]: OPENDIR(3) / [man7]: READDIR(3) / [man7]: CLOSEDIR(3) via [Python 3]: ctypes - A foreign function library for Python (POSIX specific) ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python. code_ctypes.py: #!/usr/bin/env python3 import sys from ctypes import Structure, \ c_ulonglong, c_longlong, c_ushort, c_ubyte, c_char, c_int, \ CDLL, POINTER, \ create_string_buffer, get_errno, set_errno, cast DT_DIR = 4 DT_REG = 8 char256 = c_char * 256 class LinuxDirent64(Structure): _fields_ = [ ("d_ino", c_ulonglong), ("d_off", c_longlong), ("d_reclen", c_ushort), ("d_type", c_ubyte), ("d_name", char256), ] LinuxDirent64Ptr = POINTER(LinuxDirent64) libc_dll = this_process = CDLL(None, use_errno=True) # ALWAYS set argtypes and restype for functions, otherwise it's UB!!! opendir = libc_dll.opendir readdir = libc_dll.readdir closedir = libc_dll.closedir def get_dir_content(path): ret = [path, list(), list()] dir_stream = opendir(create_string_buffer(path.encode())) if (dir_stream == 0): print("opendir returned NULL (errno: {:d})".format(get_errno())) return ret set_errno(0) dirent_addr = readdir(dir_stream) while dirent_addr: dirent_ptr = cast(dirent_addr, LinuxDirent64Ptr) dirent = dirent_ptr.contents name = dirent.d_name.decode() if dirent.d_type & DT_DIR: if name not in (".", ".."): ret[1].append(name) elif dirent.d_type & DT_REG: ret[2].append(name) dirent_addr = readdir(dir_stream) if get_errno(): print("readdir returned NULL (errno: {:d})".format(get_errno())) closedir(dir_stream) return ret def main(): print("{:s} on {:s}\n".format(sys.version, sys.platform)) root_dir = "root_dir" entries = get_dir_content(root_dir) print(entries) if __name__ == "__main__": main() Notes: It loads the three functions from libc (loaded in the current process) and calls them (for more details check [SO]: How do I check whether a file exists without exceptions? (@CristiFati's answer) - last notes from item #4.). That would place this approach very close to the Python / C edge LinuxDirent64 is the ctypes representation of struct dirent64 from [man7]: dirent.h(0P) (so are the DT_ constants) from my machine: Ubtu 16 x64 (4.10.0-40-generic and libc6-dev:amd64). On other flavors/versions, the struct definition might differ, and if so, the ctypes alias should be updated, otherwise it will yield Undefined Behavior It returns data in the os.walk's format. I didn't bother to make it recursive, but starting from the existing code, that would be a fairly trivial task Everything is doable on Win as well, the data (libraries, functions, structs, constants, ...) differ Output: [cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q003207219]> ./code_ctypes.py 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609] on linux ['root_dir', ['dir2', 'dir1', 'dir3', 'dir0'], ['file1', 'file0']]

[ActiveState.Docs]: win32file.FindFilesW (Win specific) Retrieves a list of matching filenames, using the Windows Unicode API. An interface to the API FindFirstFileW/FindNextFileW/Find close functions. >>> import os, win32file, win32con >>> root_dir = "root_dir" >>> wildcard = "*" >>> root_dir_wildcard = os.path.join(root_dir, wildcard) >>> entry_list = win32file.FindFilesW(root_dir_wildcard) >>> len(entry_list) # Don't display the whole content as it's too long 8 >>> [entry[-2] for entry in entry_list] # Only display the entry names ['.', '..', 'dir0', 'dir1', 'dir2', 'dir3', 'file0', 'file1'] >>> >>> [entry[-2] for entry in entry_list if entry[0] & win32con.FILE_ATTRIBUTE_DIRECTORY and entry[-2] not in (".", "..")] # Filter entries and only display dir names (except self and parent) ['dir0', 'dir1', 'dir2', 'dir3'] >>> >>> [os.path.join(root_dir, entry[-2]) for entry in entry_list if entry[0] & (win32con.FILE_ATTRIBUTE_NORMAL | win32con.FILE_ATTRIBUTE_ARCHIVE)] # Only display file "full" names ['root_dir\\file0', 'root_dir\\file1'] Notes: win32file.FindFilesW is part of [GitHub]: mhammond/pywin32 - Python for Windows (pywin32) Extensions, which is a Python wrapper over WINAPIs The documentation link is from ActiveState, as I didn't find any PyWin32 official documentation

Install some (other) third-party package that does the trick Most likely, will rely on one (or more) of the above (maybe with slight customizations)

Notes:

Code is meant to be portable (except places that target a specific area - which are marked) or cross: platform (Nix, Win, ) Python version (2, 3, )

platform (Nix, Win, )

Python version (2, 3, )

Multiple path styles (absolute, relatives) were used across the above variants, to illustrate the fact that the "tools" used are flexible in this direction

os.listdir and os.scandir use opendir / readdir / closedir ([MS.Docs]: FindFirstFileW function / [MS.Docs]: FindNextFileW function / [MS.Docs]: FindClose function) (via [GitHub]: python/cpython - (master) cpython/Modules/posixmodule.c)

win32file.FindFilesW uses those (Win specific) functions as well (via [GitHub]: mhammond/pywin32 - (master) pywin32/win32/src/win32file.i)

_get_dir_content (from point #1.) can be implemented using any of these approaches (some will require more work and some less) Some advanced filtering (instead of just file vs. dir) could be done: e.g. the include_folders argument could be replaced by another one (e.g. filter_func) which would be a function that takes a path as an argument: filter_func=lambda x: True (this doesn't strip out anything) and inside _get_dir_content something like: if not filter_func(entry_with_path): continue (if the function fails for one entry, it will be skipped), but the more complex the code becomes, the longer it will take to execute

Some advanced filtering (instead of just file vs. dir) could be done: e.g. the include_folders argument could be replaced by another one (e.g. filter_func) which would be a function that takes a path as an argument: filter_func=lambda x: True (this doesn't strip out anything) and inside _get_dir_content something like: if not filter_func(entry_with_path): continue (if the function fails for one entry, it will be skipped), but the more complex the code becomes, the longer it will take to execute

Nota bene! Since recursion is used, I must mention that I did some tests on my laptop (Win 10 x64), totally unrelated to this problem, and when the recursion level was reaching values somewhere in the (990 .. 1000) range (recursionlimit - 1000 (default)), I got StackOverflow :). If the directory tree exceeds that limit (I am not an FS expert, so I don't know if that is even possible), that could be a problem. I must also mention that I didn't try to increase recursionlimit because I have no experience in the area (how much can I increase it before having to also increase the stack at OS level), but in theory there will always be the possibility for failure, if the dir depth is larger than the highest possible recursionlimit (on that machine)

The code samples are for demonstrative purposes only. That means that I didn't take into account error handling (I don't think there's any try / except / else / finally block), so the code is not robust (the reason is: to keep it as simple and short as possible). For production, error handling should be added as well

Other approaches:

Use Python only as a wrapper Everything is done using another technology That technology is invoked from Python The most famous flavor that I know is what I call the system administrator approach: Use Python (or any programming language for that matter) in order to execute shell commands (and parse their outputs) Some consider this a neat hack I consider it more like a lame workaround (gainarie), as the action per se is performed from shell (cmd in this case), and thus doesn't have anything to do with Python. Filtering (grep / findstr) or output formatting could be done on both sides, but I'm not going to insist on it. Also, I deliberately used os.system instead of subprocess.Popen. (py35x64_test) E:\Work\Dev\StackOverflow\q003207219>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" -c "import os;os.system(\"dir /b root_dir\")" dir0 dir1 dir2 dir3 file0 file1 In general this approach is to be avoided, since if some command output format slightly differs between OS versions/flavors, the parsing code should be adapted as well; not to mention differences between locales).


C
Community

I really liked adamk's answer, suggesting that you use glob(), from the module of the same name. This allows you to have pattern matching with *s.

But as other people pointed out in the comments, glob() can get tripped up over inconsistent slash directions. To help with that, I suggest you use the join() and expanduser() functions in the os.path module, and perhaps the getcwd() function in the os module, as well.

As examples:

from glob import glob

# Return everything under C:\Users\admin that contains a folder called wlp.
glob('C:\Users\admin\*\wlp')

The above is terrible - the path has been hardcoded and will only ever work on Windows between the drive name and the \s being hardcoded into the path.

from glob    import glob
from os.path import join

# Return everything under Users, admin, that contains a folder called wlp.
glob(join('Users', 'admin', '*', 'wlp'))

The above works better, but it relies on the folder name Users which is often found on Windows and not so often found on other OSs. It also relies on the user having a specific name, admin.

from glob    import glob
from os.path import expanduser, join

# Return everything under the user directory that contains a folder called wlp.
glob(join(expanduser('~'), '*', 'wlp'))

This works perfectly across all platforms.

Another great example that works perfectly across platforms and does something a bit different:

from glob    import glob
from os      import getcwd
from os.path import join

# Return everything under the current directory that contains a folder called wlp.
glob(join(getcwd(), '*', 'wlp'))

Hope these examples help you see the power of a few of the functions you can find in the standard Python library modules.


Extra glob fun: starting in Python 3.5, ** works as long as you set recursive = True. See the docs here: docs.python.org/3.5/library/glob.html#glob.glob
A
Apogentus
def list_files(path):
    # returns a list of names (with extension, without full path) of all files 
    # in folder path
    files = []
    for name in os.listdir(path):
        if os.path.isfile(os.path.join(path, name)):
            files.append(name)
    return files 

P
Peter Mortensen

If you are looking for a Python implementation of find, this is a recipe I use rather frequently:

from findtools.find_files import (find_files, Match)

# Recursively find all *.sh files in **/usr/bin**
sh_files_pattern = Match(filetype='f', name='*.sh')
found_files = find_files(path='/usr/bin', match=sh_files_pattern)

for found_file in found_files:
    print found_file

So I made a PyPI package out of it and there is also a GitHub repository. I hope that someone finds it potentially useful for this code.


G
Georgy

For greater results, you can use listdir() method of the os module along with a generator (a generator is a powerful iterator that keeps its state, remember?). The following code works fine with both versions: Python 2 and Python 3.

Here's a code:

import os

def files(path):  
    for file in os.listdir(path):
        if os.path.isfile(os.path.join(path, file)):
            yield file

for file in files("."):  
    print (file)

The listdir() method returns the list of entries for the given directory. The method os.path.isfile() returns True if the given entry is a file. And the yield operator quits the func but keeps its current state, and it returns only the name of the entry detected as a file. All the above allows us to loop over the generator function.


C
Cristian Ciupitu

Returning a list of absolute filepaths, does not recurse into subdirectories

L = [os.path.join(os.getcwd(),f) for f in os.listdir('.') if os.path.isfile(os.path.join(os.getcwd(),f))]

Note: os.path.abspath(f) would be a somewhat cheaper substitute for os.path.join(os.getcwd(),f).
I'd be more efficient still if you started with cwd = os.path.abspath('.'), then used cwd instead of '.' and os.getcwd() throughout to avoid loads of redundant system calls.
f
fralau

A wise teacher told me once that:

When there are several established ways to do something, none of them is good for all cases.

I will thus add a solution for a subset of the problem: quite often, we only want to check whether a file matches a start string and an end string, without going into subdirectories. We would thus like a function that returns a list of filenames, like:

filenames = dir_filter('foo/baz', radical='radical', extension='.txt')

If you care to first declare two functions, this can be done:

def file_filter(filename, radical='', extension=''):
    "Check if a filename matches a radical and extension"
    if not filename:
        return False
    filename = filename.strip()
    return(filename.startswith(radical) and filename.endswith(extension))

def dir_filter(dirname='', radical='', extension=''):
    "Filter filenames in directory according to radical and extension"
    if not dirname:
        dirname = '.'
    return [filename for filename in os.listdir(dirname)
                if file_filter(filename, radical, extension)]

This solution could be easily generalized with regular expressions (and you might want to add a pattern argument, if you do not want your patterns to always stick to the start or end of the filename).


A
Andrew Rohne
import os
import os.path


def get_files(target_dir):
    item_list = os.listdir(target_dir)

    file_list = list()
    for item in item_list:
        item_dir = os.path.join(target_dir,item)
        if os.path.isdir(item_dir):
            file_list += get_files(item_dir)
        else:
            file_list.append(item_dir)
    return file_list

Here I use a recursive structure.


The same can be achieved just in one line with pathlib: filter(Path.is_file, Path().rglob('*'))
s
shantanoo

Using generators

import os
def get_files(search_path):
     for (dirpath, _, filenames) in os.walk(search_path):
         for filename in filenames:
             yield os.path.join(dirpath, filename)
list_files = get_files('.')
for filename in list_files:
    print(filename)

THANKS! exactly what I needed. A lot of the answers at the top are outdated and not working with Python 3.9 :)
P
Peter Mortensen

Another very readable variant for Python 3.4+ is using pathlib.Path.glob:

from pathlib import Path
folder = '/foo'
[f for f in Path(folder).glob('*') if f.is_file()]

It is simple to make more specific, e.g. only look for Python source files which are not symbolic links, also in all subdirectories:

[f for f in Path(folder).glob('**/*.py') if not f.is_symlink()]

P
Peter Mortensen

For Python 2:

pip install rglob

Then do

import rglob
file_list = rglob.rglob("/home/base/dir/", "*")
print file_list

When an external dep can be avoided, do it. What is the added value of using an external dependency when all you need is already in the language?
M
MarredCheese

Here's my general-purpose function for this. It returns a list of file paths rather than filenames since I found that to be more useful. It has a few optional arguments that make it versatile. For instance, I often use it with arguments like pattern='*.txt' or subfolders=True.

import os
import fnmatch

def list_paths(folder='.', pattern='*', case_sensitive=False, subfolders=False):
    """Return a list of the file paths matching the pattern in the specified 
    folder, optionally including files inside subfolders.
    """
    match = fnmatch.fnmatchcase if case_sensitive else fnmatch.fnmatch
    walked = os.walk(folder) if subfolders else [next(os.walk(folder))]
    return [os.path.join(root, f)
            for root, dirnames, filenames in walked
            for f in filenames if match(f, pattern)]

V
Vinodh Krishnaraju

I will provide a sample one liner where sourcepath and file type can be provided as input. The code returns a list of filenames with csv extension. Use . in case all files needs to be returned. This will also recursively scans the subdirectories.

[y for x in os.walk(sourcePath) for y in glob(os.path.join(x[0], '*.csv'))]

Modify file extensions and source path as needed.


If you are going to use glob, then just use glob('**/*.csv', recursive=True). No need to combine this with os.walk() to recurse (recursive and ** are supported since Python 3.5).
G
Georgy

dircache is "Deprecated since version 2.6: The dircache module has been removed in Python 3.0."

import dircache
list = dircache.listdir(pathname)
i = 0
check = len(list[0])
temp = []
count = len(list)
while count != 0:
  if len(list[i]) != check:
     temp.append(list[i-1])
     check = len(list[i])
  else:
    i = i + 1
    count = count - 1

print temp