ChatGPT解决这个技术问题 Extra ChatGPT

List directory in Go

go

I've been trying to figure out how to simply list the files and folders in a single directory in Go.

I've found filepath.Walk, but it goes into sub-directories automatically, which I don't want. All of my other searches haven't turned anything better up.

I'm sure that this functionality exists, but it's been really hard to find. Let me know if anyone knows where I should look. Thanks.


β
β.εηοιτ.βε

You can try using the ReadDir function in the io/ioutil package. Per the docs:

ReadDir reads the directory named by dirname and returns a list of sorted directory entries.

The resulting slice contains os.FileInfo types, which provide the methods listed here. Here is a basic example that lists the name of everything in the current directory (folders are included but not specially marked - you can check if an item is a folder by using the IsDir() method):

package main

import (
    "fmt"
    "io/ioutil"
     "log"
)

func main() {
    files, err := ioutil.ReadDir("./")
    if err != nil {
        log.Fatal(err)
    }
 
    for _, f := range files {
            fmt.Println(f.Name())
    }
}

If you only want the names of the contents of a directory and speed is of essence, note that using Readdirnames is orders of magnitude faster (around 20x faster for me)
@SquattingSlavInTracksuit: I promoted your comment here to an answer, because I didn't have comment privileges at the time. If you'd rather answer it and get the credit, LMK.
@SquattingSlavInTracksuit - that's just one order of magnitude :P
Since go 1.16, io/ioutil is deprecated. Instead of ioutil.ReadDir() you should use os.ReadDir()
m
manigandand

We can get a list of files inside a folder on the file system using various golang standard library functions.

filepath.Walk ioutil.ReadDir os.File.Readdir


package main

import (
    "fmt"
    "io/ioutil"
    "log"
    "os"
    "path/filepath"
)

func main() {
    var (
        root  string
        files []string
        err   error
    )

    root := "/home/manigandan/golang/samples"
    // filepath.Walk
    files, err = FilePathWalkDir(root)
    if err != nil {
        panic(err)
    }
    // ioutil.ReadDir
    files, err = IOReadDir(root)
    if err != nil {
        panic(err)
    }
    //os.File.Readdir
    files, err = OSReadDir(root)
    if err != nil {
        panic(err)
    }

    for _, file := range files {
        fmt.Println(file)
    }
}

Using filepath.Walk

The path/filepath package provides a handy way to scan all the files in a directory, it will automatically scan each sub-directories in the directory.

func FilePathWalkDir(root string) ([]string, error) {
    var files []string
    err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
        if !info.IsDir() {
            files = append(files, path)
        }
        return nil
    })
    return files, err
}

Using ioutil.ReadDir

ioutil.ReadDir reads the directory named by dirname and returns a list of directory entries sorted by filename.

func IOReadDir(root string) ([]string, error) {
    var files []string
    fileInfo, err := ioutil.ReadDir(root)
    if err != nil {
        return files, err
    }

    for _, file := range fileInfo {
        files = append(files, file.Name())
    }
    return files, nil
}

Using os.File.Readdir

Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.

func OSReadDir(root string) ([]string, error) {
    var files []string
    f, err := os.Open(root)
    if err != nil {
        return files, err
    }
    fileInfo, err := f.Readdir(-1)
    f.Close()
    if err != nil {
        return files, err
    }

    for _, file := range fileInfo {
        files = append(files, file.Name())
    }
    return files, nil
}

Benchmark results.

https://i.stack.imgur.com/7SAu5.png

Get more details on this Blog Post


The most complete answer here. It's worth noting that there is no memory usage or allocs reported by this benchmark. It's possible the faster implementations use more memory. It's also possible that the number of CPU cores on the tester's computer hurts/helps the concurrent filepath.Walk. Furthermore, filepath.Walk supports recursive decent while os.File.Readdir and ioutil.ReadDir do not.
Could someone explain to me how to read / interpret the benchmark results?
@xuiqzy ns/op = nanoseconds per operation, so smaller is better. But as @xeoncross mentioned, Walk might be better in concurrent environments and might be using a lot less memory in environments with lots of files.
From Go source code for ioutil.ReadDir: "As of Go 1.16, os.ReadDir is a more efficient and correct choice: it returns a list of fs.DirEntry instead of fs.FileInfo,and it returns partial results in the case of an error midway through reading a directory." The complete ioutil package functions should be replaced by functions in io or os package per documentation.
this line ` root := "/home/manigandan/golang/samples"` occurs an error because root is already declared
c
cmaher

Even simpler, use path/filepath:

package main    

import (
    "fmt"
    "log"
    "path/filepath"
)

func main() {
    files, err := filepath.Glob("*")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(files) // contains a list of all files in the current directory
}

Note that Glob ignores file system errors such as I/O errors reading directories. The only possible returned error is ErrBadPattern, when pattern is malformed.
Be sure to understand what Glob does before using it. golang.org/pkg/path/filepath/#Glob
S
Sonia

ioutil.ReadDir is a good find, but if you click and look at the source you see that it calls the method Readdir of os.File. If you are okay with the directory order and don't need the list sorted, then this Readdir method is all you need.


V
VonC

Starting with Go 1.16, you can use the os.ReadDir function.

func ReadDir(name string) ([]DirEntry, error)

It reads a given directory and returns a DirEntry slice that contains the directory entries sorted by filename.

It's an optimistic function, so that, when an error occurs while reading the directory entries, it tries to return you a slice with the filenames up to the point before the error.

package main

import (
    "fmt"
    "log"
    "os"
)

func main() {
    files, err := os.ReadDir(".")
    if err != nil {
        log.Fatal(err)
    }

    for _, file := range files {
        fmt.Println(file.Name())
    }
}

Of interest: Go 1.17 (Q3 2021) includes fs.FileInfoToDirEntry():

func FileInfoToDirEntry(info FileInfo) DirEntry FileInfoToDirEntry returns a DirEntry that returns information from info. If info is nil, FileInfoToDirEntry returns nil.

Background

Go 1.16 (Q1 2021) will propose, with CL 243908 and CL 243914 , the ReadDir function, based on the FS interface:

// An FS provides access to a hierarchical file system.
//
// The FS interface is the minimum implementation required of the file system.
// A file system may implement additional interfaces,
// such as fsutil.ReadFileFS, to provide additional or optimized functionality.
// See io/fsutil for details.
type FS interface {
    // Open opens the named file.
    //
    // When Open returns an error, it should be of type *PathError
    // with the Op field set to "open", the Path field set to name,
    // and the Err field describing the problem.
    //
    // Open should reject attempts to open names that do not satisfy
    // ValidPath(name), returning a *PathError with Err set to
    // ErrInvalid or ErrNotExist.
    Open(name string) (File, error)
}

That allows for "os: add ReadDir method for lightweight directory reading":
See commit a4ede9f:

// ReadDir reads the contents of the directory associated with the file f
// and returns a slice of DirEntry values in directory order.
// Subsequent calls on the same file will yield later DirEntry records in the directory.
//
// If n > 0, ReadDir returns at most n DirEntry records.
// In this case, if ReadDir returns an empty slice, it will return an error explaining why.
// At the end of a directory, the error is io.EOF.
//
// If n <= 0, ReadDir returns all the DirEntry records remaining in the directory.
// When it succeeds, it returns a nil error (not io.EOF).
func (f *File) ReadDir(n int) ([]DirEntry, error) 

// A DirEntry is an entry read from a directory (using the ReadDir method).
type DirEntry interface {
    // Name returns the name of the file (or subdirectory) described by the entry.
    // This name is only the final element of the path, not the entire path.
    // For example, Name would return "hello.go" not "/home/gopher/hello.go".
    Name() string
    
    // IsDir reports whether the entry describes a subdirectory.
    IsDir() bool
    
    // Type returns the type bits for the entry.
    // The type bits are a subset of the usual FileMode bits, those returned by the FileMode.Type method.
    Type() os.FileMode
    
    // Info returns the FileInfo for the file or subdirectory described by the entry.
    // The returned FileInfo may be from the time of the original directory read
    // or from the time of the call to Info. If the file has been removed or renamed
    // since the directory read, Info may return an error satisfying errors.Is(err, ErrNotExist).
    // If the entry denotes a symbolic link, Info reports the information about the link itself,
    // not the link's target.
    Info() (FileInfo, error)
}

src/os/os_test.go#testReadDir() illustrates its usage:

    file, err := Open(dir)
    if err != nil {
        t.Fatalf("open %q failed: %v", dir, err)
    }
    defer file.Close()
    s, err2 := file.ReadDir(-1)
    if err2 != nil {
        t.Fatalf("ReadDir %q failed: %v", dir, err2)
    }

Ben Hoyt points out in the comments to Go 1.16 os.ReadDir:

os.ReadDir(path string) ([]os.DirEntry, error), which you'll be able to call directly without the Open dance. So you can probably shorten this to just os.ReadDir, as that's the concrete function most people will call.

See commit 3d913a9 (Dec. 2020):

os: add ReadFile, WriteFile, CreateTemp (was TempFile), MkdirTemp (was TempDir) from io/ioutil io/ioutil was a poorly defined collection of helpers. Proposal #40025 moved out the generic I/O helpers to io. This CL for proposal #42026 moves the OS-specific helpers to os, making the entire io/ioutil package deprecated.

os.ReadDir returns []DirEntry, in contrast to ioutil.ReadDir's []FileInfo. (Providing a helper that returns []DirEntry is one of the primary motivations for this change.)


Yes, though note that as part of this 1.16 work they're also adding os.ReadDir(path string) ([]os.DirEntry, error), which you'll be able to call directly without the Open dance. So you can probably shorten this to just os.ReadDir, as that's the concrete function most people will call.
@BenHoyt Good point, thank you. I have included your comment, with additional documentation links, in the answer for more visibility.
T
Timo Huovinen

From your description, what you probably want is os.Readdirnames.

func (f *File) Readdirnames(n int) (names []string, err error) Readdirnames reads the contents of the directory associated with file and returns a slice of up to n names of files in the directory, in directory order. Subsequent calls on the same file will yield further names. ... If n <= 0, Readdirnames returns all the names from the directory in a single slice.

Snippet:

file, err := os.Open(path)
if err != nil {
    return err
}
defer file.Close()
names, err := file.Readdirnames(0)
if err != nil {
    return err
}
fmt.Println(names)

Credit to SquattingSlavInTracksuit's comment; I'd have suggested promoting their comment to an answer if I could.


T
Timo Huovinen

A complete example of printing all the files in a directory recursively using Readdirnames

package main

import (
    "fmt"
    "os"
)

func main() {
    path := "/path/to/your/directory"
    err := readDir(path)
    if err != nil {
        panic(err)
    }
}

func readDir(path string) error {
    file, err := os.Open(path)
    if err != nil {
        return err
    }
    defer file.Close()
    names, _ := file.Readdirnames(0)
    for _, name := range names {
        filePath := fmt.Sprintf("%v/%v", path, name)
        file, err := os.Open(filePath)
        if err != nil {
            return err
        }
        defer file.Close()
        fileInfo, err := file.Stat()
        if err != nil {
            return err
        }
        fmt.Println(filePath)
        if fileInfo.IsDir() {
            readDir(filePath)
        }
    }
    return nil
}

关注公众号,不定期副业成功案例分享
Follow WeChat

Success story sharing

Want to stay one step ahead of the latest teleworks?

Subscribe Now