Compare commits
No commits in common. "master" and "v2.3.14" have entirely different histories.
192
README.md
192
README.md
@ -1,66 +1,60 @@
|
|||||||
# 2021 Update
|
node-walk
|
||||||
|
====
|
||||||
|
|
||||||
Consider using [`@root/walk`](https://npmjs.org/package/@root/walk) instead.
|
| Sponsored by [ppl](https://ppl.family)
|
||||||
|
|
||||||
I created `walk` quite literally a decade ago, in the Node v0.x days.
|
|
||||||
Back then using an EventEmitter seemed like the thing to do. Nowadays,
|
|
||||||
it seems a bit overkill for the simple task of walking over directories.
|
|
||||||
|
|
||||||
There's nothing wrong with `walk` - it's about the same as it was 10 years ago -
|
|
||||||
however, at only 50 lines of code long, `@root/walk` is much simpler and much faster.
|
|
||||||
|
|
||||||
# node-walk
|
|
||||||
|
|
||||||
| a [Root](https://rootprojects.org) project
|
|
||||||
|
|
||||||
nodejs walk implementation.
|
nodejs walk implementation.
|
||||||
|
|
||||||
This is somewhat of a port python's `os.walk`, but using Node.JS conventions.
|
This is somewhat of a port python's `os.walk`, but using Node.JS conventions.
|
||||||
|
|
||||||
- EventEmitter
|
* EventEmitter
|
||||||
- Asynchronous
|
* Asynchronous
|
||||||
- Chronological (optionally)
|
* Chronological (optionally)
|
||||||
- Built-in flow-control
|
* Built-in flow-control
|
||||||
- includes Synchronous version (same API as Asynchronous)
|
* includes Synchronous version (same API as Asynchronous)
|
||||||
|
|
||||||
As few file descriptors are opened at a time as possible.
|
As few file descriptors are opened at a time as possible.
|
||||||
This is particularly well suited for single hard disks which are not flash or solid state.
|
This is particularly well suited for single hard disks which are not flash or solid state.
|
||||||
|
|
||||||
## Installation
|
Installation
|
||||||
|
----
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm install --save walk
|
npm install --save walk
|
||||||
```
|
```
|
||||||
|
|
||||||
# Getting Started
|
Getting Started
|
||||||
|
====
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
'use strict';
|
(function () {
|
||||||
|
"use strict";
|
||||||
|
|
||||||
var walk = require('walk');
|
var walk = require('walk');
|
||||||
var fs = require('fs');
|
var fs = require('fs');
|
||||||
var walker;
|
var walker;
|
||||||
var options = {};
|
|
||||||
|
|
||||||
walker = walk.walk('/tmp', options);
|
walker = walk.walk("/tmp", options);
|
||||||
|
|
||||||
walker.on('file', function (root, fileStats, next) {
|
walker.on("file", function (root, fileStats, next) {
|
||||||
fs.readFile(fileStats.name, function () {
|
fs.readFile(fileStats.name, function () {
|
||||||
// doStuff
|
// doStuff
|
||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('errors', function (root, nodeStatsArray, next) {
|
walker.on("errors", function (root, nodeStatsArray, next) {
|
||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('end', function () {
|
walker.on("end", function () {
|
||||||
console.log('all done');
|
console.log("all done");
|
||||||
});
|
});
|
||||||
|
}());
|
||||||
```
|
```
|
||||||
|
|
||||||
## Common Events
|
Common Events
|
||||||
|
-----
|
||||||
|
|
||||||
All single event callbacks are in the form of `function (root, stat, next) {}`.
|
All single event callbacks are in the form of `function (root, stat, next) {}`.
|
||||||
|
|
||||||
@ -69,15 +63,16 @@ All multiple event callbacks callbacks are in the form of `function (root, stats
|
|||||||
All **error** event callbacks are in the form `function (root, stat/stats, next) {}`.
|
All **error** event callbacks are in the form `function (root, stat/stats, next) {}`.
|
||||||
**`stat.error`** contains the error.
|
**`stat.error`** contains the error.
|
||||||
|
|
||||||
- `names`
|
* `names`
|
||||||
- `directory`
|
* `directory`
|
||||||
- `directories`
|
* `directories`
|
||||||
- `file`
|
* `file`
|
||||||
- `files`
|
* `files`
|
||||||
- `end`
|
* `end`
|
||||||
- `nodeError` (`stat` failed)
|
* `nodeError` (`stat` failed)
|
||||||
- `directoryError` (`stat` succedded, but `readdir` failed)
|
* `directoryError` (`stat` succedded, but `readdir` failed)
|
||||||
- `errors` (a collection of any errors encountered)
|
* `errors` (a collection of any errors encountered)
|
||||||
|
|
||||||
|
|
||||||
A typical `stat` event looks like this:
|
A typical `stat` event looks like this:
|
||||||
|
|
||||||
@ -100,12 +95,14 @@ A typical `stat` event looks like this:
|
|||||||
type: 'file' }
|
type: 'file' }
|
||||||
```
|
```
|
||||||
|
|
||||||
# Advanced Example
|
Advanced Example
|
||||||
|
====
|
||||||
|
|
||||||
Both Asynchronous and Synchronous versions are provided.
|
Both Asynchronous and Synchronous versions are provided.
|
||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
'use strict';
|
(function () {
|
||||||
|
"use strict";
|
||||||
|
|
||||||
var walk = require('walk');
|
var walk = require('walk');
|
||||||
var fs = require('fs');
|
var fs = require('fs');
|
||||||
@ -113,17 +110,17 @@ var options;
|
|||||||
var walker;
|
var walker;
|
||||||
|
|
||||||
options = {
|
options = {
|
||||||
followLinks: false,
|
followLinks: false
|
||||||
// directories with these keys will be skipped
|
// directories with these keys will be skipped
|
||||||
filters: ['Temp', '_Temp'],
|
, filters: ["Temp", "_Temp"]
|
||||||
};
|
};
|
||||||
|
|
||||||
walker = walk.walk('/tmp', options);
|
walker = walk.walk("/tmp", options);
|
||||||
|
|
||||||
// OR
|
// OR
|
||||||
// walker = walk.walkSync("/tmp", options);
|
// walker = walk.walkSync("/tmp", options);
|
||||||
|
|
||||||
walker.on('names', function (root, nodeNamesArray) {
|
walker.on("names", function (root, nodeNamesArray) {
|
||||||
nodeNamesArray.sort(function (a, b) {
|
nodeNamesArray.sort(function (a, b) {
|
||||||
if (a > b) return 1;
|
if (a > b) return 1;
|
||||||
if (a < b) return -1;
|
if (a < b) return -1;
|
||||||
@ -131,7 +128,7 @@ walker.on('names', function (root, nodeNamesArray) {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('directories', function (root, dirStatsArray, next) {
|
walker.on("directories", function (root, dirStatsArray, next) {
|
||||||
// dirStatsArray is an array of `stat` objects with the additional attributes
|
// dirStatsArray is an array of `stat` objects with the additional attributes
|
||||||
// * type
|
// * type
|
||||||
// * error
|
// * error
|
||||||
@ -140,20 +137,21 @@ walker.on('directories', function (root, dirStatsArray, next) {
|
|||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('file', function (root, fileStats, next) {
|
walker.on("file", function (root, fileStats, next) {
|
||||||
fs.readFile(fileStats.name, function () {
|
fs.readFile(fileStats.name, function () {
|
||||||
// doStuff
|
// doStuff
|
||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('errors', function (root, nodeStatsArray, next) {
|
walker.on("errors", function (root, nodeStatsArray, next) {
|
||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
|
|
||||||
walker.on('end', function () {
|
walker.on("end", function () {
|
||||||
console.log('all done');
|
console.log("all done");
|
||||||
});
|
});
|
||||||
|
}());
|
||||||
```
|
```
|
||||||
|
|
||||||
### Sync
|
### Sync
|
||||||
@ -168,7 +166,7 @@ I don't think I can prevent the `process.nextTick()` that `EventEmitter` calls.
|
|||||||
|
|
||||||
```javascript
|
```javascript
|
||||||
(function () {
|
(function () {
|
||||||
'use strict';
|
"use strict";
|
||||||
|
|
||||||
var walk = require('walk');
|
var walk = require('walk');
|
||||||
var fs = require('fs');
|
var fs = require('fs');
|
||||||
@ -185,81 +183,82 @@ I don't think I can prevent the `process.nextTick()` that `EventEmitter` calls.
|
|||||||
if (a < b) return -1;
|
if (a < b) return -1;
|
||||||
return 0;
|
return 0;
|
||||||
});
|
});
|
||||||
},
|
}
|
||||||
directories: function (root, dirStatsArray, next) {
|
, directories: function (root, dirStatsArray, next) {
|
||||||
// dirStatsArray is an array of `stat` objects with the additional attributes
|
// dirStatsArray is an array of `stat` objects with the additional attributes
|
||||||
// * type
|
// * type
|
||||||
// * error
|
// * error
|
||||||
// * name
|
// * name
|
||||||
|
|
||||||
next();
|
next();
|
||||||
},
|
}
|
||||||
file: function (root, fileStats, next) {
|
, file: function (root, fileStats, next) {
|
||||||
fs.readFile(fileStats.name, function () {
|
fs.readFile(fileStats.name, function () {
|
||||||
// doStuff
|
// doStuff
|
||||||
next();
|
next();
|
||||||
});
|
});
|
||||||
},
|
}
|
||||||
errors: function (root, nodeStatsArray, next) {
|
, errors: function (root, nodeStatsArray, next) {
|
||||||
next();
|
next();
|
||||||
},
|
}
|
||||||
},
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
walker = walk.walkSync('/tmp', options);
|
walker = walk.walkSync("/tmp", options);
|
||||||
|
|
||||||
console.log('all done');
|
console.log("all done");
|
||||||
})();
|
}());
|
||||||
```
|
```
|
||||||
|
|
||||||
# API
|
API
|
||||||
|
====
|
||||||
|
|
||||||
Emitted Values
|
Emitted Values
|
||||||
|
|
||||||
- `on('XYZ', function(root, stats, next) {})`
|
* `on('XYZ', function(root, stats, next) {})`
|
||||||
|
|
||||||
- `root` - the containing the files to be inspected
|
* `root` - the containing the files to be inspected
|
||||||
- _stats[Array]_ - a single `stats` object or an array with some added attributes
|
* *stats[Array]* - a single `stats` object or an array with some added attributes
|
||||||
- type - 'file', 'directory', etc
|
* type - 'file', 'directory', etc
|
||||||
- error
|
* error
|
||||||
- name - the name of the file, dir, etc
|
* name - the name of the file, dir, etc
|
||||||
- next - no more files will be read until this is called
|
* next - no more files will be read until this is called
|
||||||
|
|
||||||
Single Events - fired immediately
|
Single Events - fired immediately
|
||||||
|
|
||||||
- `end` - No files, dirs, etc left to inspect
|
* `end` - No files, dirs, etc left to inspect
|
||||||
|
|
||||||
- `directoryError` - Error when `fstat` succeeded, but reading path failed (Probably due to permissions).
|
* `directoryError` - Error when `fstat` succeeded, but reading path failed (Probably due to permissions).
|
||||||
- `nodeError` - Error `fstat` did not succeeded.
|
* `nodeError` - Error `fstat` did not succeeded.
|
||||||
- `node` - a `stats` object for a node of any type
|
* `node` - a `stats` object for a node of any type
|
||||||
- `file` - includes links when `followLinks` is `true`
|
* `file` - includes links when `followLinks` is `true`
|
||||||
- `directory` - **NOTE** you could get a recursive loop if `followLinks` and a directory links to its parent
|
* `directory` - **NOTE** you could get a recursive loop if `followLinks` and a directory links to its parent
|
||||||
- `symbolicLink` - always empty when `followLinks` is `true`
|
* `symbolicLink` - always empty when `followLinks` is `true`
|
||||||
- `blockDevice`
|
* `blockDevice`
|
||||||
- `characterDevice`
|
* `characterDevice`
|
||||||
- `FIFO`
|
* `FIFO`
|
||||||
- `socket`
|
* `socket`
|
||||||
|
|
||||||
Events with Array Arguments - fired after all files in the dir have been `stat`ed
|
Events with Array Arguments - fired after all files in the dir have been `stat`ed
|
||||||
|
|
||||||
- `names` - before any `stat` takes place. Useful for sorting and filtering.
|
* `names` - before any `stat` takes place. Useful for sorting and filtering.
|
||||||
|
* Note: the array is an array of `string`s, not `stat` objects
|
||||||
|
* Note: the `next` argument is a `noop`
|
||||||
|
|
||||||
- Note: the array is an array of `string`s, not `stat` objects
|
* `errors` - errors encountered by `fs.stat` when reading ndes in a directory
|
||||||
- Note: the `next` argument is a `noop`
|
* `nodes` - an array of `stats` of any type
|
||||||
|
* `files`
|
||||||
- `errors` - errors encountered by `fs.stat` when reading ndes in a directory
|
* `directories` - modification of this array - sorting, removing, etc - affects traversal
|
||||||
- `nodes` - an array of `stats` of any type
|
* `symbolicLinks`
|
||||||
- `files`
|
* `blockDevices`
|
||||||
- `directories` - modification of this array - sorting, removing, etc - affects traversal
|
* `characterDevices`
|
||||||
- `symbolicLinks`
|
* `FIFOs`
|
||||||
- `blockDevices`
|
* `sockets`
|
||||||
- `characterDevices`
|
|
||||||
- `FIFOs`
|
|
||||||
- `sockets`
|
|
||||||
|
|
||||||
**Warning** beware of infinite loops when `followLinks` is true (using `walk-recurse` varient).
|
**Warning** beware of infinite loops when `followLinks` is true (using `walk-recurse` varient).
|
||||||
|
|
||||||
# Comparisons
|
Comparisons
|
||||||
|
====
|
||||||
|
|
||||||
Tested on my `/System` containing 59,490 (+ self) directories (and lots of files).
|
Tested on my `/System` containing 59,490 (+ self) directories (and lots of files).
|
||||||
The size of the text output was 6mb.
|
The size of the text output was 6mb.
|
||||||
@ -296,11 +295,12 @@ Note that `find.js` omits the start directory
|
|||||||
|
|
||||||
In conclusion node.js asynchronous walk is much slower than regular "find".
|
In conclusion node.js asynchronous walk is much slower than regular "find".
|
||||||
|
|
||||||
# LICENSE
|
LICENSE
|
||||||
|
===
|
||||||
|
|
||||||
`node-walk` is available under the following licenses:
|
`node-walk` is available under the following licenses:
|
||||||
|
|
||||||
- MIT
|
* MIT
|
||||||
- Apache 2
|
* Apache 2
|
||||||
|
|
||||||
Copyright 2011 - Present AJ ONeal
|
Copyright 2011 - Present AJ ONeal
|
||||||
|
|||||||
@ -17,7 +17,7 @@
|
|||||||
},
|
},
|
||||||
"lib": ".",
|
"lib": ".",
|
||||||
"main": "./lib/walk.js",
|
"main": "./lib/walk.js",
|
||||||
"version": "2.3.15",
|
"version": "2.3.14",
|
||||||
"files": [
|
"files": [
|
||||||
"lib"
|
"lib"
|
||||||
],
|
],
|
||||||
@ -33,6 +33,9 @@
|
|||||||
"example": "examples",
|
"example": "examples",
|
||||||
"test": "test"
|
"test": "test"
|
||||||
},
|
},
|
||||||
|
"files": [
|
||||||
|
"lib"
|
||||||
|
],
|
||||||
"devDependencies": {},
|
"devDependencies": {},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"test": "./test/walk-test.sh"
|
"test": "./test/walk-test.sh"
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user