import * as betterIterators from "https://deno.land/x/better_iterators@v1.5.0/mod.ts";
Better Iterators
This module provides Lazy and LazyAsync classes which make it easy to chain together data transformations.
The lazy function is the simplest way to get started.
import { lazy, range } from "./mod.ts"
// You can use any Iterable here (such as an Array, or Generator), but
// Lazy objects are themselves Iterable:
let iterable = range({to: 1000})
let results = lazy(iterable)
.filter(it => it % 2 == 0)
.map(it => it*it)
.limit(10)
// No iteration has happened yet.
// This will trigger it:
for (let item of results) { console.log(item) }
Lazy Iteration Consumes All Input
Note that iterating a Lazy(Async) will consume its items -- the operation is not repeatable. If you need to iterate multiple times, save your result to an array with Lazy#toArray
import { lazy, range } from "./mod.ts"
let results = lazy([1, 2, 3, 4, 5]).map(it => `number: ${it}`)
for (let result of results) { console.log("first pass:", result)}
// Has no more values to yield, will throw an exception:
for (let result of results) { console.log("second pass:", result)}
Asynchronous Iteration With Promises (Not Recommended)
Other iterator libraries show examples of parallel/async iteration like this:
import { lazy, range } from "./mod.ts"
let urls = [
"https://www.example.com/foo",
"https://www.example.com/bar"
]
let lazySizes = lazy(urls)
.map(async (url) => {
let response = await fetch(url)
return await response.text()
})
// The type is now Lazy<Promise<string>> so we're stuck having to deal
// with promises for the rest of the lazy chain:
.map(async (bodyPromise) => (await bodyPromise).length)
// and so on...
// `lazySizes` also ends up as a `Lazy<Promise<number>>`, so we've got to
// await all of the items ourselves:
let sizes = await Promise.all(lazySizes.toArray())
This approach might seem to work, but it has unbounded parallelism. If you
have N URLs, .toArray()
will create N promises, and the JavaScript runtime
will start making progress on all of them simultaneously.
That might work for small workloads, but network and memory resources are not unbounded, so you may end up with worse, or less reliable performance.
Lazy Asynchronous Iteration
Better Iterators provides a simpler, safer API when working with async code:
You can convert a Lazy
to a LazyAsync
:
import { lazy } from "./mod.ts"
let urls = [
"https://www.example.com/foo",
"https://www.example.com/bar"
]
let lazySizes = lazy(urls)
.toAsync()
.map(async (url) => {
let response = await fetch(url)
return await response.text()
})
// here the type is LazyAsync<string> (not Lazy<Promise<string>>)
// so further lazy functions are easier to work with:
.map(it => it.length)
Here, LazyAsync#map does the least surprising thing and does not introduce parallelism implicitly. You've still got serial lazy iteration.
If you DO want parallelism, you can explicitly opt in like this:
import { lazy } from "./mod.ts"
let urls = [
"https://www.example.com/foo",
"https://www.example.com/bar"
]
let lazySizes = lazy(urls)
.map({
parallel: 5,
async mapper(url) {
let response = await fetch(url)
return await response.text()
}
})
// etc.
Functional Idioms
You'll find other common Functional Programming idioms on the Lazy
and LazyAsync types, like associateBy
, groupBy
, fold
, sum
,
partition
, etc.
Functions
f lazy | Create a Lazy(Async) from your (Async)Iterator. |
Returns a Lazy which will yield consecutive numbers. |
Interfaces
Passing this to the map() function indicates that you want to map values | |
The result of partitioning according to a | |
Type Aliases
Filters for matches (where boolean is true) | |
Only | |
A function that transforms from In to Out |