Better Iterators

This module provides Lazy and LazyAsync classes which make it easy to chain together data transformations.

The lazy function is the simplest way to get started.

import { lazy, range } from "./mod.ts"

// You can use any Iterable here (such as an Array, or Generator), but
// Lazy objects are themselves Iterable:
let iterable = range({to: 1000})

let results = lazy(iterable)
    .filter(it => it % 2 == 0)
    .map(it => it*it)
    .limit(10)

// No iteration has happened yet.
// This will trigger it:
for (let item of results) { console.log(item) }

Lazy Iteration Consumes All Input

Note that iterating a Lazy(Async) will consume its items -- the operation is not repeatable. If you need to iterate multiple times, save your result to an array with Lazy#toArray

import { lazy, range } from "./mod.ts"

let results = lazy([1, 2, 3, 4, 5]).map(it => `number: ${it}`)

for (let result of results) { console.log("first pass:", result)}

// Has no more values to yield: (may throw an exception in the future!)
for (let result of results) { console.log("second pass:", result)}

Asynchronous Iteration With Promises (Not Recommended)

You could use a Lazy directly for async work, but it has some problems:

import { lazy, range } from "./mod.ts"
let urls = [
    "https://www.example.com/foo",
    "https://www.example.com/bar"
]
let lazySizes = lazy(urls)
    .map(async (url) => {
        let response = await fetch(url)
        return await response.text()
    })
    // The type is now Lazy<Promise<string>> so we're stuck having to deal
    // with  promises for the rest of the lazy chain:
    .map(async (bodyPromise) => (await bodyPromise).length)
    // and so on...

// `lazySizes` also ends up as a `Lazy<Promise<number>>`, so we've got to 
// await all of the items ourselves:
let sizes = await Promise.all(lazySizes.toArray())

This approach might seem to work, but it has unbounded parallelism. If you have N URLs, .toArray() will create N promises, and the JavaScript runtime will start making progress on all of them simultaneously.

Lazy Asynchronous Iteration

For a simpler, safer API when working with async code, you can convert a Lazy to a LazyAsync:

import { lazy, range } from "./mod.ts"
let urls = [
    "https://www.example.com/foo",
    "https://www.example.com/bar"
]
let lazySizes = lazy(urls)
    .toAsync()
    .map(async (url) => {
        let response = await fetch(url)
        return await response.text()
    })
    // here the type is LazyAsync<string> (not Lazy<Promise<string>>)
    // so further lazy functions are easier to work with:
    .map(it => it.length)

Here, LazyAsync#map does the least surprising thing and does not introduce parallelism implicitly. You've still got serial lazy iteration.

If you DO want parallelism, the LazyAsync#mapPar and LazyAsync#mapParUnordered methods let you explicitly opt in at your chosen level of parallelism.