1. Why
2. Goals
3. API
4. Status
5. Problem Solving?

1. Why Why reinvent streams

The Streams user experiance is bad

Streams performance is underwhelming

"If you think streams are a problem in Node.js.... they are." - @matteocollina

2. Goals

• Performance and ease-of-use are key.
• Implementable in a performant and usable way for both JS and C++.
• Browser portability is preferable.

Protocol Goals

• Pull-based (backpressure)
• Binary-only (simplicity)
• Stateless (in-protocol)
• One-to-one (no events)
• Timing agnostic (sync or async)
• No buffering (in-protocol)
• In-line control flow (errors and "end")

3. API

Some terms:

• Consumer / "Sink" - API where data goes to.


• Producer / "Source" - API where data comes from.


• "Protocol" - combination of a Consumer & Producer

Keep in mind that BOB is a protocol specification, not an inheritable api

Sink API

class Sink {
  constructor (opts) {}

  bindSource (source, bindCb) {
    this.source = source
    this.source.bindSink(this)
    return this
  }

  next (status, error, buffer, bytes) {
    // do stuff with received data
    this.source.pull(null, buffer)
  }
}

Source API

class Source {
  constructor (opts) {}

  bindSink (sink) {
    this.sink = sink
  }

  pull (error, buffer) {
    // get data
    this.sink.next(status_type.continue, null,
                   buffer, bytesWritten)
  }
}

class PassThrough {
  bindSource (source) {
    source.bindSink(this)
    this.source = source
    return this
  }

  bindSink (sink) {
    this.sink = sink
  }

  next (status, error, buffer, bytes) {
    this.sink.next(status, error, buffer, bytes)
  }

  pull (error, buffer) {
    this.source.pull(error, buffer)
  }
}

Composing a Stream

const { Stream } = require('bob-streams')

const source = new Source(/* args */)
const sink = new Sink(/* args */)

const stream = new Stream(source, sink)
stream.start(error => {
  // The stream is finished when this is called.
})
    

Composing a Stream

const { Stream } = require('bob-streams')

const source = new Source(/* args */)
const sink = new Sink(/* args */)

const stream = new Stream(source, sink)
await util.promisify(stream.start.bind(stream))()
// The stream is finished when the await completes

    

Composition with a transform

const { Stream } = require('bob-streams')

const source = new Source(/* args */)
const xform = new Transform(/* args */)
const sink = new Sink(/* args */)

const stream = new Stream(source, xform, sink)
stream.start(error => {
  // The stream is finished when this is called.
})
    

API Verification

const { Stream, Verify } = require('bob-streams')

const source = new Source(/* args */)
const xform = new Transform(/* args */)
const sink = new Sink(/* args */)

const stream = new Stream(
  source,
  new Verify(), // Checks protocol between source and xform
  xform,
  new Verify(), // Checks protocol between xform and sink
  sink
)
stream.start(error => {
  // The stream is finished when this is called.
})
    

github.com/Fishrock123/bob

/diagrams

4. Status

Current Modules

• bob-streams
• fs-source
• fs-sink
• zlib-transform
• crc-transform
• github.com/Fishrock123/socket

Current Helpers

Stream(source, ...sinks)

WritableSource() extends Readable
ReadableSink() extends Writable

BufferSource(buffers)

AssertionSource(assertions)
AssertionSink(assertions)
    

Current Status is... stalled

... and possibly (not?) defunct...

Discussion!

github.com/Fishrock123/bob

Problem Solving

Groups of 3-5

openjs-foundation/summit/issues/196

• API Enforcement
• Content-addressibility
• "writev"
• Buffer allocation hints

How to enforce the API? (Currently only convention backed by `Verify()`)

Is content-addressibility desirable / worthwhile? (As in specifying a binary index?)

Use-cases that are better served via writev? (Aside from theoretical performance?)

An API for neatly giving buffer allocation hints? (i.e. source to sink?)

Slides: Fishrock123/nodejs-collab-summit-montreal-2019