iowo/docs/design/iowo-design.typ

#import "../template.typ": conf
#import "util/graphics.typ"
#import graphics: arrow

#show: conf.with(
  title: [iOwO design],
  subtitle: [don't worry, we're just dreaming],
)

= Term overview

/ Processing stages:
  Whole workflow of iOwO,
  from the source to evaluation.

/ Source:
  Nice textual representation of what iOwO is supposed to do.
  Consists of instructions and pipelines.

/ Graph IR:
  Intermediate machine-readable representation of the source.
  Can be modified by the optimizer
  and is evaluated or "ran" by the runtime.

/ Optimizer:
  Simplifies the graph IR and makes it faster to run.

/ Runtime:
  All-encompassing term for what's done
  after a graph IR is optimized and fully ready to go.

/ Scheduler:
  Looks at the graph IR
  and decides which evaluator gets to run which part of it.

/ Evaluator:
  One specific implementation of how to run
  through the whole graph IR to get its results.

/ Function:
  On the source level and before the graph IR,
  anything that can be run with inputs,
  possibly receiving outputs.

/ Instruction:
  Function, but in the graph IR and at runtime.
  Ask schrottkatze on why the differentiation is important.

/ Input:
  Received by a function or instruction.
  Different inputs may result in different behavior
  and/or in different outputs.

/ Argument:
  On the source level,
  an input which is given ad-hoc
  instead of provided through a pipeline.

/ Output:
  Returned by a function or instruction,
  and can be fed into other functions or instructions.

/ Consumer:
  Function or instruction that takes at least 1 input.

/ Streamer:
  Function or instruction that returns at least 1 output.

/ Modifier:
  Function or instruction that is _both_ a consumer and a streamer.

/ Pipeline:
  Any chain of streamers and consumers,
  possibly with modifiers in-between,
  that may branch out
  and recombine arbitrarily.

= Processing stages

#graphics.stages-overview

iOwO operates in stages.
This has a number of benefits and implications:

- One can just work on one aspect of the stages without having to know about the rest.
- Bugs are easier to trace down to one stage.
- Stages are also replacable, pluggable and usable somewhere else.
  - For example,
    one could write a Just-In-Time compiler as a new evaluator to replace the runtime stage,
    while preserving the source #arrow() graph IR step.

However, this also makes the architecture somewhat more complicated. So here we try our best to describe how each stage looks like. If you have any feedback, feel free to drop it on #link("https://forge.katzen.cafe/katzen-cafe/iowo/issues")[the issues in the repository]!

== Source <source>

```iowo
open base.png    >|
open stencil.png >|
mask
|> show
|> (invert | show)
```

Functional and pipeline-based.
However, in contrast to classic shell commands,
functions can have multiple outputs and multiple inputs.

=== Functions

`open`, `invert`, `mask`, `show` and friends.

The foundation of actually "doing" something.

- Can have any, even infinite, amount of inputs and outputs.
  - Their amounts may or may not be equal.
- Inputs and outputs must have their types explicitly declared.
- An function with
  - at least one output is called a streamer.
  - at least one input is called a consumer.
  - _both_ at least one input and at least one output is _both_ a streamer and a consumer,
    and culminatively called a modifier.
- May also contain spaces in its name.

==== Inputs <input>

- Based on position.
- Inputs can be provided both through the pipeline and ad-hoc.
- Ad-hoc inputs are called arguments.
- So all of these are equivalent:

```iowo
add 1 2        -- all inputs as arguments
[ 1 2 ] | add  -- all inputs through the pipeline
1 | add 2      -- one input through the pipeline, one as argument
```

==== Outputs

- Also based on position.

=== Pipelines <pipeline>

- Exchange data between streamers and consumers.

==== Simple forwarding

In the simplest case,
where inputs map to outputs bijectively#footnote[one to one],
pipelines are just pipes and forward unchanged:

```iowo
open owo.png | invert | save owo.png
```

==== Splitting <splitting>

To handle each output of a streamer individually, they can be _split_:

```iowo
mask
|> show    -- would show the masked part
|> invert  -- would invert the unmasked part
```

==== Combination <combination>

To throw multiple streamers into the inputs of a consumer,
they can be _combined_:

```iowo
open base.png    >|
open stencil.png >|
mask
```

However, since lists are automatically spliced into inputs,
this is equivalent to the above:

```iowo
[
  open base.png,
  open stencil.png,
]
| mask
```

=== Comments

Done with any of `--` or `//`.

=== Data types

==== Lists

- Signified by `[]` braces.
- If thrown into a pipeline, it automatically works like a streamer.
- Can be delimited by commas.
  - Must be delimited by commas if a contained streamer takes an input.
- May have a trailing comma.
- Outputs of streamers are spliced into the contained list.
  - In effect, they are automatically flattened.

== Graph IR

#graphics.graph-example

The parsed representation of the source, and also what the runtime operates on.

In a way, this is the AST, except that it's not a tree.

It is represented in iOwO using adjacencies, where essentially
the vertices#footnote[Nodes or functions in this case.]
and their edges#footnote[Connections or pipes in this case.]
are stored separately.

=== Optimizer

Merges and simplifies functions in the graph IR.

== Runtime <runtime>

Runs through all functions in the graph IR.
It does not have any significantly different representation,
and despite its name there's _no_ bytecode involved.

=== Scheduler

Looks at the graph IR and decides when which evaluator gets to evaluate what.

=== Evaluator

Runs instructions given to it in a specific way,
such as for example on the GPU using OpenCL.

= Open questions

- @input
  - At which position are arguments injected into function inputs?
    - How could that be controlled if set to e.g. the end by default?
    - Not all inputs are order-independent, e.g. `div`
  - Should inputs and outputs really be positional?
    - Could make more complex functions hard to read
    - But keyworded could also make code very noisy
    - Maybe a middle ground, such that at most 1 input is allowed to be positional?
- @pipeline
  - We need some way to reshuffle and reorder outputs and inputs in a pipeline
- @splitting
  - How would one split different outputs into a list?
- @runtime
  - What is the difference between an instruction and a function?
- Should outputs that are not thrown into a consumer
  be automatically displayed in some kind of debug view?
  - Or should that be done instead using a debug `show` function or the like?
- Should consumers be called sinks instead?
  - Shorter
  - More ambiguous if only looking at the first char though