Reference: a call tree's log

replay(events) -> LogState

What a live watcher of this form of a log saw once it had these events.

A reply that came whole, shown as one text piece per field (streaming.md, law 6).

replay!(state, e) -> state

Apply one event (laws 1 to 4): started adds its call; request and retry empty the call's fields; text adds to its field; done and failed end the call. A kind this reader does not know changes nothing.

LogState

What replaying a form of a log gives after each event (contract/streaming.md, "Replaying"): for each call started so far, its CallState; and whether the log is finished (its outermost call ended).

What a watcher of a form was shown of one call: whether it ended (:done, :failed, or nothing) and each field's text so far.

resume(events, after) -> Vector{Event}

What a source holding these events (one form of one log) gives a reader that has events up to after (a Position, or nothing): the events after it. A source that does not have that event refuses event-unknown (the reader then starts again from the beginning).

Follower(form = :kept)

A reader following logs live (contract/streaming.md, "Following a log"), keeping for each tree the events it holds. form is the form it reads: :kept (the kept form, or a view made from it: the same from every source) or :live (the whole log, or a view made from it: it may show values the kept form lacks, and only the process that gave them can give them again). Feed it events with receive!; LogState(f, tree) is what it shows; recover! reads again from a source after a loss.

receive!(f::Follower, e; from = e.writer) -> Symbol

Take one event received live, and say what it was: :kept (the next event), :duplicate or :stale (dropped), :rewind (a later writer went on from an event the reader holds: it drops what it holds after that event and takes this one), :loss (events were lost: read again, recover!), or :unknown_format (it stops following: nothing after is read). Positions are compared whole, writer and seq together. from is the process that gave it (a writer number, or :store), which decides where it may resume.

An object that says it is of format 2 but is not an event of it (it does not pass the contract's schema) is :malformed: dropped, changing nothing. Its position cannot be trusted, so the next event shows whether any was lost.

recover!(f::Follower, tree, source, from) -> reads

Read a tree again after a loss (contract/streaming.md, "Resuming"): after the last event it holds when it may resume in place, and from the beginning when it may not, or when the source answers event-unknown. source is what gives the events: a store (events_after(source, tree, after)) or a vector of the events it holds; from is :store or the writer number of the process. Returns its reads, each (after, events) or (after, refusal).

kept_event(e, keep::Keep, program) -> Event or nothing

The kept form of one event (contract/streaming.md, "The kept form"), given which fields of its call are kept and its call's program (the started event's: its kind and answer). nothing: not in the kept log. It is made from the event's JSON form only: nothing else of the call goes with it.

kept_log(events, keeps) -> Vector{Event}

The kept form of a whole log: each event's kept form, by its call's Keep (keeps[call]), with after set to the kept event before it.

Journal(store; required = false, retries = 2, timeout = 30)

Where each call tree's kept log is kept while it is written (contract/ streaming.md, "Keeping a log while it is written"): a store that keeps logs by the store rules (MemoryStore, or any EventStore). Best effort (the default) never makes a call wait: the store's code runs on a task of its own, as an observer's does (Julia's tasks share threads: see FunctAI.drain for a store that holds its thread without yielding). required = true: the call waits until its events are confirmed at its start, before each tool runs, and at its end, and raises JournalError when they are not. An event is sent at most 1 + retries times in a row before the writer gives up on it for now; a send the store has not answered after timeout seconds (nothing: no limit) counts as no answer.

with_settings(journal = FunctAI.Journal(store; required = true)) do … end
FunctAI.configure!(journal = store)          # best effort, for every tree

JournalError

A call tree's journal breaking (contract/streaming.md): code is "journal-policy" (a closer setting replaces or removes a journal the host set, or weakens a required one: refused before the tree runs), "journal-scope" (a required journal set only inside a tree), "journal-barrier" (a required journal did not confirm the call's start or a tool call: it did not go on), or "journal-end" (it did not confirm the call's end). A journal-end error holds the call's outcome, as the call record does ((done = value,) or (failed = exception,)), names its terminal event by position in the log tree (the outermost call's id), and says journal: "refused" or "unknown" (no answer: settle(err) reads the journal). cause is what the journal met, when it met something (a store's refusal, an error, a send it never answered); store is the journal's store.

FunctAI.drain(timeout = 5.0) -> Bool

Wait, at most timeout seconds, until every observer has been given the events made so far and every journal writer has sent what it was given; whether they all were. Calls never wait for them (a slow observer or a best-effort journal does not slow a call): a script that must see them all before it goes on, or before it exits, drains. FunctAI drains for 2 seconds when Julia exits.

Observers and stores run on Julia tasks, which share threads and take turns when one yields (sleeps, waits, reads or writes). Code that holds its thread without yielding (Libc.systemsleep, a long computation, a C library's blocking call) holds every task on that thread until it is done. With one default thread (julia --threads=1, and a spawned call under Julia's default of one default thread beside the interactive one) calls on that thread then wait for it; start Julia with several threads (julia --threads=auto) to keep such an observer or store off the calls' threads, or make it yield.

FunctAI.detached(f)

Run f() outside the call it is in: the calls it makes start trees of their own (their own log, journal and record root) instead of being steps of that call. A call waits, before it ends, for every call made inside it; work meant to outlive it (a task that warms a cache, say) is started detached:

@program function answer(q::String)::String
    FunctAI.detached() do
        @async warm_up(q)          # not a step of answer: its own tree
    end
    reply(q)
end

FunctAI.EventStore

What keeps call trees' logs for others to read, by the rules every store keeps (contract/streaming.md, "The rules a store keeps"). A store is a subtype with three methods:

• FunctAI.keep!(store, event_or_batch) appends an Event (or a vector of them, one step): returns :kept or :duplicate, or throws a StoreRefusal with the contract's code. Any other exception is "no answer" (the append may have been kept: the writer sends it again), except a store that cannot be called (MethodError, UndefVarError, …), which stops the journal writer, with its cause.
• FunctAI.claim!(store, tree) gives a later writer a Claim.
• FunctAI.events_after(store, tree, after) reads the kept events after a position (every event after nothing), or refuses event-unknown.

MemoryStore is one, in memory.

MemoryStore(name = "memory")

A store of logs in memory, by the rules every store keeps (contract/ streaming.md, "The rules a store keeps"): claim!, keep! (an event, or a batch), events_after. Each claim and each append is one step per log. Every event is checked against the contract's event schema before anything else, whether it comes as JSON or as an Event. A journal (Journal) keeps call trees' logs in it while they are written:

store = FunctAI.MemoryStore()
with_settings(journal = FunctAI.Journal(store; required = true)) do
    support("Where is my parcel?")
end
FunctAI.events_after(store, tree, nothing)

claim!(store, tree) -> Claim

A later writer claims an unfinished log: the store gives it a writer number (one more than any it gave for the log) and the position of the last kept event, and from then on refuses every event of an earlier writer (fencing). Refused event-unknown for a log it does not have, event-after-end for a finished one.

What a claim gives a later writer: its writer number, and the position of the last kept event it numbers on from.

keep!(store, event) -> :kept or :duplicate
keep!(store, events::AbstractVector) -> :kept or :duplicate

Append an event (or a JSON object) to its tree's log, as the store rules check it: an event sent again is :duplicate; otherwise it throws a StoreRefusal (event-malformed, event-conflict, event-gap, event-after-end, event-start). A batch is checked event by event as if each were appended alone, and kept whole or not at all: :duplicate when every event is one; a refusal names the first event refused. What is kept is a copy: nothing the caller does to what it gave changes the log.

events_after(store, tree, after) -> Vector{Event}

The kept events of a tree's log after the one after names (all of them after nothing). Refused event-unknown when the store does not have that event. Reading is never fenced.

settle(store, tree, event::Position) -> :kept, :not_kept or :another_end

What reading the journal says of an end its writer could not confirm (JournalError journal-end with journal == "unknown"; contract/ streaming.md): the log holds that event (:kept); it does not and the log is finished (:another_end: another writer ended it, and the outcome was not kept); or it does not and the log is unfinished (:not_kept, final only once the caller has claimed the log).

settle(err::JournalError) -> :kept, :not_kept or :another_end

What the journal says of the end a journal-end error could not confirm: settle(err.store, err.tree, err.event).

StoreRefusal

A store refusing an append, a claim or a read (contract/streaming.md, "The rules a store keeps"): code is event-malformed, event-conflict, event-gap, event-after-end, event-start or event-unknown; event the position of the event refused (in a batch, the first refused).

LogWriter(journal, tree)

Appends one tree's kept events to its journal, in order, as they are made (contract/streaming.md, "Appending"): each event is sent at most 1 + retries times in a row; an event the journal answers :kept or :duplicate is confirmed; one it refuses stops the writer (it appends nothing more); one that gets no answer is kept and sent again with the next event. A store that cannot be called (MethodError, UndefVarError: not an answer, and sending again would not help) stops it too. Sending runs on a task of its own, at most one send at a time (a send the store has not answered is waited for again, never started again beside itself), so a call never waits for a best-effort journal; confirmation waits for an event's own round (a required journal's barrier).

confirmation(w::LogWriter, n) -> :confirmed, :unanswered or :refused

Wait until the nth event given to the writer is confirmed, or has had its round (after the writer's own resends) without being confirmed. Events given after it (a call running beside this one) are not waited for.

Wait until the journal confirmed the event (every event up to it: appends are in order), or gave up on it: :confirmed, :unanswered or :refused. :confirmed without a journal.