Scaling Observability via Attention and Awareness

Competitive Signaling

The attention schema theory (AST) of consciousness is an evolutionary and neuropsychological scientific theory developed by neuroscientist Michael Graziano. The theory proposes that brains construct subjective awareness as a schematic model of the process of attention. Because of limited processing resource capacities, brains focus more on some signals than on others – basically, signals compete for the brain’s attention. This internal competition is partially under the bottom-up influence of a sensory stimuli model and somewhat under top-down control of other mental states, including goals – this is very similar to how situational awareness is theorized to operate optimally.

Attention and Awareness

Attention, the ability to process select information in a focused manner, builds a set of information, or a representation, descriptive of attention – a schema or model. The construct of subjective awareness is then the brain’s efficient but imperfect model of its attention. Because awareness is based on an attention model, the model will pertain to the same information domain. This simplified internal model is similar to what is seen in dynamical systems control. A controller employs an internal model of the item it controls for increased effectiveness and efficiency. Awareness is a feedback mechanism used to regulate the attentional state. One could say that monitoring is to observability what awareness is to attention.


Scaling Blueprint

OpenSignals tackles scaling of observability on many fronts – at the machine and human interfaces and along the data pipeline starting at the source. Before discussing how OpenSignals addresses situational awareness in the context of a system of (micro)services (deferred for a future post), it is crucial to consider the foundational model elements independently of a domain. Irrespective of the domain, services, resources, schedules, etc., OpenSignals consists of a small set of concepts, with signal and status being the two most important. A signal falls into two categories: an operation or an operation outcome of some intercommunication, interaction, or disturbance within an environment. A signal is said to be fired by a thing within the domain. In the case of a system of services, that thing is a service.

A Signal is not a Message

A signal is not an event or message – it has no data payload associated. A signal is a meaningful token that signifies something of significance, specific to the domain, related to a thing. When a signal is captured and recorded by OpenSignals, it has an orientation to indicate whether the signal was observed (emitted by) by the thing itself or whether it was observed (a receipt of) by another thing or observer. Within a domain, the set of signals should be small.

Operational Goal Driven

A status is some inferred operational state for a thing. It is not fired by instrumentation but is generally emitted via some underlying inference engine employed by an OpenSignals service provider that either subscribes to signal callbacks or decorates the thing type instances. The set of status values should be, for the most part, dictated by the operational goal. Likewise, the set of signals within a domain should be selected based on their relevance to a states’ inferencing. Ideally, a signal should map to a single status value or none at all. When not mapped to a status value, a signals’ relevance comes into play within the context of a sequence of other signals. In some cases, a status value, like DEVIATING, is not directly mapped to a single signal.

Inductive to Reductive

The set of status values should be much smaller than the set of signal values. The scaling of observability in this model happens at the source by translating an event, response, or error into a signal – from a big bag of data and details to a single token. While signaling will have much the same frequency of the events themselves, the transferred data is significantly reduced. That is, if the transmission does indeed need to happen, subscribers can relay only the inferred status when changed for a thing. Changing a thing’s operational status while driven by signal will typically occur much less frequently, usually scaled in its calculation to some time window and operation phase.

Locality and Loops

OpenSignals moves much of the processing of operational status to the source, making it highly efficient and effective in seeing the quality of service from many interaction points and perspectives (groupings) while allowing local customization of the inference engine in terms of signal sensitivity and sequence scoring. And again, this all operates on tokens meaningful to humans and machines at different space and time scales, offering immediate and local adaptation to attention and awareness. The feedback loop in humans between attention and awareness is similarly mirrored in signals and status. A status change can alter future sensitivities to signals – understanding of the current status controls how attentive the inference engine is to a specific signal.