Learning From One Bit of Feedback in a Modular Hierarchical rApp

Learning through a common interface

Every 15 minutes, the rApp decides whether a capacity cell can be switched off. Its components select a switching strategy, a load threshold and a source of context such as weather or road traffic. A source can become unreliable, and an accurate forecast can still be poorly suited to an operator's priorities. The application needs to learn from the consequences of its choices.

Only the root receives the binary judgement of the completed switching decision explicitly. It raises or lowers the weight of the component it selected. That component reads the direction of the change, updates its own selected option, and passes the next weight-change bit down the path. Each selector retains its own context, alternatives and learning rule.

The bit carries the outcome's information. It does not remove the need for feedback. Under the method's conditions, the direction of each selected weight change recovers the outcome, preserving the choices and weights of a hierarchy given that outcome at every level. This paper applies the hierarchical learning method to a modular rApp and evaluates its behaviour.

One selected path, five local choices

The base rApp has eight selectors. Adding the forecast service gives eleven selectors in total, with up to five on a selected path. The service learns product and alert-threshold choices without receiving the operator's scoring rule or the rApp's decision record. In this experiment its selectors share weights across callers and each uses a single context.

What the replays show

The evaluation replays 83 days of public counters from 277 capacity cells, with 2,074,760 decisions per full run. Conditions and forecast products are simulated. Comparisons use five condition timelines and four selection seeds per timeline.

The forecast-service gains have paired 95% intervals of [2.22, 5.22] and [1.13, 4.15] percentage points. These are gains on B's requests. Across all rApp decisions, the corresponding differences are +0.10 points [−0.05, +0.25] and +0.04 points [−0.08, +0.17], which do not establish a whole-rApp gain.

In the source experiment, the failing source's mean selection weight falls from 0.381 at the end of its healthy period to 0.251 after drift and 0.155 after outage. Ordinary learning updates drive the change, without a separate failure detector. Compared with freezing source weights, adaptation increases switch-offs and reduces errors among them, while increasing wrong switch-offs as a fraction of all decisions.

Source: Sections V–VI of the paper. Decision score is the mean binary outcome under the stated operator objective. The forecast experiment uses a missed-opportunity reward probability of 0.8, and the source experiment uses 0.5. Confidence intervals use five timeline averages, each averaging four selection seeds.

What this establishes

This is an application architecture and a replay evaluation. It demonstrates useful adaptation under the specified objectives and preservation of explicit-outcome learning. It adds no new optimality or regret guarantee and does not establish superiority over other adaptive controllers.

The traffic trace is fixed, so switching decisions do not change later traffic. Conditions, forecast products and the transfer of traffic to another cell are modelled. The results do not measure energy savings or the performance of actual forecast products. All runs share one recorded network trace, and some preprocessing uses the full trace.

The service boundary is modelled within one process. A deployment would need to match each delayed bit to its request, retain the selected local choices and preserve update order. Stable decoding also matters near floating-point limits. The implementation uses the sibling-weight sum when the selected weight exceeds one half. A testbed with separately controlled components and measured network outcomes is the next step.

Paper, code and supporting evidence

The reproduction package, release 2.0.0, contains frozen code, processed counter inputs, cached model responses, reference results and checksums. Its main archive holds 900 complete runs, including the source comparison and forecast-service evaluations at three operator objectives. A supplementary archive holds 560 cost-sensitivity runs. Those additional runs reuse the same trace and condition timelines.

Verification commands check saved results, recompute reported numbers and redraw figures. Separate replay commands rerun the experiments. Cached responses mean no model endpoint or fresh inference is needed after installing the listed dependencies. The language model used as a context source is not trained by this feedback.

• Paper record, posted 20 October 2026 and matching seven-page PDF.
• Code, processed inputs and reference results. All reproduction versions.
• Underlying public network-counter dataset, using Dataset_03.
• The underlying hierarchical learning method and its public preprint.
• CAMINO, the earlier context-aware intent-mediation application.

Abstract

An energy-saving rApp decides every 15 minutes whether a capacity cell can be switched off. We build it as a modular hierarchy whose components learn to choose switching strategies, load thresholds and context sources. Only the top-level component receives the outcome of the rApp's final switching decision. According to that outcome, it raises or lowers the weight of the component it selected. Each lower component learns from whether the weight assigned to it by the component above increased or decreased, then updates the weights for its own choices. This carries one bit of feedback down the selected path while each component keeps its context and internal choices to itself. A simulated weather forecast service illustrates how a component with its own internal hierarchy uses the same feedback to tune its choices to the operator's switching priorities. The forecast service doesn't receive the rApp's decision record or its scoring rule. We evaluate the hierarchy by replaying 83 days of public counters from 277 capacity cells, with simulated conditions and simulated forecast products. The replays demonstrate adaptation away from a failing context source and tuning of forecast responses to an operator prioritising avoidance of wrong switch-offs. Across different component contexts and learning rules, the hierarchy reproduces the decisions and learned weights obtained with explicit outcome feedback, even when feedback arrives only after each 15-minute interval closes.