Simulations

Simulations exercise a Deployment before production by driving the Runtime closed loop:

1. POST /invocations for a sampled customer (ranked offers in final_result).
2. Simulated take-up (or none).
3. On accept, POST /response (singular) with that row’s uuid and offer_name.

params on /invocations is a JSON string, never a nested object. Use a stringified empty object, or a stringified input / value arrays object.

Two ways to run that loop:

Workbench is easier. Python is more flexible. Both still need a pushed Dynamic Interaction deployment and a reachable Runtime.

Workbench 2

Open Simulation at /entities/simulation (not the old Laboratory / Jupyter notebook UI).

Identity

Each simulation document has a name, project, deployment, and deployment version. New simulations inherit the header project, then the first deployment/version when those are empty.

Save is never blocked on those fields. Empty names become New simulation / New simulation N. Duplicate explicit names still conflict.

Run Simulation stays disabled until setup is complete. Blockers shown on the Setup card:

• Simulation name
• Project, deployment, and version
• ecosystem-server username and password
• MongoDB available
• ecosystem-notebooks available

The learning algorithm is not chosen on the simulation form. It is resolved from the Dynamic Engagement linked to the deployment. If none is linked, the Runtime scores naively.

Runs

A simulation can have many runs. All run settings (configuration, scenario, customer source, connections, advanced, logs, results) sit under the selected run.

Add Run clones the first run’s configuration, with a new UUID and RunN name so logs and plots stay isolated.

Default new-run values (override per run):

Run configuration

Offers & algorithm

• Number of offers — numberoffers sent to /invocations.
• Offer selection (among presented) — which ranked offer is credited on take-up:
  • top — highest scoring
  • second — always rank 2
  • random — uniform among the Runtime slate
  • distribution — score-weighted
• Algorithm — read-only, from the deployment’s Dynamic Engagement.

Iterations & take-up

• Number of iterations — how many /invocations calls.
• Default take-up rate — Bernoulli accept probability unless historical or policy modes override it.
• Reward binary threshold — used when the deployment’s reward path needs a binary outcome.

Connections

Open the Connections accordion (not buried in Advanced). Simulations authenticate to ecosystem-server with these credentials, not the Workbench UI login.

Click Validate (POST /simulations/validate-connections). The Workbench backend checks:

Typical local notebooks port is 8011 if the container maps that way. If Validate says notebooks is down, start ecosystem-notebooks or change Notebooks URL. If the pulse responder cannot resolve ecosystem-runtime, use a host the notebook server can reach (for example http://localhost:8099 or http://localhost:8091).

Describe scenario (optional)

Natural language → reviewable ScenarioSpec. Failures stay in this panel and never block Save or Run.

1. Describe peaks, take-up drift, or random uptake among presented offers (for example telco weekday peaks 6–8am and 3–7pm).
2. Analyze — intents traffic, take_up, selection.
3. Apply to run — writes the patch into this run only after you confirm.

If the scenario cannot be expressed as config, Generate custom Python produces a script you copy and run yourself. Workbench Run does not execute that script.

Apply can set:

• Traffic profile — virtual day/week compressed into iterations (weekday peaks 6–8 and 15–19 by default). Does not wait for wall-clock hours.
• Take-up policy — flat, historical, linear_drift, cyclical, rank_decline, offer_decline.
• Selection policy — static (use Offer Selection) or drift_lower_sequence (acceptance moves toward lower ranks over the run).

You can also set those under Advanced without using Analyze.

Customer source (optional)

By default, customers come from the deployment Parameter Access lookup.

Enable Use alternate customer source to sample /invocations customers from another Mongo database/collection/lookup field, with an optional aggregation pipeline (then the engine adds $sample). Example lookup field customer; pipeline [{"$match":{"region":"ZA"}}].

If the lookup key does not match the collection, Run fails with a hint to fix Parameter Access or the alternate source.

Advanced

Run, logs, results

1. Save the simulation (drafts are allowed).
2. Validate connections.
3. Run Simulation — Workbench FastAPI proxies to notebooks POST .../process_simulation. Progress shows in the run banner.
4. Logs — GET /simulations/logs?run_uuid=... (Mongo simulations.simulation_logs; legacy ecosystem_meta.threads is still read). Cancel hits notebooks /cancel_simulation.
5. Results — options-store parameter snapshots (simulation_param_outputs) and Beta / plot series (simulation_plot_outputs) when those outputs are enabled. Use plots to inspect Ecosystem Rewards (and similar) arm distributions after the run.

Failures return a message and a hint (unreachable Runtime, empty customer collection, missing server password, notebooks down, bad customer pipeline).

Workbench APIs (operator)

Thin UI; processing stays on the backend:

• GET /simulations/status — Mongo + notebooks health
• POST /simulations/validate-connections
• POST /simulations/run / cancel
• GET /simulations/logs, DELETE /simulations/logs
• GET /simulations/results/parameters, /results/options, /results/plot
• Scenario analyze/apply via the agents router (optional)

Python simulations

Use the Python package (prediction.apis.prediction_engine) when you need a take-up rule Workbench cannot express. Keep the same closed loop.

import random
import time
 
empty_params = "{}"  # params must be a JSON string
invocations_body = {
    "campaign": deployment_id,
    "subcampaign": "simulation",
    "channel": "simulation",
    "customer": customer_id,
    "userid": "simulation",
    "numberoffers": 1,
    "params": empty_params,  # JSON string, e.g. "{}"
}
offer_response = predictor_engine.invocations(auth_runtime, invocations_body)
row = offer_response["final_result"][0]
# Ranked offers: parse offer_name from the row (field names follow the Runtime result)
 
if random.random() < takeup_rate:
    predictor_engine.response(
        auth_runtime,
        {
            "uuid": row["uuid"],
            "offer_name": row.get("offer_name") or row.get("result", {}).get("offer"),
        },
    )
time.sleep(delay_seconds)

Do not call /responses (plural). After the loop, inspect the options store or run Interactions Analytic on the logging collections.

For Beta plots from Mongo alpha/beta (Ecosystem Rewards), filter the options store by contextual variables and overlay scipy.stats.beta.pdf — same idea as the Workbench plot output.

Related

• Deployments
• Runtime access — /invocations and /response
• Ecosystem Rewards — reading Beta plots
• Model convergence
• Interactions Analytic
• Runtime Console — single-call tests without a full simulation