Simulations

Simulations exercise a Deployment before production by driving the Runtime closed loop:

1. POST /invocations for a sampled customer (ranked offers in final_result).
2. Simulated take-up (or none).
3. On accept, POST /response (singular) with that row’s uuid and offer_name.

params on /invocations is a JSON string, never a nested object. Use a stringified empty object, or a stringified input / value arrays object.

Two ways to run that loop:

Workbench runs the loop in-process on the Workbench FastAPI backend (Motor Mongo + async Runtime HTTP). Python is more flexible for custom scripts. Both still need a pushed Dynamic Interaction deployment and a reachable Runtime.

Workbench 2

Open Simulation at /entities/simulation (not the old Laboratory / Jupyter notebook UI).

Identity

Each simulation document has a name, project, deployment, and deployment version. New simulations inherit the header project, then the first deployment/version when those are empty.

Save is never blocked on those fields. Empty names become New simulation / New simulation N. Duplicate explicit names still conflict.

Run Simulation stays disabled until setup is complete. Blockers shown on the Setup card:

• Simulation name
• Project, deployment, and version
• MongoDB available (Workbench database)

The learning algorithm is not chosen on the simulation form. It is resolved from the Dynamic Engagement linked to the deployment. If none is linked, the Runtime scores naively.

Runs

A simulation can have many runs. All run settings (configuration, scenario, customer source, connections, advanced, logs, results) sit under the selected run.

Add Run clones the first run’s configuration, with a new UUID and RunN name so logs and plots stay isolated.

Default new-run values (override per run):

Server URL, Notebooks URL, and ecosystem-server credentials are not required (legacy fields on old documents are ignored).

Run configuration

Offers & algorithm

• Number of offers — numberoffers sent to /invocations.
• Offer selection (among presented) — which ranked offer is credited on take-up:
  • top — highest scoring
  • second — always rank 2
  • random — uniform among the Runtime slate
  • distribution — score-weighted
• Algorithm — read-only, from the deployment’s Dynamic Engagement.

Iterations & take-up

• Number of iterations — how many /invocations calls.
• Default take-up rate — Bernoulli accept probability unless historical or policy modes override it.
• Reward binary threshold — used when the deployment’s reward path needs a binary outcome.

Connections

Open the Connections accordion. Simulations call the Client pulse responder (Runtime) from the Workbench backend.

Click Validate (POST /simulations/validate-connections). The Workbench backend checks:

If the pulse responder cannot resolve ecosystem-runtime, use a host the Workbench API can reach (for example http://localhost:8099 or http://localhost:8091).

Describe scenario (optional)

Natural language → reviewable ScenarioSpec. Failures stay in this panel and never block Save or Run.

1. Describe peaks, take-up drift, or random uptake among presented offers (for example telco weekday peaks 6–8am and 3–7pm).
2. Analyze — intents traffic, take_up, selection.
3. Apply to run — writes the patch into this run only after you confirm.

If the scenario cannot be expressed as config, Generate custom Python produces a script you copy and run yourself. Workbench Run does not execute that script.

Apply can set:

• Traffic profile — virtual day/week compressed into iterations (weekday peaks 6–8 and 15–19 by default). Does not wait for wall-clock hours.
• Take-up policy — flat, historical, linear_drift, cyclical, rank_decline, offer_decline.
• Selection policy — static (use Offer Selection) or drift_lower_sequence (acceptance moves toward lower ranks over the run).

You can also set those under Advanced without using Analyze.

Customer source (optional)

By default, customers come from the deployment Parameter Access lookup.

Enable Use alternate customer source to sample /invocations customers from another Mongo database/collection/lookup field, with an optional aggregation pipeline (then the engine adds $sample). Example lookup field customer; pipeline [{"$match":{"region":"ZA"}}].

If the lookup key does not match the collection, Run fails with a hint to fix Parameter Access or the alternate source.

Advanced

Run, logs, results, Explain

1. Save the simulation (drafts are allowed).
2. Validate connections.
3. Run Simulation — Workbench starts an in-process asyncio engine. Progress shows in the run banner. Cancel cooperatively stops the loop.
4. Logs — GET /simulations/logs?run_uuid=... (Mongo simulations.simulation_logs). Success versus failed invocations is charted when those counters appear.
5. Results — options-store parameter snapshots (simulation_param_outputs) and Beta / plot series (simulation_plot_outputs) when those outputs are enabled.
6. Explain — on the Results panel, Explain produces a read-only overview (purpose, process, interpreted results) via the Workbench LLM. Versions are stored on the simulation document (max 10). The Explanations table at the bottom of the editor opens a saved version without a new model call.

Failures return a message and a hint (unreachable Runtime, empty customer collection, bad customer pipeline, unsaved run uuid).

Workbench APIs (operator)

Thin UI; processing stays on the backend:

• GET /simulations/status — Mongo health (engine is in-process)
• POST /simulations/validate-connections
• POST /simulations/run / cancel
• GET /simulations/logs, DELETE /simulations/logs
• GET /simulations/results/parameters, /results/options, /results/plot
• Explain jobs: kind=simulation_result on the agents explain API
• Scenario analyze/apply via the agents router (optional)
• MCP: start_simulation_run / cancel_simulation_run (confirm), status, logs, results, explain_simulation_result

Python simulations

Use the Python package (prediction.apis.prediction_engine) when you need a take-up rule Workbench cannot express. Keep the same closed loop.

import random
import time
 
empty_params = "{}"  # params must be a JSON string
invocations_body = {
    "campaign": deployment_id,
    "subcampaign": "simulation",
    "channel": "simulation",
    "customer": customer_id,
    "userid": "simulation",
    "numberoffers": 1,
    "params": empty_params,  # JSON string, e.g. "{}"
}
offer_response = predictor_engine.invocations(auth_runtime, invocations_body)
row = offer_response["final_result"][0]
# Ranked offers: parse offer_name from the row (field names follow the Runtime result)
 
if random.random() < takeup_rate:
    predictor_engine.response(
        auth_runtime,
        {
            "uuid": row["uuid"],
            "offer_name": row.get("offer_name") or row.get("result", {}).get("offer"),
        },
    )
time.sleep(delay_seconds)

Do not call /responses (plural). After the loop, inspect the options store or run Interactions Analytic on the logging collections.

For Beta plots from Mongo alpha/beta (Ecosystem Rewards), filter the options store by contextual variables and overlay scipy.stats.beta.pdf — same idea as the Workbench plot output.

Related

• Deployments
• Runtime access — /invocations and /response
• Ecosystem Rewards — reading Beta plots
• Model convergence
• Interactions Analytic
• Runtime Console — single-call tests without a full simulation