Algorithms Overview

The ecosystem.Ai runtime includes a comprehensive library of dynamic interaction algorithms for real-time offer scoring and recommendation. Each algorithm implements a different strategy for learning which offers, actions, or content to present to users.

All algorithms share the same operational architecture: a background rolling process periodically updates offer statistics in the options store, and real-time scoring reads those pre-computed statistics to rank offers for each API request.

Architecture

Algorithm selection is driven by the randomisation object stored in the dynamic recommender configuration document in MongoDB:

{
  "randomisation": {
    "approach": "binaryThompson",
    "sub_approach": "",
    "epsilon": 0.0,
    "success_reward": 1.0,
    "fail_reward": 1.0,
    "processing_window": 86400000,
    "processing_count": 5000,
    "decay_gamma": 1.0,
    "interaction_count": 0
  }
}

The approach field selects the top-level algorithm. When approach is behaviorAlgos, the sub_approach field selects the specific behavioral economics algorithm.

Algorithm Routing

Behavioral Sub-Approaches

When approach = "behaviorAlgos", the sub_approach field selects one of:

Exploration

All algorithms share a deployment-level epsilon exploration mechanism. On each API request, the runtime rolls a random number against the configured epsilon. When exploration is triggered (explore = 1), the algorithm is bypassed entirely and all offers receive uniform random scores. This applies to every algorithm without exception.

In addition, some algorithms have their own built-in exploration that operates during normal (non-explore) scoring:

Algorithm Comparison

Cold-Start Behavior

Recommendations are always returned, regardless of algorithm or data availability. The RollingBehavior (and equivalent Rolling classes) always produces a scored options array that is passed to the post-score class:

• No history at all: Every offer in the options store receives a uniform random score.
• Partial history: Offers scored by the algorithm use their computed scores; unscored offers receive a random fallback score.
• Explore triggered: When deployment-level epsilon triggers exploration, all offers receive random scores regardless of available history.

The post-score class then controls the final offer selection, eligibility filtering, and response formatting. The table below documents what each algorithm contributes on top of this platform-level behavior.

Cold-Start Summary

Recommended Cold-Start Strategies

Configuring Priors for Thompson Sampling

The alpha_zero and beta_zero fields in the options store control the Thompson Sampling prior. Different priors can be set per offer to encode domain knowledge:

Rolling Process and Background Learning

Every dynamic interaction algorithm follows the same two-phase architecture:

1. Background learning (write path): A scheduled process periodically reads logging data (offers presented) and response data (offers accepted), computes updated statistics for each offer arm, and writes the results to the options store in MongoDB.

2. Real-time scoring (read path): When an API request arrives, the runtime reads the pre-computed arm statistics from the options store, applies the explore/exploit logic for the selected algorithm, and returns ranked offers. No database writes occur during scoring.

This separation means learning is decoupled from serving. The options store acts as the bridge.

Scheduling

Background learning is triggered in two ways:

Automatic Scheduler (recommended for production): The MultiCampaignScheduler runs on a Spring @Scheduled fixed-delay loop controlled by the monitoring.delay property (in seconds, default 60).

On-Demand via /learning API: Useful for forcing an immediate update after bulk data loads, testing, or manual intervention.

curl -X POST http://localhost:8091/learning

Options Store Update Cycle

Each background learning cycle performs these steps for every option:

1. Read current options from the options store
2. Aggregate logging data (presentations) — filtered by time window, count limit, decay
3. Aggregate response data (acceptances) within the same window
4. Compute arm_reward using the algorithm-specific reward strategy
5. Write updated options back to the options store via upsert

Process vs Score

For more detail on the options store structure, see Options Store. For the processing pipeline configuration, see Process.

Scenario Guide

Scenario 1: New Product Launch (Cold Start)

Problem: New catalog of offers with no interaction history.

Recommended: binaryThompson with default priors

{
  "approach": "binaryThompson",
  "epsilon": 0.0,
  "success_reward": 1.0,
  "fail_reward": 1.0,
  "processing_window": 86400000,
  "processing_count": 1000,
  "decay_gamma": 1.0
}

Scenario 2: Mature Catalog with Popularity Bias

Problem: Top offers get disproportionate impressions. Long-tail offers are never shown.

Recommended: behaviorAlgos with coverageAwareThompson

{
  "approach": "behaviorAlgos",
  "sub_approach": "coverageAwareThompson",
  "epsilon": 0.05,
  "processing_count": 5000,
  "decay_gamma": 1.0
}

Scenario 3: High Cost of Rejection

Problem: Showing irrelevant offers damages customer trust (financial products, insurance).

Recommended: behaviorAlgos with lossAversion

{
  "approach": "behaviorAlgos",
  "sub_approach": "lossAversion",
  "processing_count": 5000,
  "processing_window": 604800000,
  "decay_gamma": 1.0
}

Scenario 4: Conservative / Regulated Industry

Problem: Consistency and predictability matter more than maximizing acceptance rate.

Recommended: behaviorAlgos with riskAversion

{
  "approach": "behaviorAlgos",
  "sub_approach": "riskAversion",
  "processing_count": 5000,
  "processing_window": 2592000000,
  "decay_gamma": 1.0
}

Scenario 5: Simple A/B Testing

Problem: Basic performance tracking with minimal complexity.

Recommended: epsilonGreedy with epsilon between 0.05 and 0.20

{
  "approach": "epsilonGreedy",
  "epsilon": 0.1,
  "processing_count": 1000,
  "processing_window": 86400000,
  "decay_gamma": 1.0
}

Scenario 6: Sequential Customer Journey

Problem: Optimal next offer depends on what the customer has already seen/accepted.

Recommended: QLearning

{
  "approach": "QLearning",
  "learning_rate": 0.25,
  "discount_factor": 0.75,
  "random_action": 0.2,
  "max_reward": 10,
  "processing_count": 5000,
  "training_data_source": "logging"
}

Scenario 7: Marketing Campaign with Behavioral Nudging

Problem: Recommendations should be informed by how humans actually make decisions.

Recommended: behaviorAlgos with prospectTheory

{
  "approach": "behaviorAlgos",
  "sub_approach": "prospectTheory",
  "epsilon": 0.1,
  "processing_count": 5000,
  "processing_window": 604800000,
  "decay_gamma": 1.0
}

Scenario 8: Engagement Fatigue Analysis

Problem: Determine optimal frequency and intensity of customer engagement.

Recommended: behaviorAlgos with sentimentalEquilibrium

{
  "approach": "behaviorAlgos",
  "sub_approach": "sentimentalEquilibrium",
  "processing_count": 5000,
  "decay_gamma": 1.0
}

Configuration Reference

Common Parameters

Processing Window Examples

Reward Strategy Factory

The RewardStrategyFactory maps the approach to a reward computation:

For custom formulas, see Custom Reward Functions.