Author’s note: This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.

Stuck behind a firewall. Read the article for free HERE

Running a walk-in urgent care clinic comes with a recurring operational headache of figuring out how many clinicians to roster on any given day. When they are too few, then chances are that the waiting room backs up and patients drift to the next nearest Emergency Department (ED). On the contrary, then you’re paying for doctor hours that you don’t need and vice versa. Most clinics base their patient numbers on a mix of last year's numbers, gut feel, and a bit of guesswork, especially around flu season. That’s the problem I wanted to tackle in this article, where I turn urgent care demand into something that can actually be forecasted, day-by-day, with a confidence interval that a rostering manager can plan against.

Forecasting urgent care demand is not the same as forecasting foot traffic at a shopping center. For example, if a mall misses the mark by 50 customers on a Tuesday, there is a high chance that nothing much will happen. Maybe the food court will just be overstaffed or the budget takes a small hit. However, in urgent care, the cost of being wrong is heavier. Understaffing means longer waits for people who may be genuinely unwell. On the contrary, overstaffing creates the opposite problem where it burns money that thin-margin clinics might not have. So this isn’t just a neat forecasting exercise but an operational problem with real consequences.

The clinic I’m building this around is a walk-in urgent care in Melbourne, and the city, as always, is a good fit for the exercise, and hey, I live here too. Melbourne’s health demand drivers are unusually rich, from Victorian public holidays (where EDs typically receive higher numbers as GPs will not be at work) to the Southern Hemisphere winter flu season. The city also experiences the October–November hay fever window and odd freak events like the 2016 thunderstorm-asthma that overwhelmed emergency services across the city.

I’ve split the article into two parts. Part 1 (this one) covers synthetic data generation, feature engineering, model training, and evaluation. Part 2 will cover deployment with a FastAPI backend serving predictions over REST, a React + Tailwind dashboard for the rostering team, and a Supabase layer for logging predictions against real outcomes. As always, all code is open-sourced HERE. Clone it and adapt for your own clinic or venue.

1. The Problem and Why Prediction Matters

Walk-in urgent care clinics sit awkwardly between general practice and emergency departments. They handle the complaints that can’t wait for a three-day GP appointment but don’t quite need an ambulance, i.e., sprains, cuts, chest infections, children’s fevers, minor fractures, post-injury follow-ups, etc. This makes their demand erratic. For example, on a typical Tuesday they might see 90 patients, whereas numbers can hit 200 on Easter Monday, half of them being people whose regular GP was closed for the holiday.

A properly thought-through predictive model quantifies this upfront. It should be able to output a number, a confidence interval, and a staffing tier that the rostering manager can actually rely and act on. With such a model in place, the question shifts from “How many patients will walk through the door?” to “Do we need the Low, Medium, or High clinician roster today?” Let's walk through what I built and how I did it.

2. Dataset Design & Simulation

I didn’t have access to real clinic records (and wouldn’t publish them if I did). So I generated a synthetic dataset that reflects realistic urgent care patterns in Melbourne. This is useful not just because of privacy but because it lets you control the signal-to-noise ratio and demonstrate the methodology cleanly before the approach gets applied to messy real-world data.

The dataset spans January 2023 to December 2025, giving 1,096 daily records.

The light line shows raw daily patient counts, while the red line shows the 30-day moving average. The smoothed line makes the synthetic demand pattern easier to read: flu-driven winter peaks, a softer spring rise from hay fever, quieter summer months, and occasional spikes from thunderstorm asthma events or heatwaves.

Each record captures the following:

• Temporal features: day of week, month, quarter, week of year
• Weather: temperature (Melbourne-specific ranges), precipitation (mm), relative humidity, weather type (Sunny / Partly Cloudy / Cloudy / Rainy)
• Calendar events: Victorian public holidays, Victorian school holidays, and crucially, the day after a public holiday
• The feature set also captures key local health and weather drivers: winter flu season with a July peak, hay fever season from October to November, rare thunderstorm-asthma events, and temperature-extreme flags for days above 35°C or below 5°C.

It's worth pointing out that several of these drivers have the opposite effect of what they’d be in other contexts. For example, rain might reduce visits to entertainment joints but slightly increases urgent care demand (slips, falls, and people who can’t put off the chest infection any longer). I spent some good time on these sign choices because getting them wrong would produce a plausible-looking dataset that models can be trained on, but the learned relationships would be backwards. This is exactly how a seemingly validated model ends up giving bad rostering advice in production.

Left: mean daily patients by weather type. Rainy days run slightly above sunny days. Middle: patients vs. temperature. Right: pollen index vs. patients, flat until index hits ~7, then climbs sharply (the hayfever-to-asthma pathway).

Above are four views of the same temporal story.

The plots suggest a strong calendar effect. Mondays are the busiest day, sitting about 15% above midweek levels. July and August are the peak months, which lines up with winter pressure, and Q3 has the highest quarterly demand. In the heatmap, Monday in July is the busiest day-month combination.

The patient count itself is generated using multiplicative factors applied to a base of 100 patients/day:

# Day of week effect (Monday is peak due to weekend GP backlog)
day_multiplier = {0: 1.15, 1: 1.00, 2: 0.95, 3: 0.95, 4: 1.00, 5: 0.85, 6: 0.75}
df['patients'] *= df['day_of_week'].map(day_multiplier)
# Seasonal effect (Southern Hemisphere - winter = flu season)
season_multiplier = {
1: 0.85, 2: 0.90, 3: 0.95, 4: 1.00, 5: 1.10, 6: 1.30,
7: 1.45, 8: 1.35, 9: 1.15, 10: 1.10, 11: 1.05, 12: 0.90
}
df['patients'] *= df['month'].map(season_multiplier)
# Calendar effects - NOTE: public holidays INCREASE demand here (GPs closed)
df.loc[df['is_public_holiday'] == 1, 'patients'] *= 1.40
df.loc[df['is_day_after_public_holiday'] == 1, 'patients'] *= 1.22
df.loc[df['is_school_holiday'] == 1, 'patients'] *= 1.15
# Epidemiological drivers
df.loc[df['is_flu_peak'] == 1, 'patients'] *= 1.50
df.loc[df['is_hayfever_season'] == 1, 'patients'] *= 1.05
df.loc[df['is_thunderstorm_asthma'] == 1, 'patients'] *= 2.30
# Weather impact (different sign from the museum case)
df.loc[df['weather_type'] == 'Rainy', 'weather_factor'] = 1.08
df.loc[df['temp_extreme_hot'] == 1, 'weather_factor'] *= 1.25
df.loc[df['temp_extreme_cold'] == 1, 'weather_factor'] *= 1.20
df['patients'] *= df['weather_factor']

Each factor has a measurable and verifiable uplift in the final data as quantified in the chart below.

Each panel compares mean patient count with and without the factor active. Flu peak (+54%), public holidays (+40%), and day-after-public-holiday (+22%) dominate. Hayfever season has a small but consistent lift (+5%). By the way, not every hay fever day leads to asthma, but across the full two-month window, the effect stacks up.

To make the synthetic demand less perfectly behaved, I added Gaussian noise with a standard deviation equal to 12% of expected daily traffic, a 2% outlier rate where traffic spikes to 1.5–2.5× normal levels, and a small daily growth trend of about 0.01% to reflect a slowly expanding catchment population. The final values are clipped between 30 and 500 patients/day, which is a realistic operating range for a single walk-in clinic. Remember, this is simulated data and should NOT be used as a replacement for real clinic records.

The resulting distribution has the right-skewed shape typical of event-driven demand. Most days cluster around 100–150 patients, with a long tail of flu-season weekdays and public-holiday spikes that push above 250.

The histogram shows skewness ≈ 1.14, with the mean sitting above the median and a visible right tail of high-volume days.

3. Feature Engineering

I’ve made the same point in previous articles and will keep making it: raw data gets you baseline predictions while feature-engineered data gets you good ones. Starting from 31 raw columns, I engineered 87 features grouped into eight categories:

Feature groups:
  Temporal    :  17
  Calendar    :   3
  Season      :   4
  Weather     :  11
  Epidemio    :   5
  Interaction :   6
  Lag         :  14
  Rolling     :  27
  Total       :  87

Most of the techniques here are around cyclical encoding, lag features, and rolling stats, so I’ll briefly touch on those and linger on the ones that are specific to the urgent care domain.

a) Cyclical Encoding — Why Day 0 and Day 6 Should Be “Close”

Here’s the subtle problem with temporal features again. If you feed day_of_week directly into a model as values 0 to 6, then the model will treat Monday (0) and Sunday (6) as being far apart as much as they’re adjacent. The same applies to months. December (12) and January (1) are close and not opposites.

Cyclical encoding using sin/cos transforms is the best workaround for this:

df['dow_sin'] = np.sin(2 * np.pi * df['day_of_week']/7)
df['dow_cos'] = np.cos(2 * np.pi * df['day_of_week']/7)
df['month_sin'] = np.sin(2 * np.pi * (df['month']-1)/12)
df['month_cos'] = np.cos(2 * np.pi * (df['month']-1)/12)
df['doy'] = df['date'].dt.dayofyear
df['doy_sin'] = np.sin(2 * np.pi * df['doy']/365)
df['doy_cos'] = np.cos(2 * np.pi * df['doy']/365)

The best way to think around this is to assume that you’re placing each day on the face of a clock. Monday and Sunday end up right next to each other, and December sits beside January. Both sin and cos components are needed because sin alone can’t distinguish between two days that map to the same y-coordinate (e.g., Tuesday and Friday might have the same sin value but different cos values).

On the left, days of the week are mapped onto a circle via sin/cos. On the right, months on the same circular encoding. Both components are needed to uniquely identify each position on the circle.

I applied this to day-of-week, month, day-of-year, and week-of-year.

b) Lag Features—"Yesterday Predicts Tomorrow”

Recent history turned out to be one of the strongest predictive signals in this dataset. If 150 patients walked in yesterday, today’s number is more likely to sit near 150 than near 80. I replicated this in a way that intervals correspond to real rhythms: short-term momentum, weekly cycle, fortnightly, monthly, and seasonal.

for lag in [1, 2, 3, 7, 14, 21, 28, 60, 90]:
    df[f'patients_lag_{lag}'] = df['patients'].shift(lag)

# Same-weekday average over the last 4 occurrences
df['mean_last_4_same_dow'] = df[['lag_7_same_dow','lag_14_same_dow',
                                 'lag_21_same_dow','lag_28_same_dow']].mean(axis=1)

Why these specific intervals, if you may ask? Lags 1–3 capture short-term momentum. Lag 7 captures the same-day-last-week pattern (which matters enormously here, since Mondays and Sundays behave very differently). Lags 14, 21, and 28 capture fortnightly and monthly rhythms. Lags 60 and 90 capture seasonal trends. The same-weekday average is particularly powerful in urgent care because it answers the clinically grounded question, "How many patients typically walk in on a Monday?" which is a very different question from “How many typically walk in on a Sunday?" Mondays in this dataset run ~50% above Sundays.

The ACF shows significant autocorrelation at multiples of 7 days (the weekly cycle) and a slower decay that persists out past 60 days (the seasonal structure). The PACF highlights lags 1 and 7 as carrying the most independent predictive information, exactly the lags prioritised in the feature set.

c) Rolling Window Statistics

Beyond point lags, I computed rolling means, standard deviations, max, and min across multiple windows, plus exponentially weighted moving averages:

for window in [3, 7, 14, 30, 60, 90]:
    df[f'rolling_mean_{window}d'] = df['patients'].shift(1).rolling(window).mean()
    df[f'rolling_std_{window}d']  = df['patients'].shift(1).rolling(window).std()
    df[f'rolling_max_{window}d']  = df['patients'].shift(1).rolling(window).max()
    df[f'rolling_min_{window}d']  = df['patients'].shift(1).rolling(window).min()

#  Exponentially Weighted Moving Average(EWMA) — more weight on recent days
for span in [7, 14, 30]:
    df[f'ewma_{span}d'] = df['patients'].shift(1).ewm(span=span, adjust=False).mean()

The shift(1) is critical as it prevents data leakage by ensuring that we only use information available before the prediction date. If you get this wrong, then you’ll have leaked future data into training, and test metrics will look amazing until the model hits production and falls apart. The EWMA variants give extra weight to recent observations, which is useful because patterns can shift gradually (think of the ramp-up at the start of flu season).

d) Interaction Features

In most cases, urgent care demand is driven by combinations and not individual factors. A rainy Monday during flu peak is a fundamentally different day from a sunny Tuesday in February, even if you break down each feature independently. I captured the most clinically plausible combinations as follows, and of course this can be extended to your liking:

df['ph_x_monday']=df['is_public_holiday']*df['is_monday']
df['flu_peak_x_rainy']= df['is_flu_peak']*(df['weather_type']=='Rainy').astype(int)
df['hayfever_x_high_pollen']= df['is_hayfever_season']*(df['pollen_index']>=7).astype(int)
df['school_holiday_x_weekend']= df['is_school_holiday']*df['is_weekend']
df['extreme_cold_x_flu_season']= df['temp_extreme_cold']*df['is_flu_season']
df['day_after_ph_x_monday']= df['is_day_after_public_holiday']*df['is_monday']

# Count of concurrent illness drivers
df['illness_driver_count']=(df['is_flu_season']+df['is_hayfever_season']
                              +df['is_day_after_public_holiday']
                              +df['temp_extreme_hot']+df['temp_extreme_cold'])

The illness_driver_count feature captures a simple idea, which is how many illness-related factors are active on a given day. For example, a day with three active drivers is fundamentally different from a day with zero, and the stacking is nearly monotonic.

The plot above shows a clear stacking effect. Days with no illness drivers average about 111 patients across 605 days, while those with one driver average about 153 patients across 475 days. Days with two concurrent drivers are rare, appearing only 16 times in the three-year window, but they average about 175 patients.

e) Melbourne Seasons

This matters because Melbourne’s seasonal pattern is flipped relative to Northern Hemisphere cities. December is summer, not winter, and if the model gets that wrong, its seasonal assumptions will definitelypoint in the wrong direction.

def get_season(month):
    if month in [12, 1, 2]: return 'Summer'
    if month in [3, 4, 5]:  return 'Autumn'
    if month in [6, 7, 8]:  return 'Winter'
    return 'Spring'

In urgent care, seasonal alignment matters because Melbourne’s winter months, June to August, map directly to flu season. If the model encodes seasonality incorrectly, then the flu-related demand signals get diluted across the wrong part of the calendar.

f) Log-Transform of the Target

Patient counts as in the original data are right-skewed (skewness ≈ 1.14), and log1pbrings the distribution much closer to normal, which helps gradient-based models converge and stops them from overweighting outlier days at the expense of typical ones:

df['patients_log'] = np.log1p(df['patients'])

The left plot shows the raw target distribution, with a skewness of 1.14 and a clear right tail from high-volume days. After applying log1p, skewness drops to 0.30, and the distribution becomes much closer to normal. The models train on this transformed target, then predictions are converted back to patient counts using np.expm1() before being returned. I’ll revisit this in Part 2 when discussing the production API.

With all 87 features engineered, the next step was to see which ones actually carried signal against the target.

The rolling 7/14-day averages cluster at the top (correlations between 0.61 and about 0.69), followed by the ‘rolling_mean7d’.

The longest lag feature looked back 90 days, so the first 90 rows had to be removed after feature engineering. That left about 1,006 usable observations, enough for model training and a separate 90-day holdout set.

4. Model Training & Comparison

With the features ready, I trained and compared four modeling approaches. Before that, I needed to get the split strategy right.

Train/Test Split — Respecting Time

With time series data, random train/test splits do not really work well. If you shuffle the dates, the model can learn from future patterns while being evaluated on earlier ones. That is data leakage, and it makes the metrics look better than they really are.

I kept the split chronological, where the final 90 days became the holdout test set, and everything before that was used for training.

Train: 915 rows (2023–04–01 → 2025–10–02)
Test : 90 rows (2025–10–03 → 2025–12–31)

For cross-validation, I used TimeSeriesSplit with five expanding-window folds. Each fold trained only on past data and validated on future data, keeping the evaluation aligned with how the model would be used in practice.

The Models

Before reaching for more sophisticated algorithms, I started with naive baselines. These are simple prediction strategies that rely on arithmetic rather than learned patterns. If a more complex model cannot beat them, then it's not earning its complexity.

I used three naive baselines, each slightly more informed than the last.

• Global mean: This predicts every day as the training set average, which was about 130 patients. It ignores everything, including day of week, weather, flu season, and public holidays. It is the “shrug” prediction, but it anchors the lower end of the performance range.
• Last week’s count (lag 7): This predicts today’s patient volume using the count from the same day last week. It captures a weekly rhythm, but nothing else. That means it cannot adapt to flu peaks, weather changes, school holidays, or public holidays.
• 7-day moving average: This predicts today’s volume as the average of the previous seven days. It is smoother than the lag-7 baseline, but still blind to why demand is changing. It follows recent drift, but it does not understand the drivers behind it.

The shared limitation of all three is that they use a single signal and can’t combine information. A sunny Saturday during school holidays with a thunderstorm-asthma alert is nothing like a rainy Tuesday in summer, and none of these baselines can tell the difference. That’s the gap ML fills.

I used three modeling algorithms for this problem.

1. Random Forest used 300 decision trees, with each tree trained on a random slice of the data and a random subset of features using max_features='sqrt'. Individual trees can learn rules such as "If it is Monday, flu activity is high, and temperature is below 10°C, expect higher patient volume.” The final prediction averages the output from all 300 trees, which smooths out the quirks of any single tree. I also set min_samples_leaf=5 to stop the trees from memorising small pockets of noise. Like the other tree-based models, Random Forest was trained on the log-transformed target.
2. XGBoost works differently. Instead of growing trees independently, it builds them in sequence. Each new tree focuses on the errors left behind by the previous trees. In simple terms, Random Forest averages many independent trees, while XGBoost builds trees one after another, with each tree chasing the remaining residuals. I used 500 trees and a learning rate of 0.05, meaning each tree only made a small correction. That usually requires more trees, but it produces a more stable model. I trained XGBoost on log1p(patients) so that high-volume days did not dominate the loss function.
3. Prophet (Facebook’s time series model) takes a different angle. Instead of relying on the hand-built features, it decomposes the time series into trend, seasonality, and holiday effects, then fits those components directly. I used multiplicative seasonality so the seasonal swings could scale with the baseline level of demand. I also added external regressors for public holidays, flu peak, pollen index, and weather.

Here is the XGBoost training block.

xgb_model = xgb.XGBRegressor(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=6,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=5,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    n_jobs=-1,
    verbosity=0,
)
# Train on log-transformed target
xgb_model.fit(X_train, y_train_log)
# Predictions need inverse transform
xgb_preds = np.expm1(xgb_model.predict(X_test))

The code for the other two approaches can be found HERE.

Results

XGBoost produced the best results across all metrics. Its MAPE of 11.89% means that, on an average day of about 130 patients, the forecast error is roughly 15 patients. The point is not that the model predicts every day exactly, but that it provides a reliable enough signal to support staffing decisions.

| Model                | MAE   | RMSE  | R²    | MAPE   |
|----------------------|-------|-------|-------|--------|
| XGBoost              | 17.04 | 27.69 |  0.30 | 11.89% |
| Prophet              | 17.98 | 28.98 |  0.23 | 12.84% |
| Random Forest        | 18.34 | 29.35 |  0.21 | 12.91% |
| Baseline (7d MA)     | 22.93 | 32.36 |  0.04 | 17.18% |
| Baseline (mean)      | 24.04 | 33.04 | -0.00 | 18.06% |
| Baseline (last week) | 24.09 | 35.87 | -0.18 | 17.22% |

The improvement over baselines is substantial. The best baseline (7-day moving average) has an MAE of 22.93 patients, while XGBoost achieves 17.04, a 26% reduction in error. MAPE drops from 17.2% to 11.9%, where this gap is the difference between useful predictions and educated guesses.

Two things worth calling out on these numbers:

• The negative R² for the last-week baseline is not a bug. It just means that baseline actually performs worse than predicting the constant mean on this test set. Baselines aren’t guaranteed to beat constant-mean, and that’s part of why they’re useful as they calibrate expectations.
• An R² of about 0.30 looks modest, but that is partly a signal-to-noise issue. Daily urgent care volumes have a lot of short-term variation that calendar, weather, and epidemiological features cannot fully explain. MAPE is more useful here because it measures the typical percentage error in the forecast. At 11.9%, it gives a clearer view of whether the model is useful for planning staff.

On the test data, actuals vs predicted numbers are as below:

XGBoost tracks the observed series most closely, capturing both the weekly cycle and the late-year flu ramp. Random Forest follows the overall shape but underestimates peak demand. Prophet captures the broad seasonal movement but smooths over short-term variation because it relies more on decomposed trend and seasonality than on the engineered feature set used by the tree-based models.

5. Model Evaluation & SHAP Explainability

Raw metrics tell you how well the model performs while evaluation tells you where it struggles and why, and that’s the part that actually matters when you’re deciding whether to trust it.

Residual Analysis

Overall, the residuals look sensible. They are roughly centred around zero, with no obvious time-based drift across the holdout period. The predicted-versus-actual plot tracks the diagonal reasonably well, although the model still under-predicts some of the highest-volume days. Most percentage errors fall within about ±20%, with a slight negative skew.

The model has a small negative bias of about 6.5 patients per day, meaning it tends to under-predict demand slightly, which is fundamental in rostering. Understaffing is riskier than modest overstaffing, so the production pipeline should not treat the point forecast as the final staffing number. A safer approach is to roster closer to the upper bound of the 80% prediction interval.

SHapley Additive exPlanations (SHAP) Explainability

I used SHAP to understand what was driving the model’s predictions. This mattered for two reasons. First, it helped check whether the model was relying on clinically sensible signals rather than spurious patterns. Second, it gave me a concrete way to explain the forecasts to clinical directors and rostering managers when they ask why the model expects demand to rise or fall.

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_shap)
shap.summary_plot(shap_values, X_shap, plot_type='dot', max_display=20)

The top features line up with what a clinician would expect:

The figure above is a SHAP beeswarm plot, showing how each feature pushes predictions up or down across the holdout set. Each dot represents one observation for that feature, coloured by the feature value, with red indicating higher values and blue indicating lower values. is_flu_peak, illness_driver_count, and patients_lag_7 dominate the top of the ranking, suggesting the model relies heavily on epidemiological pressure and recent demand. Lag and rolling-window features also contribute meaningfully, but they sit further down the ranking.

I also generated a waterfall plot for the single worst prediction day. This is the kind of plot you want ready when a clinical director asks, “Why did the model get yesterday so wrong?” It breaks the forecast into feature-level contributions, showing which signals pushed the prediction up or down and by how much.

That matters because not every bad prediction has the same cause. Sometimes the model misses because something happened outside the feature set, such as a nearby GP closure, a road accident, or another local disruption. Other times, the model may be facing a rare combination of feature values it has not seen often enough during training. The waterfall plot helps separate those two cases.

Prediction Intervals

A point prediction alone isn’t enough. Rostering managers need to know the range of uncertainty, which is the reason I computed bootstrap prediction intervals using training residuals:

np.random.seed(42)
n_bootstrap = 500
base_pred = test['pred'].values
bootstrap_preds = np.array([
    base_pred + np.random.choice(cv_resid, size=len(base_pred), replace=True)
    for _ in range(n_bootstrap)
])
lower_80 = np.percentile(bootstrap_preds, 10, axis=0)
upper_80 = np.percentile(bootstrap_preds, 90, axis=0)
lower_95 = np.percentile(bootstrap_preds, 2.5, axis=0)
upper_95 = np.percentile(bootstrap_preds, 97.5, axis=0)

The red line shows the model’s prediction, while the black line shows the actual patient count. The darker band represents the 80% prediction interval, and the lighter band represents the 95% interval. The actual values fall within these bands at roughly the expected rates, which suggests the uncertainty estimates are reasonably well calibrated.

For a rostering manager, this is more useful than a single number. Instead of saying, “We predict 145 patients tomorrow,” the model can say, “We predict 145 patients tomorrow, with an 80% prediction interval of 113 to 177.” That gives the manager a planning range rather than a false sense of precision.

There is one caveat, though. This bootstrap method assumes the residuals are drawn from the same error distribution across all conditions, which is unlikely to be true. Public holidays, flu peaks, and unusual demand days probably have wider errors than quiet weekdays. A more refined version would estimate condition-specific intervals, but for a first deployment, this gives a useful and honest uncertainty range.

6. Stakeholders Engagement

From experience, MAE and RMSE do not mean much to clinical directors or rostering managers on their own. They are useful modeling metrics, but they do not directly answer the operational question: How many people should we roster tomorrow?

So I framed the model output in two more practical ways.

Staffing Tier Classification

Instead of only returning a raw patient-count forecast, the model can also classify each day into a staffing tier that maps directly to roster planning. The exact thresholds should be defined by the clinical operations team, but a simple example could look like this:

Low (<80 patients): 2 doctors, 3 nurses, standard roster 
Medium (80–150): 3 doctors, 4 nurses, extra triage nurse 
High (>150): 4+ doctors, surge protocol, ED overflow coordination

This makes the forecast easier to act on. Each tier has a concrete staffing implication, a cost implication, and a clinical risk implication. The rostering manager no longer has to interpret patient-count deltas in isolation. They can use the model output as a planning signal tied directly to staffing decisions.

Operational Impact

The real value is not just lower MAE. It is better roster decisions. If the model gets the demand tier right, the clinic avoids wasting money on unnecessary staff during quiet days and reduces the risk of understaffing during demand spikes.

That is the version a clinical director can use. Instead of saying, “The model has an MAE of 17 patients,” the better message is, "We can identify the correct staffing tier about 8 out of 10 days, and we can give you an 80% prediction interval before the roster is locked in.”

That’s Part 1 done, and I’m sorry to admit that I ran it long. I promise (just like I’ve done in the past), Part 2 will be way shorter and more intuitive. We’ve gone from breaking down the business problem through data generation, feature engineering, model training, evaluation, and stakeholder framing. The XGBoost model with 87 engineered features hits a MAPE of 11.9% and produces well-calibrated prediction intervals that can be used for rostering decisions. I’ve basically provided for you a framework that you can plug your code and data into and forecast results.

In Part 2, I’ll take this trained model and deploy it as a full-stack application: a FastAPI backend serving predictions through a REST API, a React + Tailwind dashboard for stakeholders, and possibly a Supabase database for logging predictions against actuals so we can backfill real outcomes and monitor drift. I’ll also cover deployment considerations and what I’d do differently with real clinic data. If you’d like an article on optimising predicted numbers to actual staffing levels, then let me know.

As always, all code is open-sourced HERE. Feel free to clone it and adapt it for your own use case. If you found this useful, please clap, leave a comment, or share it with someone working on forecasting or operational AI. You can always find my other articles on my profile, and I’m always happy to connect via LinkedIn.

Predicting Daily Patient Volume for a Melbourne Urgent Care Clinic was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stuck behind a firewall? Read the article for FREE here.

In Part 1, I covered what agentic AI actually means and why a SIM card replacement workflow is a good way to test whether an application is genuinely agentic. I introduced the four building blocks that every agent should ideally have: state, tools, nodes, and edges. If you haven’t read the first one yet, I’d suggest starting there because this article builds directly on it.

The goal in this part is to translate the theoretical concepts in Part 1 to a practical and working agentic prototype. I’ll stick to LangGraph , an orchestration framework to build, manage and deploy stateful AI agents. The good thing with LangGraph is that it allows for better design and customisation of sub-agents and related artefacts. My hope is that you’ll have a good and practical grasp of agentic AI through a working agentic application that can run the full SIM swap flow for a telcom company. This ranges from identity verification all the way to customer notification about the final swap. The representative flowchart, just like what is in Part 1 is below:

As always, all the code is open-source and can be found here, so you can clone and run it yourself.

So let’s get into the interesting bits : -

1. The State Object

Remember the notebook analogy from Part 1? State is what the agent carries from step to step which is everything it knows at any given moment. In LangGraph, state can be defined as a Python TypedDict.

The choice of TypedDictover a class is deliberate as it serialises cleanly, it’s straightforward to checkpoint and restore, and LangGraph can easily track incremental state changes without any custom logic. Here’s the full state for the SIM swap agent:

from typing import Annotated, TypedDict
from langgraph.graph.message import add_messages

class SimSwapState(TypedDict):
    # Subscriber context
    subscriber_id: str
    subscriber_name: str
    msisdn: str
    alternate_msisdn: str
    id_number: str

    # Request context
    request_id: str
    request_reason: str  # "lost" | "stolen" | "damaged" | "upgrade"
    request_timestamp: str
    channel: str    # "shop" | "call_centre" | "ussd"

    # Identity verification - Knowledge-Based Authentication(KBA)
    id_verification_status: str  # "pending" | "passed" | "failed"
    id_verification_attempts: int
    id_max_retries: int
    kba_questions_correct: int

    # Line status checks
    line_status: str  # "active" | "suspended" | "barred" | "churned" | "not_found"
    fraud_flag: bool
    pending_swap: bool
    line_check_passed: bool

    # New SIM capture
    new_iccid: str
    new_imsi: str
    sim_validation_status: str # "pending" | "valid" | "invalid"
    sim_validation_attempts: int
    sim_max_retries: int
    sim_validation_errors: list[str]

    # Current (to-be-blocked) SIM
    old_iccid: str
    old_imsi: str
    old_sim_blocked: bool

    # Provisioning
    new_sim_provisioned: bool

    # M-PESA
    mpesa_registered: bool
    mpesa_balance: float
    mpesa_reactivated: bool
    mpesa_temporary_pin_sent: bool

    # Network
    network_activated: bool

    # Notifications
    confirmation_sms_sent: bool

    # Process metadata
    current_step: str
    audit_log: list[dict]
    error_messages: list[str]
    process_status: str  # "in_progress" | "completed" | "rejected" | "halted"

    # Conversation
    messages: Annotated[list, add_messages]

That looks like a lot of fields, but if you read it in groups like in the above code, then it will be easy to make sense of. The fields in the subscriber context are the details that a customer would need to have before they are served. Their MSISDN (phone number), ID number, and an alternate number to send the final confirmation SMS to etc. Thereafter, the request context has fields that capture the reason for the interaction with the system.

From there, every block maps directly to a stage in the workflow. KBA tracking holds the attempt counts, while the line status checks whether fraud or a pending swap was found. The SIM capture block tracks validation retries with pipeline fields (blocked, provisioned, M-PESA, network) flipping from False to True as each step completes. The metadata block at the very end gives the audit log, error accumulation, and an overall process status.

Most state fields follow a simple last-write-wins rule. If a node returns a new value for fraud_flag, the old one is overwritten. The exception field is messages, which uses Annotated[list, add_messages] to tell LangGraph to merge new messages into the existing list rather than replace it. That's how the agent accumulates a full conversation history as it moves through the steps.

One thing worth noting is that the graph itself is stateless as it has no memory between runs. Basically, the state object carries everything, including every business rule in the workflow, from retry limits to fraud flags to M-PESA reactivation checks.

2. Tools

If the state object is the notebook, the tools are what the agent uses to fill it in. In this implementation, each tool gets its own service classand they are all defined here:

| Tool                          | What it simulates 
|-------------------------------|---------------------------------------------|
| IdentityVerificationService   | KBA challenge: ID number, frequently called numbers, recent M-PESA transaction, last top-up amount 
| LineStatusService             | HLR lookup: line status (active/suspended/barred) + fraud flag + pending swap check 
| SimValidationService          | Validates ICCID format (19–20 digits, `89254…` Kenya prefix) and IMSI uniqueness 
| SimManagementService          | Blocks the old SIM, provisions the new one (binds IMSI to MSISDN in the HLR) 
| MpesaService                  | Restores the M-PESA wallet on the new SIM and resets the PIN 
| NetworkActivationService      | Activates voice, SMS, data, and USSD on the new SIM 
| NotificationService           | Sends the confirmation SMS to the alternate number 

I mocked all seven tools in this example. They use configurable success rates and a short asyncio.sleep to simulate real-world latency. Here's what IdentityVerificationService looks like:

class IdentityVerificationService:
    def __init__(self, success_rate: float | None = None):
        self.success_rate = (
            success_rate if success_rate is not None
            else config.IDENTITY_VERIFICATION_SUCCESS_RATE
        )

    async def verify(self, subscriber_id: str) -> dict:
        await asyncio.sleep(random.uniform(0.6, 1.4))

        questions = ["id_number", "frequently_called", "recent_mpesa", "last_topup"]
        per_question: dict[str, bool] = {
            q: random.random() < self.success_rate for q in questions
        }
        correct = sum(per_question.values())
        passed = correct >= config.KBA_QUESTIONS_REQUIRED

        return {
            "verification_id": str(uuid.uuid4()),
            "status": "verified" if passed else "failed",
            "questions_total": config.KBA_QUESTIONS_TOTAL,
            "questions_required": config.KBA_QUESTIONS_REQUIRED,
            "questions_correct": correct,
            "per_question": per_question,
            ...
        }

The KBA challenge asks the customer four independent questions. They need to get at least three right to pass, which is configurable in config.py:

KBA_QUESTIONS_TOTAL = 4
KBA_QUESTIONS_REQUIRED = 3
MAX_IDENTITY_RETRIES = 3
MAX_SIM_SERIAL_RETRIES = 3

You might be wondering why I used mock APIs. The answer is that what is being built and tested here is the agent’s decision-making logic, including the retries, routing, state transitions, and fraud handling. That logic is completely separate from any actual Safaricom API. Mocking the services isolates the agent logic so I can run dozens of deterministic scenarios without needing a live network, and the mock classes can be swapped for real HTTP clients whenever the system is ready for production.

3. The Nodes

Nodes are the individual steps of the workflow, and in LangGraph they follow a simple contract where each node is an async function that takes the full state as input and returns a dictionary of just the fields it changed. LangGraph merges the partial update back into the state automatically.

Here’s the first node, initiate_request which fires when a customer starts the process:

async def initiate_request(state: dict) -> dict:
    request_id = f"SWP-{uuid.uuid4().hex[:8].upper()}"
    now = datetime.now().isoformat()

    msg = (
        f"Karibu Safaricom. I'm here to help you replace your SIM for a "
        f"**{state.get('request_reason', 'lost')}** line.\n\n"
        f"Your request reference is **{request_id}**. "
        f"I'll take you through the swap step by step.\n\n"
        f"**Step 1:** Let's first verify your identity."
    )

    audit_log = _log(state, "request_initiated", {
        "request_id": request_id,
        "request_reason": state.get("request_reason", "lost"),
        "subscriber_id": state.get("subscriber_id", ""),
        "channel": state.get("channel", "shop"),
    })

    return {
        "request_id": request_id,
        "request_timestamp": now,
        "current_step": "initiate_request",
        "process_status": "in_progress",
        "id_verification_status": "pending",
        "id_verification_attempts": 0,
        "id_max_retries": config.MAX_IDENTITY_RETRIES,
        "sim_validation_status": "pending",
        "sim_validation_attempts": 0,
        "sim_max_retries": config.MAX_SIM_SERIAL_RETRIES,
        "old_sim_blocked": False,
        "new_sim_provisioned": False,
        ...
        "audit_log": audit_log,
        "messages": [AIMessage(content=msg)],
    }

This node does three things. It generates a unique request reference, sets every counter and flag to its initial value, and appends the opening message. That last point is intentional as all initialisation lives here and not scattered across downstream nodes.

Now let’s look at verify_identity, because this is the first node where things can actually go wrong:

async def verify_identity(state: dict) -> dict:
    attempts = state.get("id_verification_attempts", 0) + 1
    result = await identity_service.verify(state["subscriber_id"])

    correct = result["questions_correct"]
    required = result["questions_required"]
    total = result["questions_total"]

    if result["status"] == "verified":
        msg = (
            f"Identity verified — you answered **{correct}/{total}** questions "
            f"correctly (minimum {required} required). Thank you!\n\n"
            f"**Step 2:** I'll now check the status of your line."
        )
        status = "passed"
    else:
        remaining = state.get("id_max_retries", config.MAX_IDENTITY_RETRIES) - attempts
        reasons = "; ".join(result.get("failure_reasons", []))
        if remaining > 0:
            msg = (
                f"I got **{correct}/{total}** correct — I need at least **{required}**. "
                f"Specifically: {reasons}.\n\n"
                f"Let's try again. You have **{remaining}** attempt(s) remaining."
            )
        else:
            msg = (
                f"Identity verification failed after "
                f"{state.get('id_max_retries', config.MAX_IDENTITY_RETRIES)} attempts. "
                f"For your security, I cannot proceed with the SIM swap."
            )
        status = "failed"

    return {
        "current_step": "verify_identity",
        "id_verification_status": status,
        "id_verification_attempts": attempts,
        "kba_questions_correct": correct,
        "audit_log": _log(state, "identity_verification", {...}),
        "messages": [AIMessage(content=msg)],
    }

The node increments id_verification_attempts itself, and produces a contextual message depending on the outcome. If retries remain, the customer is told how many are left. If they are exhausted, the node delivers the rejection message. Most importantly though, the node does not decide what happens next. It simply sets id_verification_status to "passed" or "failed" and returns. The decision about whether to retry or reject lives elsewhere.

The other nodes follow the same pattern. check_line_statusqueries the Home Location Register (HLR) and sets fraud_flag, pending_swap, and line_check_passed. capture_new_sim validates the new SIM’s ICCID and IMSI. Thereafter, the core pipeline runs sequentially: block_old_sim, provision_new_sim, reactivate_mpesa, activate_on_network, and finally send_confirmation_sms, which assembles the full swap summary table for the customer.

There’s also a shared log helper used by every node:

def _log(state: dict, action: str, details: dict) -> list[dict]:
    entry = {
        "timestamp": datetime.now().isoformat(),
        "step": state.get("current_step", "unknown"),
        "action": action,
        "request_id": state.get("request_id", ""),
        **details,
    }
    return state.get("audit_log", []) + [entry]

It appends one structured entry per node execution and returns the full updated list. Because audit_log is just another field in the state, it travels with the swap from start to finish, building up a complete record of every step without needing any external logging system.

4. Edges — Where the Decisions Actually Live

This is the most important component of the agentic framework. If nodes are the checklist items then edges are the logic between them. In LangGraph, an edge is a function that reads the current state and returns the name of the next node to run.

There are two kinds. Linear edges connect nodes that always run in sequence, no matter what. The core swap pipeline is entirely linear once the gate checks pass:

graph.add_edge("block_old_sim", "provision_new_sim")
graph.add_edge("provision_new_sim", "reactivate_mpesa")
graph.add_edge("reactivate_mpesa", "activate_on_network")
graph.add_edge("activate_on_network", "send_confirmation_sms")

The linear logic in the above code is that you cannot provision before blocking and activate before provisioning. The order is non-negotiable and the linear edges encode that directly.

On the other hand, Conditional edges are where the agentic behaviour actually comes from and they are three of them. Let’s look at the one after identity verification:

def route_after_identity(state: SimSwapState) -> str:
    if state.get("id_verification_status") == "passed":
        return "check_line_status"
    if state.get("id_verification_attempts", 0) >= state.get(
        "id_max_retries", config.MAX_IDENTITY_RETRIES
    ):
        return "reject_request"
    return "verify_identity"

Three possible outcomes are here. If KBA passed, move on to the line check. If it failed but retries remain, return verify_identityof the same node that just ran. If attempts are exhausted, send to reject_request. That last return value, routing a node back to itself, is the retry loop. A chatbot doesn’t have this. It can’t decide to revisit a step based on what just happened. This function is five lines and it’s actually the difference between a pipeline and an agent.

The line check router is deliberately simpler:

def route_after_line_check(state: SimSwapState) -> str:
    if state.get("line_check_passed"):
        return "capture_new_sim"
    return "reject_request"

It has two outcomes and no retry. Fraud flags, suspended lines, and pending swaps all terminate immediately because none of those are recoverable through self-service. There’s no loop here by design as the edge just encodes the business rule.

The SIM validation router mirrors the identity one, with pass, retry up to the limit, or reject as the three possible outcomes:

def route_after_sim_validation(state: SimSwapState) -> str:
    if state.get("sim_validation_status") == "valid":
        return "block_old_sim"
    if state.get("sim_validation_attempts", 0) >= state.get(
        "sim_max_retries", config.MAX_SIM_SERIAL_RETRIES
    ):
        return "reject_request"
    return "capture_new_sim"

The value of keeping routing logic in edges rather than nodes is so that you can unit test each router as a pure function of state, pass it a dict with the relevant fields and assert on the returned string. You don’t need to wire up the full graph, call any services, or manage async. In addition, if a business rule changes, say the fraud policy becomes a soft warning rather than a hard stop, then you update one function in edges.py and nothing else in the system needs to change.

Putting It All Together

The graph assembly lives in builder.py. It is a straightforward sequence that follows this path: create the graph, register each node, connect them with edges, and compile:

from langgraph.graph import StateGraph, END

def build_graph():
    graph = StateGraph(SimSwapState)

    # Register nodes
    graph.add_node("initiate_request", initiate_request)
    graph.add_node("verify_identity", verify_identity)
    graph.add_node("check_line_status", check_line_status)
    graph.add_node("capture_new_sim", capture_new_sim)
    graph.add_node("block_old_sim", block_old_sim)
    graph.add_node("provision_new_sim", provision_new_sim)
    graph.add_node("reactivate_mpesa", reactivate_mpesa)
    graph.add_node("activate_on_network", activate_on_network)
    graph.add_node("send_confirmation_sms", send_confirmation_sms)
    graph.add_node("reject_request", reject_request)

    # Entry point
    graph.set_entry_point("initiate_request")

    # Linear edge: entry flows into first gate
    graph.add_edge("initiate_request", "verify_identity")

    # Conditional routing at the three decision points
    graph.add_conditional_edges("verify_identity", route_after_identity, {
        "check_line_status": "check_line_status",
        "reject_request": "reject_request",
        "verify_identity": "verify_identity",   # the retry self-loop
    })
    graph.add_conditional_edges("check_line_status", route_after_line_check, {
        "capture_new_sim": "capture_new_sim",
        "reject_request": "reject_request",
    })
    graph.add_conditional_edges("capture_new_sim", route_after_sim_validation, {
        "block_old_sim": "block_old_sim",
        "capture_new_sim": "capture_new_sim",   # retry self-loop
        "reject_request": "reject_request",
    })

    # Linear pipeline: core swap
    graph.add_edge("block_old_sim", "provision_new_sim")
    graph.add_edge("provision_new_sim", "reactivate_mpesa")
    graph.add_edge("reactivate_mpesa", "activate_on_network")
    graph.add_edge("activate_on_network", "send_confirmation_sms")

    # Terminal nodes
    graph.add_edge("send_confirmation_sms", END)
    graph.add_edge("reject_request", END)

    return graph.compile()

One thing I like about LangGraph is that the compiled graph can export itself as a Mermaid diagram for a graphical view as follows:

def get_graph_mermaid() -> str:
    compiled = build_graph()
    return compiled.get_graph().draw_mermaid()

The diagram is generated from the same code that runs in production, which means it can never go stale. Change a routing rule and the diagram reflects it on the next export. Anyone who has ever maintained a system architecture diagram in a slide deck, only to watch it become outdated within a week, will appreciate why that matters.

Five Scenarios, Five Paths Through the Same Graph

I added a demo that you can setup locally and run. The demo comes with five built-in scenarios, each using a different subscriber from the seed data. You run them like this:

python -m demo.cli happy_path
python -m demo.cli identity_retry
python -m demo.cli fraud_flag
python -m demo.cli bad_iccid
python -m demo.cli full_failure

Here’s what each one exercises:

| Scenario       | Subscriber                     | What actually happens |
|----------------|--------------------------------|---------------------------|
| happy_path     | Wanjiku Kamau (+254712345678)  | All gates pass on the first attempt, full pipeline runs, swap completes |
| identity_retry | —                              | KBA fails once, the self-loop fires, passes on the second attempt |
| fraud_flag     | Njoroge Mwangi (+254745678901) | Line check finds a fraud flag, `route_after_line_check` returns `reject_request` immediately |
| bad_iccid      | —                              | SIM validation fails on the first attempt, retries, eventually rejects |
| full_failure   | —                              | Identity verification exhausted after three attempts, process rejected |

The important thing to observe is that all five paths run through the same compiled graph. The agent doesn’t have a separate code path for fraud versus normal flow. The fraud flag is a field in the state that is read by the edge, and is followed by routing. That’s all there is to it.

Below is an example of a happy path that I ran. It produces an output that looks roughly like this:

Agent: Karibu Safaricom. I'm here to help you replace your SIM for a lost line.

Your request reference is SWP-3F8A21DC. I'll take you through the swap step by step.

Step 1: Let's first verify your identity.

Agent: Identity verified — you answered 3/4 questions correctly (minimum 3 required). Thank you!

Step 2: I'll now check the status of your line.

Agent: Line status confirmed.
- MSISDN: +254712345678
- Name: Wanjiku Kamau
- Status: Active
- Current SIM serial: 89254…5678
- M-PESA: Registered (balance Ksh 4,320.00)

Step 3: Please hand me the new SIM so I can capture its serial (ICCID) and IMSI.

...

Agent: Your new SIM is now LIVE on the Safaricom network — voice, SMS, data and USSD are all enabled.

SIM Swap Summary
| Request Ref       | SWP-3F8A21DC          |
| MSISDN            | +254712345678         |
| Old SIM           | 89254…1234 (Blocked)  |
| New SIM           | 89254…5678 (Active)   |
| M-PESA            | Restored (Ksh 4,320)  |
| Confirmation SMS  | Delivered             |

AUDIT LOG (9 entries)
  [2026-04-25T10:12:03] request_initiated
  [2026-04-25T10:12:04] identity_verification
  [2026-04-25T10:12:05] line_status_check
  [2026-04-25T10:12:06] sim_validation
  [2026-04-25T10:12:07] block_old_sim
  [2026-04-25T10:12:08] provision_new_sim
  [2026-04-25T10:12:09] mpesa_reactivate
  [2026-04-25T10:12:10] network_activation
  [2026-04-25T10:12:11] confirmation_sms

Final status: completed

The above shows nine audit entries, one per node, each with a timestamp and a request ID. The full state at any point in that run can be checkpointed and replayed. That’s not something you get out of any conventional chatbot.

What You Actually Have Now

If you run this locally, here’s what you’ve actually built:

• The graph is the documentation: The Mermaid export is generated from live code. The retry loops, the fraud halt, and the M-PESA skip for unregistered lines are all visible in the diagram and come directly from the routing functions. They’re all visible in the diagram and they all come directly from the routing functions. Nobody has to keep a separate flowchart up to date.
• The state is the audit trail: Every step appends one entry to audit_log. By the time the process reaches its terminal node, you have a complete, timestamped record of exactly what the agent did and why. That record lives in the same object as the result, no separate logging infrastructure required for the prototype.
• The edges are the policy: If Safaricom changes the KBA requirement from three-out-of-four to four-out-of-four, that’s a one-line change in config.py. If fraud policy changes from immediate termination to a human-escalation path, you update route_after_line_check and add a new node. The rest of the graph is not really affected.

What’s Still Missing

What I’ve tried put together here is a solid prototype. But there’s a meaningful gap between prototype and production, and I’ll try cover this gap in Part 3.

Remember I haven’t connected this to anything external yet. There’s a FastAPI wrapper that exposes the agent as an OpenAI-compatible endpoint so Open WebUI can talk to it, but I haven’t walked through that here. I’ve also not added observability. Right now, if something fails mid-run, you would need to dig through the state manually. There is also no human-in-the-loop interrupt, meaning an operator cannot pause the flow and review it before the old SIM gets blocked. For a production SIM swap system, that last one is not optional.

I promise to cover most of these in Part 3: the FastAPI service, hooking into Langfuse for tracing, and adding interrupt-based human review at the fraud flag step. The code will be extended in the same repo.

The full source code and Readme on how to run it is available HERE.

I hope this was useful. If you found it helpful, then please follow me for more of such, leave a comment, clap for me and of course let me know what else you’d like to see in Part 3 . You can also find all my other articles on my profile and connect with me on LinkedIn. Remember all my articles are FREE to read.

From Theory to Code — Building a SIM Replacement Agent in LangGraph (Part 2) was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stuck behind a firewall? Read the article for FREE here

It’s been quite some time since I started writing “seriously” about data analytics and science, optimisation and, most recently, generative AI. In the Gen AI space, I’ve tried covering topics around RAG, finetuning, agentic AI, and a fair bit in between. If you’ve read any of my articles, I’m sure you know I lean towards not only a simple but also a practical angle rather than theoretical ones. I also share all links on my profile, and all are FREE to read.

Agentic AI is THE buzzword right now, and I’m happy to join the bandwagon with a series on it. After all, I’m just human 😂.

But here’s what actually pushed me to write this. I kept reading articles that promised to explain “agentic AI” and ended up describing a chatbot with a slightly longer prompt. Most of them are wrappers around an LLM (Large Language Model) with one or two API calls bolted on, dressed up as something brand new. Simply put, agentic AI is an AI system that can accomplish a specific goal with limited human supervision, exhibiting real autonomy, goal-driven behaviour, and adaptability rather than just operating within predefined constraints. I don’t buy the watered-down versions, and of course I’m NOT going to claim I’ve built something wildly novel either.

But I did build one that actually is agentic. An agent that runs the full SIM card replacement process for a mobile network on its own, from identity verification all the way to customer notification, with no human pushing it along at each step. I built it as a reference implementation around the Safaricom Kenya SIM swap flow just in case a customer needs it done.

This is Part 1 of a three-part series. Here I’ll cover what agentic AI actually means, why a SIM replacement workflow is a good example of it, and the four mental building blocks every agent needs. Part 2 turns everything here into working open-source Python code with LangGraph. Part 3 is about what it takes to ship one of these into production. So don’t expect any code in this part, just the concepts you’ll need to make sense of the rest of the articles.

What Makes an AI “Agentic”?

Let me clear something up, because this word gets abused a lot.

A standard chatbot is reactive. You say X, it says Y. It has no memory of what it did five messages ago. It cannot call external systems on your behalf. And critically, it cannot decide what to do next based on what just happened. Every turn is independent. Have you ever found yourself typing “talk to a human” or “speak to an agent” into a customer service chatbot just to get transferred to a real person? The bot did not figure out on its own that you were frustrated or that your issue was beyond it. You had to spell it out. That is exactly the limitation we are talking about.

An agentic AI is different. It decides which action to take next based on the current situation. It calls real tools like APIs, databases, and provisioning systems. It handles failures gracefully, retrying or escalating as needed. It maintains its own running state across many steps. Lastly, it works towards a goal without a human directing every move unless fully specified to be so in its design.

Think of the difference between a customer service rep reading from a script versus an experienced retail agent who knows the process, handles edge cases, and uses their own judgment at each step. That’s the gap we’re bridging. The script-reader is your chatbot, while the experienced agent is your agent.

The SIM replacement use case is a good testbed for this because it is genuinely multi-step, genuinely branching, and genuinely fails in interesting ways. Identity verification can fail several times before the system gives up. The new SIM serial may not match what the customer reads out. M-PESA reactivation can stall. A line flagged for fraud terminates the process entirely. A simple chatbot collapses at the first unexpected branch while a properly configured agent handles all of it.

The Four Building Blocks of Every Agent

Before we get into the specifics of the SIM replacement workflow, here’s the mental model that makes everything else click. Every agent, regardless of framework, comes down to four concepts.

Concept   | Question it answers                    | Analogy
----------|----------------------------------------|--------------------------------
State     | What does the agent know right now?    | A notebook the agent carries
Tools     | What can the agent do?                 | The agent's hands
Nodes     | How does each step work?               | Individual tasks on a checklist
Edges     | What should the agent do next?         | The decision logic between tasks

Let me walk through each one in plain language.

1. State is the agent’s running memory. It’s everything the agent knows at any moment, including who the customer is, what’s been verified, what’s failed, and what’s still pending. In the SIM replacement scenario, state holds the customer’s MSISDN (their phone number), their ID details, how many verification attempts have been used, the new SIM’s serial number, whether the old line has been blocked, whether the new SIM has been activated, whether M-PESA has been re-linked, and so on. Think of it as a notebook the agent carries from step to step. Every step reads from it and writes to it.
2. Tools are what the agent can actually do in the real world. APIs to call, databases to query, provisioning systems to update, SMS gateways to message. In the SIM replacement example, the tools include an identity verification service, a line status check, the SIM provisioning system that blocks the old IMSI and binds the new one, an M-PESA reactivation service, and a notification service. Without tools, an agent is just talk. With tools, it can take action.
3. Nodes are the individual steps of the workflow. Each node has one job. In this example, it could be to verify the customer’s identity, check the line status, block the old SIM, activate the new one, re-link M-PESA or even send the confirmation SMS. Each step does its work, usually by calling one or more tools, then updates the state. Keeping each node focused on a single job is what makes the whole system testable and debuggable later.
4. Edges are the decision logic between steps. After a node finishes, an edge looks at the current state and decides where to go next. This is where the “agentic” behaviour actually lives. For example, if identity verification passes, then go to the line status check. If it failed but retries remain, loop back and try again, and if retries are exhausted, terminate the request. Edges are how an agent decides, dynamically, what to do next based on what just happened. A chatbot doesn’t have edges. It just responds.

That’s the entire mental model. State, tools, nodes, and edges. Once you have these four ideas in your head, every agent you ever look at will make sense.

The Problem We’re Actually Solving

When a Safaricom customer loses their SIM card, the process at an M-PESA agent or retail shop looks roughly like this, and this is just one path that I’m keen to automate:

Each step depends on the previous one. Some steps, like identity verification, can loop within a limit while some are terminal. For example, a fraud flag ends the process immediately while others adapt to circumstances. If the customer has forgotten their original PIN, the workflow takes a different verification path using deeper account history questions instead of failing outright. This is exactly the kind of workflow that exposes whether your “agent” is real and works or is simply a marketing gimmick.

It will be hard for a traditional chatbot to do this. It can answer “what happens when I lose my SIM” with a script. However, it cannot actually run the process, handle the branching, retry failed steps, or decide between standard verification and the deeper PIN-recovery path based on what the customer remembers. An agent can and that’s the whole point.

Why This Matters Now

The framing I keep coming back to is this: an agent is not a smarter chatbot. It’s a different architecture entirely. You stop thinking about responses and start thinking about processes. You stop thinking about prompts and start thinking about state. That shift in mental model is what separates the people building real agentic systems from the people slapping the label on a chatbot and hoping no one notices.

There’s a practical reason this matters now. The cost of LLM calls has dropped enough that running a multi-step workflow is no longer prohibitive. Frameworks like LangGraph have matured to the point where the plumbing is no longer the hard part. The constraint has shifted from “can we build this” to “do we understand what we’re building.” For organisations sitting on workflows that look like the SIM replacement process, and most telcos, utilities, and service providers have dozens of them, the door is open.

The interesting question is no longer whether agents work. It’s which workflows are worth turning into agents and what you need to know to build them properly.

What Comes Next

That’s Part 1, and I intended to keep it as theoretical as possible. I covered what agentic AI actually means, why a SIM replacement workflow is a good test of whether something is genuinely agentic, and the four mental building blocks (state, tools, nodes, edges) that every agent shares.

In Part 2, I’ll translate all of this into working Python code using LangGraph. We’ll build the actual state object, wire up the tools, write the nodes, and define the edges that handle the retry loops, the fraud flag termination, and the alternative verification paths. Everything in this article will be there in the code that will be open for anyone to customise.

In Part 3, I’ll cover deployment. This will largely be what it takes to move from a working prototype locally to a production system: observability, human-in-the-loop interrupts, the FastAPI service, etc.

I hope this walkthrough was useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to cover anything specific in Part 2 or Part 3, whether that’s a particular LangGraph pattern, a deployment detail, or something else entirely. If you want to check out my other articles, you can find them on my profile. I’m also happy to connect via LinkedIn.

What Agentic AI Actually Is: A SIM Replacement Use Case (Part 1) was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Author’s note: This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.

Stuck behind a firewall? Read the article for FREE here

In Part 1, I covered the data science workflow behind this project. This included the process of generating synthetic museum visitor data, engineering 80+ features from 26 raw columns, training and comparing multiple models, and evaluating the winner. XGBoost took the top spot with a MAPE of 14.0%. I also detailed the process of framing model performance in business terms that museum directors actually care about.

But a model sitting in a Jupyter notebook is completely useless to a museum operations manager who needs to plan staffing for next Tuesday. The real value comes from making predictions accessible to the people who need them through a working application with a user interface.

In this part, I’ll take the trained model and deploy it as a full-stack application: a FastAPI backend that exposes the model as an API, a React dashboard for the frontend, and a Supabase database for logging predictions. By the end of this article, you’ll understand not just what I built, but why each architectural decision was made.

I recommend once again that you start with Part 1 if you haven’t read it yet as it literally covers the why and what behind the model. On the other hand, this article covers the how of getting it into production. As always, all code is open-sourced here.

From a Notebook to Production — The Full Stack

1. Why We Need a Backend at All

Before diving into code, let’s establish why a backend in a production setting exists in the first place. The trained XGBoost model is basically a Python object that lives in a .json file on disk and can only be used by Python code that knows how to load it.

On the other hand, a React web dashboard is JavaScript running in a browser that cannot directly load a Python model. Therefore, the solution is a backend server which is just a Python application that loads the model and exposes it over HTTP. Any client e.g., a browser, mobile app, or spreadsheet plugin can then send it a request and receive a prediction in return.

This pattern is called a REST API(Representational State Transfer Application Programming Interface). Think of it like a restaurant: you (the frontend) don’t walk into the kitchen and cook your own food. You hand a request to a waiter (the API), the kitchen (the backend) prepares the response, and the waiter brings it back to you in a standard format. You never need to know how the kitchen works.

a) Backend — FastAPI

I used a FastAPI framework that is a modern Python one for building APIs quickly and correctly. Two things make it particularly good for ML serving:

1. Automatic data validation — You just define the shape of your request and response using Python type hints, and FastAPI automatically validates incoming requests and rejects malformed ones before they can crash your model.
2. Auto-generated documentation — FastAPI produces a live /docs page that lists every endpoint, what it accepts, and what it returns. This is invaluable when frontend developers (or your future self) need to understand how to call the API.

The backend in this application exposes four endpoints, each serving a distinct purpose:

- `POST /api/predict` - The core endpoint. Accepts a date and optional context parameters (weather, holidays, special events), runs them through the model, and returns a prediction with confidence intervals and a traffic tier label.
- `GET /api/historical` - Returns paginated historical visitor data with optional date filters. This powers the charts in the frontend's Historical tab.
- `GET /api/insights` - Returns model performance metrics, feature importance rankings, and a comparison table of all the models trained in Part 1. This powers the Insights tab.
- `GET /api/date-info` - A smart helper endpoint: given a date, it automatically detects Victorian public holidays and fetches weather forecasts. The frontend calls this whenever the user picks a date, so the prediction form populates itself without requiring manual input.

Loading the Model Once — The Lifespan Pattern

One of the first problems you’ll encounter when deploying an ML model is the thought of deciding when the model is to be loaded. The naive approach is to load the model at the top of your script as a global variable. This works, but it has a problem in production where web servers typically run multiple worker processes to handle concurrent requests. A global variable in one process is not shared with another process meaning you’d be loading the model independently in each worker, which wastes memory.

A better approach and I stand to be corrected is FastAPI’s lifespan context manager. This is a special function that runs once when the application starts up, before it begins accepting any requests:

@asynccontextmanager
async def lifespan(app: FastAPI):
    ModelService.load()
    yield

app = FastAPI(
    title="Museum Visitor Prediction API",
    description="Predict daily visitor numbers for the Melbourne Museum of Migration",
    version="1.0.0",
    lifespan=lifespan,
)

The yield keyword is what makes this a context manager. Everything before yieldruns at startup and everything after yield (if you add cleanup code) runs at shutdown. The ModelService.load() call loads the XGBoost model from disk, reads the feature list, loads historical data, and computes residual statistics. This is the heavy one-time work that should not be repeated on every request.

The ModelService Singleton

The ModelService class is designed as a singleton, which is just a software pattern where a class has at most one instance, and that instance is shared across the whole application.

Rather than using Python’s normal instance creation model = ModelService(), all the methods are marked as @classmethod. This means they operate on the class itself, not on an instance. The model, feature list, historical data, and residual stats are stored as class-level variables, not instance variables. The effect is straightforward; there’s exactly one copy of the model in memory no matter how many API calls come in simultaneously, and every call reads from the same shared state.

Normal class: ModelService() → creates new object every time
Singleton pattern: ModelService.predict(…) → always uses the same shared state

This matters because XGBoost models can be large (tens or hundreds of megabytes). Loading one per request would exhaust memory within seconds under real traffic. Therefore loading one at startup and sharing it is the correct production pattern. Don’t worry if the above doesn’t make much sense as I’ll dig deeper in the interesting parts of the code.

Before running any of these steps, please follow this setup guide to run the backend processes first. You’d normally get an output like the following if the setup worked correctly:

Remember are all these are setup for a local deployment.

When a user clicks the Predict button in the frontend that I’ll describe later, this is what really happens:

• Request arrives — The frontend sends an HTTP POST request to /api/predictwith a JSON body containing the date and any contextual parameters the user provided or the auto-fetch populated (weather, holidays, events).
• Feature construction — The raw request has only a handful of fields. But the model was trained on 81 features. The build_feature_row() function bridges this gap. It takes the date and context, looks up recent visitor history (needed for lag features), applies the same cyclical encodings and interaction terms that were built during feature engineering in Part 1, and returns a single-row DataFrame with all 81 features in the exact order the model expects. For context, this is one of the most common failure points in ML deployments where a training-serving skew happens. This is where if the feature engineering code in the backend differs even slightly from what was done during training, the model will silently produce wrong predictions. I stored the feature list in a features.json(the same file produced by the training notebook), so the backend reads that file and uses it as the authoritative column ordering.
• Model inference and inverse transformation — There’s something interesting that’s happening in the code below. In Part 1, the XGBoost model was trained on log1p(visitors) rather than visitorsdirectly. If you remember, it was because visitor counts were right-skewed. This meant that on most days the museum received around 200–400 visitors, but some peak days spiked to approximately 1,200. This skew can confuse tree-based models that try to minimise squared error , meaning they’ll underpredict peaks to avoid large penalties on the majority of ordinary days. Taking the log then compresses this scale in the lines of log1p(200)≈5.3 and log1p(1200)≈7.1. The model now sees a more uniform distribution and learns better. But the prediction comes back in log-space, so we have to invert it. np.expm1(raw_prediction)converts the log-space output back to visitor counts. The max(0,point)call in the code clips any negative predictions (which are physically impossible for visitor counts) to zero.

@classmethod
def predict(cls, feature_row: pd.DataFrame) -> dict:
    raw = cls._model.predict(feature_row)
    point = float(np.expm1(raw[0] + cls._xgb_log_offset)) \
        if cls._best_model_name == "XGBoost" else float(raw[0])
    point = max(0, point)

• Prediction intervals — A single number e.g., 347 visitors as the prediction is less useful than say a range (“between 290 and 405 visitors with 80% confidence”). Normally, prediction intervals communicate uncertainty, which is critical for staffing decisions such that if the interval is wide, schedule conservatively etc.

std = cls._residual_std or 50.0
lower_80 = max(0, int(point - 1.28 * std))
upper_80 = int(point + 1.28 * std)
lower_95 = max(0, int(point - 1.96 * std))
upper_95 = int(point + 1.96 * std)

The residual_std above is the standard deviation of the model’s prediction errors on the training data. To simplify it, residuals are just the differences between what the model predicted and what actually happened. For example, if the model predicted 350 but 410 came in, then the residual is 60. Computing the standard deviation of all residuals gives us a single number that summarises how spread out the errors typically are.

The multipliers 1.28 and 1.96 come from the normal distribution whereby under a normal distribution:

- 80% of values fall within ±1.28 standard deviations of the mean
- 95% of values fall within ±1.96 standard deviations of the mean

Therefore, point ± 1.28 * std gives us the 80% prediction interval whereby in 8 out of 10 days, actual visitors should fall inside this range. The 95% interval is wider as it needs to capture more of the possible outcomes. This is called a Gaussian approximation of the prediction interval. It’s not perfect (residuals are rarely perfectly normal), but it’s a practical and interpretable first approximation that works well in production.

• Traffic tier classification and response

if point < 250:   tier = "Low"
elif point < 450: tier = "Medium"
else:              tier = "High"

return {
    "predicted_visitors": int(round(point)),
    "lower_80": lower_80, "upper_80": upper_80,
    "lower_95": lower_95, "upper_95": upper_95,
    "traffic_tier": tier,
    "confidence": round(max(0.0, min(1.0, float(r2))), 3),
    "model_used": cls._best_model_name,
}

The traffic tier translates the raw number into an operational instruction. A museum operations manager doesn’t just need a number, they need an action. This could be something like call in extra staff, run normal operations, or reduce floor coverage. Low/Medium/High maps directly to those three staffing modes. This is an example of designing the output around the decision it needs to support.

The Date Info Endpoint — Smart Defaults That Reduce Friction

The /api/date-info endpoint demonstrates something I think is under-appreciated in data product design: reducing the cognitive load of correct usage.

The prediction form needs several inputs beyond just the date: Is it a public holiday? What will the weather be? Is there a school holiday? A naive form would present empty text boxes and leave it to the user to look all this up. That creates friction and introduces human error. For example, the user might not know that Anzac Day (April 25th) is a Victorian public holiday, or might enter the wrong temperature.

Instead, when the user selects a date, the frontend automatically calls /api/date-info which:

1. Checks the Victorian public holiday calendar — a hardcoded list of known dates plus rules for floating holidays (like Easter).
2. Fetches weather from the Open-Meteo API — a free weather API that provides both historical data and 7-day forecasts. For dates in the forecast window, it returns the forecast. For past dates, it returns historical actuals. For dates beyond the forecast window (more than 7 days out), it falls back to seasonal defaults based on Melbourne’s average climate by month.

The user sees the prediction form pre-populated with contextually appropriate values, each labelled with a badge showing its source: Auto, Forecast, Historical, or Seasonal Default. If they want to override any of these, they can as the form switches to manual mode and stops auto-fetching that field. This preserves user intent while still providing smart defaults.

This is an example of UX design thinking applied to a data product. The goal is not just for the model to be accurate, but for the application to be usable by someone who is not a data scientist.

Frontend — React + Tailwind Dashboard

The frontend is a React 18 application built with Vite and styled with Tailwind CSS as shown below and can also be accessed HERE. It has three tabs, each serving a distinct purpose for a different type of user question.

Why React? React is a JavaScript library that makes it practical to build UIs where the data and the visual representation stay in sync. For example, when a user selects a new date, the form fields, weather badges, and prediction result all need to update together. React’s component model makes this manageable where each piece of the UI is a self-contained component that re-renders when its data changes, instead of requiring you to manually update the DOM for every change.

Why Vite? Vite on the other hand is the tool that bundles and serves the React application. For development, it provides instant reloading such that when you save a file, the browser updates in under a second without losing the current UI state. For production, it produces an optimised static bundle (HTML, CSS, JavaScript files) that can be hosted on any static file host like Vercel or Netlify.

Why Tailwind CSS? Tailwind is a utility-first CSS framework. Instead of writing custom CSS classes, you apply small single-purpose utility classes directly in your JSX markup (`text-lg font-bold text-blue-600`). The result looks verbose but produces consistent, responsive designs without the overhead of managing a separate stylesheet.

Predict Tab

This is the primary view. It presents a form where the user selects a date. It then auto-populates the weather and holiday fields appear immediately after date selection via the /api/date-info call. The user can override any field if they have better information (e.g., they know a special event is happening that day).

When users submit, the prediction result panel appears alongside the form as below:

• The large point estimate (e.g., “538 visitors”)
• The 80% and 95% confidence intervals displayed as a range bar
• The traffic tier badge (Low / Medium / High) in a colour-coded chip
• Estimated revenue that is computed by multiplying the predicted visitor count by an assumed average spend per visitor. This can be customised.
• The model name and its R² score, so the user understands which model produced this result

Displaying confidence intervals rather than just a point estimate is a deliberate choice. A museum director who sees “Low: 526/ High: 549” immediately understands there is uncertainty in the prediction and can plan for the upper bound when in doubt. However, if the person only sees “538” might staff for exactly 538 and be caught out on a day that comes in at say 600.

Historical Tab

This tab shows interactive charts of historical visitor patterns over the past year, built with Recharts — a charting library for React that renders responsive SVG charts.

The primary value here is context. Before making a staffing decision based on a prediction, a manager might want to know something like: “Is next Saturday typically busy at this time of year? Has that changed recently?” The historical chart lets them visually scan for seasonal patterns, spot anomalies, and build intuition about the data the model was trained on.

The data is fetched from the GET /api/historicalendpoint, which supports date-range filtering and pagination so large date ranges don’t overwhelm the browser.

Insights Tab

This tab is aimed at a more technically curious user. Perhaps an internal analytics team or a curious manager who wants to understand why the model is making certain predictions.

It shows:

• Model performance metrics — MAE, RMSE, R², MAPE for the best model, displayed as summary stat cards. I described some of them in Part 1.
• Feature importance rankings — the top features the model relies on, pulled from the SHAP analysis done in Part 1. This answers questions like “Is the model mostly driven by day-of-week, or does weather matter a lot?”
• Model comparison table — shows all the models trained (Random Forest, XGBoost, Prophet) side by side, so the user can see how much better XGBoost is compared to the alternatives and understand the trade-offs

The summary stat cards at the top of the whole dashboard i.e., the average daily visitors, peak day visitors, days of history, best model R² etc., give any user immediate orientation before they start exploring. These numbers answer the first questions anyone asks when they encounter a new dataset: “What are we dealing with here?”

Deployment Considerations

As is, the frontend can be deployed to Vercel (static files, CDN-served) and the backend to Railway (Python process, model artifacts bundled). Keeping them separate means you can redeploy a CSS fix without touching the model server, and scale each independently. If you want to add a database layer like I did for prediction logging, the repo includes a Supabase setup — see setup_guide.md.

Remember to set environment variables to keep secrets out of source code:

# Backend
APP_ENV=production
# Frontend
VITE_API_URL=https://your-backend.railway.app

One security note to consider before going live: the backend currently has allow_origins=["*”], which lets any website call your API from a browser. Remember to restrict it to your frontend domain in production as follows:

allow_origins=["https://museum-dashboard.vercel.app"]

Would I do it differently with real data?

Building on synthetic data is a great way to develop and validate a system end-to-end, but I want to be honest about where the assumptions break down.

1. Data quality is the hard part — Synthetic data is clean by construction. Every day has exactly one record, all fields are populated, and the relationships between variables are consistent because I programmed them to be. Real visitor logs are messy. A real data pipeline would encounter missing days (scanners were down, the museum was closed for maintenance, or records simply weren’t captured), duplicate entries (the same group scanned through multiple times), scanner malfunctions (a door counter stuck reporting 12 visitors per hour regardless of actual traffic etc. These systematic errors are hard to detect without cross-validation against staff headcounts or revenue data), and inconsistent recording (different staff members classifying “school group” visits differently across years, making that feature unreliable as a historical signal). Budget significant time, often more than modelling itself for data cleaning as a sophisticated model cannot compensate for unreliable input data.
2. Features you cannot simulate — Real venues have data signals I couldn’t credibly replicate in my experimentation such as:

• Social media mentions — For example a viral Instagram post of a new exhibition can drive a spike in visitors 3–5 days later. Social listening APIs can quantify this.
• Google search trends — search volumes for the museum’s name are a leading indicator of visit intent, available from Google Trends.
• Nearby parking occupancy and public transport ridership — visitors can’t come if they can’t park or get there. Another example is the free transport in Victoria that could spike the numbers.
• Competitor events — a major festival at a nearby venue can either draw visitors away (substitution) or increase foot traffic to the whole area (complementarity).

Each of these would require integration work but could meaningfully improve predictions. The XGBoost model we built is good at learning from tabular signals. Therefore, adding better signals is often more valuable than tuning the model itself.

3. Model retraining — A model trained today will degrade over time, a concept called model drift. Normally, the patterns in the world change, but the model’s learned parameters don’t. For a museum, drift sources include a new exhibition type that attracts a different visitor demographic, a pricing change that affects visit frequency, construction nearby that reduces foot traffic for several months, or a global event (a pandemic, an economic recession) that fundamentally shifts cultural activity patterns etc.

The practical solution to such changes is a scheduled retraining pipeline. Monthly or quarterly, retrain the model on accumulated historical data (including recent data not available at initial training time), compare it against the current production model on a holdout period, and promote the new model if it performs better. A prediction log table that records inputs, outputs, and eventually actual visitor counts is the foundation for this as it gives you a growing dataset of prediction errors that reveals whether the model is drifting.

4. Shadow mode before going live — Before using model predictions to make actual staffing decisions, run the model in shadow mode for several weeks. The model generates predictions silently in the background and logs them but they’re not shown to the operations team, who continue making decisions based on their existing process.

After a few weeks, compare the model’s predictions against actual visitor counts alongside the team’s existing estimates against actual visitor counts.

This serves two purposes. First, it validates that the model is actually better than the human baseline before it influences real decisions . If its not, then you need to understand why before deploying it. Second, it builds trust with the operations team. When they can see side-by-side that the model’s numbers were closer to reality on 73% of days, they stop treating it as a black box and start treating it as a useful tool.

Skipping shadow mode and deploying directly to operations is a common and costly mistake. The model might be good, but if the team doesn’t trust it, they’ll ignore it. If it makes a bad call on week one (which it will, eventually), they’ll dismiss it entirely and go back to manual estimates. Shadow mode lets you demonstrate value before you ask people to change their behaviour.

Conclusion

That’s it for now. Across these two articles, I’ve taken you through the entire process of building a visitor prediction system, from simulating realistic museum data, through feature engineering with 80+ features, training and comparing multiple models, evaluating with SHAP explainability, and deploying as a full-stack application with a FastAPI backend and React dashboard.

A few things worth noting from the two articles (Part 1 is here). XGBoost wins with a MAPE of 14.0%, which is production-ready for operations planning. A 14% error on a day with 350 expected visitors means being off by about 49 people which is manageable for staffing adjustments. Seasonality and context dominate: day-of-year, day-of-week, and contextual factors (boost count, weather, holidays) outrank individual lag features in SHAP importance. The model is essentially learning that “Saturdays in summer are busy; rainy Tuesdays are not.” Feature engineering matters more than model selection as most of the performance gain comes from them. A well-engineered feature set with a simple model often beats a poorly-engineered one with a sophisticated model. In addition, business framing is everything. For example, traffic tier classification and revenue estimates speak louder than R² scores. An R² of 0.89 means nothing to a museum director. However, identifying a “Low/Medium/High staffing tier” is immediately actionable.

If you combine this temporal prediction with the spatial analytics from my WiFi article, you have a complete visitor intelligence system. You’ll have the power to predict how many will come, then optimise where they go within the venue. This sounds like a million-dollar startup idea 🚀.

As always, all the code is open-sourced here. Clone the repo, run the notebooks in order (01 through 05), spin up the backend with uvicorn app.main:app --reload, and the frontend with npm run dev . Please feel free to adapt it for your own venue, whether it’s a museum, gallery, shopping mall, or event space.

I hope this walkthrough was useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to extend this to real-time prediction with streaming data or visitor segmentation (families vs solo visitors vs groups). If you want to check out my other articles, you can find them on my profile. I’m also happy to connect via LinkedIn.

From Jupyter Notebook to Full-Stack ML App with FastAPI, React, and Supabase was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

How I built an XGBoost forecasting pipeline to turn visitor demand into staffing insight for a Melbourne museum.

Author’s note: This article uses independently created code and synthetic data for illustration. The methods described are standard forecasting and machine learning techniques commonly used across many application areas.

Stuck behind a firewall. Read the article for free HERE

It’s been a while since I wrote my last article on WiFi Analytics. For context, I outlined a simple process of analyzing foot traffic patterns to identify the best locations for premium advertisements or exhibitions in malls using WiFi pings. It was a graph-inspired analysis to rank the locations by reach, dwell time, and flow. Please read it here for free to get the whole idea. As usual, I also shared all the code for reuse.

But here’s the thing: knowing where people go is only half the puzzle. The other half is knowing how many are likely to show up in the first place. I’m sure every mall/museum/exhibition operations manager has sleepless nights when the question “How many visitors should we expect today?” crosses their mind.

This is what led me to writing this article. It is a natural progression from the WiFi analytics piece and is split in two parts. Whereas the WiFi piece focused on spatial patterns, this one (Part 1) focuses on temporal prediction of visitor numbers. This is not a simple process contrary to what some people might think. I’ll try to provide a picture of how prediction of visitor numbers happens in such venues.

I also pivoted a little from the shopping/malls use cases and actually focused on the museum space and specifically the Immigration Museum in Melbourne, Australia. This venue actually exists and I chose it based on several factors. Melbourne’s Southern Hemisphere seasons, Victorian public holidays, and unpredictable weather make for an interesting prediction challenge as these are all features that will be packaged in the modelling process. In addition, and this is personal, it’s the only museum that I pass by when travelling into the city of Melbourne. Maybe that’s why the name and interest stuck.

As outlined earlier, Part 1 covers the synthetic data generation process through to feature engineering, model training and evaluation. Part 2 will cover the deployment part with a production-ready FastAPI backend and a React dashboard. All together, they form an end-to-end pipeline that anyone can leverage when building a related solution. As always, all code is open-sourced here. Please feel free to clone the repo and adapt it for your own venue.

1. The Problem and Why Prediction Matters

Museums and cultural venues face daily operational challenges whereby if you staff too many people on a quiet Tuesday, then you're burning budget. On the contrary, if you staff too few on a public holiday or weekend then visitor experience suffers. The same logic applies to other related areas like ticketing, catering, exhibit rotations, etc.

A properly thought-through predictive model captures all of these factors simultaneously and outputs a number along with a confidence interval and even a traffic tier that the operations team can actually rely and act on. Therefore, with such a model in place, operations managers in such venues are likely to ask something like "Do we need the Low, Medium, or High staffing plan today?" instead of "how many visitors will we get?". Enough of the theory. Let's jump to the interesting bits.

2. Dataset Design & Simulation

The first challenge in any predictive modelling project is data. I didn’t have access to real museum visitor logs, so I opted to generate a synthetic dataset that largely reflects realistic museum visitation patterns. You can also replicate this and especially if you’re wanting to control the signal-to-noise ratio and demonstrate some methodology before deploying on messy real-world data.

I simulated a visitations dataset spanning three years (January 2023 to December 2025). This amounted to about 1,096 daily records.

The greyish line in the plot is the raw daily count; the red line is the 30-day moving average that reveals the seasonal rhythm e.g., summer peaks, winter troughs, and a slight upward trend over the three years.

Each record captures the following:

• Temporal features: day of week, month, quarter, week of year
• Weather: temperature (Melbourne-specific ranges), precipitation, weather type (Sunny/Partly Cloudy/Cloudy/Rainy)
• Calendar events: Victorian public holidays (Australia Day, Anzac Day, Melbourne Cup Day, etc.) and school holidays
• Venue events: special exhibitions (9 across 3 years, each lasting 2–3 months), local events (F1 Grand Prix, Arts Festival), marketing campaigns, ticket promotions

This approach might present some deviation from the real data but the key here is “realism”. For example, Melbourne’s climate is specific where January averages about 20°C while July averages 9.5°C (Southern Hemisphere, remember). Victorian public holidays like Melbourne Cup Day are usually held on the first Tuesday of November and are unique to this state.

From the data and the plot above, sunny days average noticeably higher visitor counts than rainy ones, and there’s a clear positive trend between temperature and visitor numbers. Anyone who’s experienced a Melbourne winter knows outdoor plans take a hit, and this is exactly what you’d expect for an outdoor-accessible venue in Melbourne.

Alternatively, the above plot has four views of the same temporal story: weekends (Saturday and Sunday, highlighted in red) drawing 30 to 45% more visitors than midweek; December and January peak at ~40% above average thanks to summer holidays; Q1 and Q4 dominate quarterly traffic. The heatmap in the bottom-right reveals a deeper interaction whereby a Saturday in December is the busiest cell in the entire grid.

In the simulation process, the visitor count itself is generated using multiplicative factors applied to a base of 250 (arbitrary number that can be changed) visitors/day as follows:

# Day of week effect
day_multiplier = {0: 0.90, 1: 0.85, 2: 0.88, 3: 0.92, 4: 1.00, 5: 1.30, 6: 1.25}
df['visitors'] *= df['day_of_week'].map(day_multiplier)

# Seasonal effect (Southern Hemisphere)
season_multiplier = {
    1: 1.40, 2: 1.35, 3: 1.15, 4: 1.05, 5: 0.90, 6: 0.80,
    7: 0.75, 8: 0.85, 9: 1.00, 10: 1.10, 11: 1.20, 12: 1.45
}
df['visitors'] *= df['month'].map(season_multiplier)

# Holiday effects
df.loc[df['is_public_holiday'] == 1, 'visitors'] *= 1.50
df.loc[df['is_school_holiday'] == 1, 'visitors'] *= 1.30

# Weather impact
df.loc[df['weather_type'] == 'Sunny', 'weather_factor'] = 1.20
df.loc[df['weather_type'] == 'Rainy', 'weather_factor'] = 0.70
df['visitors'] *= df['weather_factor'] * df['temp_factor']

# Event and promotion effects
df.loc[df['special_exhibition'] == 1, 'visitors'] *= 1.25
df.loc[df['local_event'] == 1, 'visitors'] *= 1.40

There is a high chance that the base number will be different but I hope you get the point. Beyond the core multipliers, each factor has a measurable impact on visitor numbers. The chart below shows the average uplift from each binary factor. Public holidays boost attendance by ~50%, local events by ~40%, and even marketing campaigns deliver a measurable ~15% lift. Remember all these factors are hypothetical but I believe they largely represent the reality.

Each panel in the above plot compares average daily visitors with and without that factor active. The percentage labels show the relative uplift. Public holidays and local events dominate, but the smaller effects (marketing, promotions) are still meaningful when they stack together.

I also added 12% Gaussian noise, a 2% outlier rate (days with 1.5 to 2.5x normal traffic), and a slow growth trend (~0.01% daily). The final values are clipped between 50 and 1,200 visitors. These choices create a dataset that’s statistically rich enough to demonstrate the methodology while being reproducible. Remember, this is a simulated dataset and should not be used as a replacement for real data.

Looking at the year-over-year monthly averages, you can see the seasonal pattern is consistent across all three years, with 2025 sitting slightly above 2023 thanks to that slow growth trend baked into the simulation.

All three years follow the same seasonal arc. A summer peak in December to January, a winter period between June and July, and a gentle upward shift each year reflecting the long-term growth factor.

The resulting distribution shows the right-skewed shape typical of real visitor data: most days cluster between 200 and 400 visitors, with a long tail stretching past 800 on peak days.

The histogram on the left shows the characteristic right skew (skewness ≈ 0.9), with the mean sitting above the median and a handful of high-traffic outlier days pulling the average upward. The Q-Q plot confirms the departure from normality in the tails, motivating the log transform we’ll apply during feature engineering.

3. Feature Engineering

I’ve always insisted in my previous articles that raw data can get you baseline predictions. However, feature-engineered ones are more likely to get you good predictions. Starting from ~26 raw columns, I engineered 81 features grouped into eight categories:

Feature groups:
  Temporal     : 15
  Calendar     :  2
  Events       :  5
  Weather      :  7
  Season       :  4
  Interaction  :  7
  Lag          : 14
  Rolling      : 27

Here are the most important ones:

a) Cyclical Encoding — Why Day 0 and Day 6 Should Be “Close”

Here’s a subtle problem with temporal features. If you feed day_of_week directly into a model as values 0 to 6, the model treats Monday (0) and Sunday (6) as being far apart. In reality, they're adjacent days. The same applies to months. December (12) and January (1) are close, not opposites.

The fix is cyclical encoding using sin/cos transforms:

df['dow_sin'] = np.sin(2 * np.pi * df['day_of_week'] / 7)
df['dow_cos'] = np.cos(2 * np.pi * df['day_of_week'] / 7)

# Month: 1-12 -> cyclical
df['month_sin'] = np.sin(2 * np.pi * (df['month'] - 1) / 12)
df['month_cos'] = np.cos(2 * np.pi * (df['month'] - 1) / 12)

# Day of year: 1-365 -> cyclical
df['doy'] = df['date'].dt.dayofyear
df['doy_sin'] = np.sin(2 * np.pi * df['doy'] / 365)
df['doy_cos'] = np.cos(2 * np.pi * df['doy'] / 365)

Think of it this way: you’re placing each day on the face of a clock. Monday and Sunday end up right next to each other, and December sits beside January. Both sin and cos components are needed because sin alone can’t distinguish between two days that map to the same y-coordinate (e.g., Tuesday and Friday might have the same sinvalue, but different cos values).

On the left, days of the week are mapped onto a circle via sin/cos. On the right, months on the same circular encoding. Both sin and cos components are needed to uniquely identify each position on the circle.

I applied this to day-of-week, month, day-of-year, and week-of-year.

b) Lag Features — “Yesterday Predicts Tomorrow”

Recent history turned out to be one of the most important predictive signals in this dataset. For example, if 400 people visited yesterday, chances are today will be closer to 400 than to 200. Therefore, I replicated the same lags at strategic intervals:

for lag in [1, 2, 3, 14, 21, 28, 60, 90]:
    df[f'visitors_lag_{lag}'] = df['visitors'].shift(lag)

# Same weekday average (last 4 occurrences)
df['mean_last_4_same_dow'] = df[['lag_7_same_dow','lag_14_same_dow',
                                  'lag_21_same_dow','lag_28_same_dow']].mean(axis=1)

Why these specific intervals? Lags 1 to 3 capture short-term momentum. Lag 7 captures the same-day-last-week pattern (crucial for museums where weekdays and weekends behave very differently). Lags 14, 21, and 28 capture fortnightly and monthly rhythms. Lags 60 and 90 capture seasonal trends. The same-weekday average of the last 4 weeks is particularly powerful because it answers the question: “How many people typically visit on a Wednesday?”

The autocorrelation plot from the EDA confirms why these lag choices matter. There’s a clear spike at lag 7 (the weekly cycle) and a gradual decay that persists out to 60+ days, reflecting the seasonal structure.

The ACF shows significant autocorrelation at multiples of 7 days (the weekly rhythm), while the PACF reveals that lags 1 and 7 carry the most independent predictive information, exactly the lags we prioritised in feature engineering.

c) Rolling Window Statistics

Beyond point lags, I computed rolling means, standard deviations, maximums, and minimums across multiple windows:

for window in [3, 7, 14, 30, 60, 90]:
    df[f'rolling_mean_{window}d'] = df['visitors'].shift(1).rolling(window).mean()
    df[f'rolling_std_{window}d']  = df['visitors'].shift(1).rolling(window).std()
    df[f'rolling_max_{window}d']  = df['visitors'].shift(1).rolling(window).max()
    df[f'rolling_min_{window}d']  = df['visitors'].shift(1).rolling(window).min()

# Exponentially Weighted Moving Average (EWMA) - (more weight on recent days)
df['ewma_7d']  = df['visitors'].shift(1).ewm(span=7, adjust=False).mean()
df['ewma_14d'] = df['visitors'].shift(1).ewm(span=14, adjust=False).mean()
df['ewma_30d'] = df['visitors'].shift(1).ewm(span=30, adjust=False).mean()

The shift(1) is critical here as it prevents data leakage by ensuring we only use information available before the prediction date. The EWMA variants give extra weight to recent observations, which is also useful because visitor patterns can shift gradually (e.g., after a new exhibition opens).

d) Interaction Features — When Combinations Matter

Some effects compound. For example, a sunny weekend during school holidays isn’t just “sunny” + “weekend” + “school holiday”. It’s a perfect storm that drives numbers far above what any individual factor predicts. I captured this with interaction features:

df['weekend_x_public_holiday'] = df['is_weekend'] * df['is_public_holiday']
df['exhibition_x_weekend'] = df['special_exhibition'] * df['is_weekend']
df['sunny_weekend'] = ((df['weather_type'] == 'Sunny') & (df['is_weekend'] == 1)).astype(int)
df['marketing_x_exhibition'] = df['marketing_campaign'] * df['special_exhibition']

# Count of active boost factors
df['boost_count'] = (
    df['is_public_holiday'] + df['is_school_holiday'] +
    df['special_exhibition'] + df['local_event'] +
    df['marketing_campaign'] + df['ticket_promotion']
)

The boost_count feature is a favourite of mine. It answers a simple question: “How many things are happening today that attract visitors?” Basically, a day with three active boost factors will almost certainly see more visitors than a day with one. The chart below quantifies this stacking effect.

The relationship is nearly monotonic in that each additional active factor adds roughly 50 to 100 visitors on average. Days with zero boost factors average around 300 visitors; days with 3+ factors routinely exceed 500.

e) Melbourne Seasons (Southern Hemisphere)

This might seem trivial and personal to me, but it’s the kind of thing that trips up models if you’re deriving your thought process from, say, the Northern Hemisphere. December in Melbourne is summer, not winter:

def get_season(month):
    if month in [12, 1, 2]: return 'Summer'
    elif month in [3, 4, 5]: return 'Autumn'
    elif month in [6, 7, 8]: return 'Winter'
    else: return 'Spring'

January visitors run ~40% above average (summer holidays), while July drops to ~25% below average (winter). Getting this wrong would confuse the model completely.

f) Log Transform of the Target

The visitor count distribution is right-skewed whereby most days cluster between 200 and 400, but outlier days push above 800. A log transform (log1p) makes this more normally distributed, which helps gradient-based models converge:

df['visitors_log'] = np.log1p(df['visitors'])

Left: the original distribution with a skewness of ~0.9 and the long right tail of peak days. Right: after log1p, skewness drops dramatically and the distribution becomes much closer to normal. This is the version that the models will be trained on.

This means the model trains on the log-transformed target, and predictions need to be inverse-transformed using np.expm1(). I’ll come back to this when we discuss more details on modelling and production API.

After engineering all 81 features, I checked which ones correlated most strongly with the target. Unsurprisingly, the lag and rolling features dominate the top of the list, with recent history being the single best predictor of tomorrow’s visitors as shown in the below feature correlation plot.

The rolling means and EWMA features cluster at the top with correlations close to 0.6, followed by the raw lag features. Temporal and calendar features contribute meaningfully but sit in the 0.2 to 0.4 range. This hierarchy directly informed which features the models would lean on most.

After feature engineering, the longest lag (90 days) means I drop the first 90 rows of data, leaving 1,006 usable rows (still plenty for training and evaluation).

4. Model Training & Comparison

With features ready, I trained and compared four approaches. But first, the split strategy.

Train/Test Split — Respecting Time

This is a crucial one. With time series data, you cannot use random train/test splits. If you randomly shuffle dates, the model will see future data during training and that’s data leakage, and your metrics are likely to lie to you.

Instead, I used the last 90 days as a hold-out test set, with everything before as training data. Something like the following:

Train: 916 rows  (2023-04-01 → 2025-10-02)
Test :  90 rows  (2025-10-03 → 2025-12-31)

For cross-validation during training, I used TimeSeriesSplit with 5 folds, which creates expanding windows that always train on past data and validate on future data. At least no cheating here.

The Models

Before reaching for sophisticated algorithms, I needed to answer a basic question: How good can one get without any machine learning at all? That’s what baselines are for and is a good practice. This is the same thought process that I applied here. They’re deliberately simple prediction strategies that use just arithmetics and no learned patterns. If your fancy model can’t beat these, then it’s not earning its complexity.

For the Baselines, I used three naive approaches, each slightly more informed than the last:

• Global mean: predict every day as the average of the training set (~370 visitors). This ignores everything: day of week, weather, holidays. It’s the “shrug” prediction, but it anchors the bottom of our performance range.
• Last week’s visitors (lag 7): This predicts today’s visitors as whatever happened on the same day last week. This captures the weekly rhythm (Saturdays are busy, Tuesdays are quiet) but nothing else and it can’t adapt to weather changes, holidays, or trends.
• 7-day moving average: This predicts today as the average of the past 7 days. This approach is slightly smoother than the lag-7 approach because it not only dampens single-day spikes, but also has no awareness of why visitor counts change. Basically, it just follows the recent trend.

The key limitation of all three baselines is that they use a single signal. They can’t combine information. For example, they don’t know that a sunny Saturday during school holidays with a special exhibition is a fundamentally different day from a rainy Tuesday in winter.

That’s where the ML models come in. They learn to weigh dozens of features simultaneously and capture non-linear interactions between them. I made use of the following algorithms:

1. Random Forest — an ensemble of 300 decision trees, each trained on a random subset of features (max_features='sqrt') and data. Each tree independently learns rules like "if it's a weekend AND there's a special exhibition AND temperature > 20°C, expect high traffic." The final prediction averages across all 300 trees, which smooths out individual tree quirks. The min_samples_leaf=5 constraint prevents trees from memorising noise in the training data. Like XGBoost, it was trained on the log-transformed target.
2. XGBoost with 500 sequential estimators where each new tree specifically targets the errors the previous trees got wrong (gradient boosting). Whereas Random Forest builds trees independently and averages them, XGBoost builds them one at a time, with each tree focusing on the residual mistakes. The learning rate of 0.05 keeps each tree’s contribution small, which requires more trees but produces a more robust model. I trained it on log1p(visitors) rather than raw visitor counts. This compresses the scale so the model doesn't disproportionately chase high-traffic outlier days at the expense of typical ones.
3. Prophet — This is Facebook’s time series model that takes a fundamentally different approach. Instead of learning from engineered features, it decomposes the time series into trend + seasonality + holiday effects and fits each component separately. I configured it with multiplicative seasonality (because seasonal swings scale with the baseline level) and added external regressors for holidays, exhibitions, and weather. It’s purpose-built for time series but less flexible at capturing the kind of complex feature interactions that tree-based models handle naturally.

Here’s the XGBoost training code:

xgb_model = xgb.XGBRegressor(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=6,
    subsample=0.8,
    colsample_bytree=0.8,
    min_child_weight=5,
    reg_alpha=0.1,
    reg_lambda=1.0,
    random_state=42,
    n_jobs=-1,
    verbosity=0
)
# Train on log-transformed target
xgb_model.fit(X_train, y_train_log)
# Predictions need inverse transform
xgb_preds = np.expm1(xgb_model.predict(X_test))

The code for the other two approaches can be found here.

Results

Below are the results:

| Model               | MAE   | RMSE  |   R²  |  MAPE |
|---------------------|-------|-------|------ |-------|
| XGBoost             | 69.9  | 90.7  | 0.62  | 14.0% |
| Prophet             | 74.3  | 92.7  | 0.60  | 16.3% |
| Random Forest       | 78.4  | 103.7 | 0.50  | 16.3% |
| Baseline (7d MA)    | 119.3 | 152.7 | -0.09 | 26.5% |
| Baseline (last week)| 125.0 | 162.5 | -0.24 | 27.4% |
| Baseline (mean)     | 143.9 | 189.8 | -0.68 | 26.2% |

From the results above, XGBoost was the winner across all metrics. A MAPE of 14.0% means the model is typically off by about 14%. What this typically means is that for a museum expecting 350 visitors, that’s roughly plus or minus 50 people. This is well within the range that operations teams can plan around.

Note that the negative R² values for the baselines don’t indicate a bug. A negative R² simply means the model performs worse than just predicting the mean of the test set, which confirms these baselines are too simplistic to be useful.

The improvement over baselines is substantial. The best baseline (7-day moving average) has an MAE of 119.3 visitors, while XGBoost achieves 69.9, a 41% reduction in error. That gap is the difference between useful predictions and educated guesses.

XGBoost tracks the actual visitor pattern most closely, capturing both the weekly rhythm and the December holiday surge. Random Forest follows the shape but tends to undershoot peaks. Prophet captures the broad seasonal trend but smooths over the day-to-day variability that the tree-based models pick up. It performs surprisingly well given it doesn’t use the full feature set. Its strength is in capturing the yearly and weekly seasonality automatically. However, it can’t leverage the 81 engineered features as effectively as XGBoost.

5. Model Evaluation & SHAP Explainability

Raw metrics tell you how well the model performs. However, evaluation tells you where it struggles and why.

Residual Analysis

The four-panel residual analysis below gives a comprehensive picture of the model’s error behaviour.

Top-left: residuals over time, the green and red shading shows where the model over- and under-predicts. Top-right: the residual distribution is roughly normal and centred near zero. Bottom-left: the predicted-vs-actual scatter follows the diagonal reasonably well, though the model underestimates the highest-traffic days. Bottom-right: percentage errors cluster within plus or minus 20%, with a slight negative skew.

The model has a slight negative bias of about -25 visitors/day, meaning it tends to under-predict on average. This is conservative and it’s actually better to under-promise and over-deliver when it comes to staffing. The residual distribution is roughly normal and centred near zero, which is what we want.

Error Breakdown

The model doesn’t perform equally well across all conditions:

Top row: MAE by day of week (weekends show higher error), month, and season. Bottom row: MAE split by public holidays, special exhibitions, and weekends. Public holidays show the highest MAE as they are the outlier days with unpredictable spikes.

• Weekdays vs weekends: Higher error on weekends (more variable traffic)
• Public holidays: The highest MAE. These are outlier days with unpredictable spikes
• Special exhibitions: Also higher error, but less than public holidays
• Seasons: Summer and spring (high-traffic seasons) have larger absolute errors, though percentage errors remain similar

This is expected. Extreme days are harder to predict because they’re driven by unique combinations of factors. A model that’s “off by 70 visitors” on a 300-visitor day is performing differently than one that’s “off by 70” on a 900-visitor day.

SHAP Explainability

I used SHAP (SHapley Additive exPlanations) to understand what drives the model’s predictions. This is crucial for two reasons: (1) it validates that the model is learning sensible patterns, and (2) it gives you ammunition for explaining predictions to non-technical stakeholders.

explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_shap)

shap.summary_plot(shap_values, X_shap, plot_type='dot', max_display=20)

The results confirm what you’d expect:

The beeswarm plot shows each feature’s impact on predictions. Each dot is a single prediction with colour representing the feature value (red = high, blue = low). Seasonality and day-of-week features dominate the top, followed by contextual factors like precipitation, temperature, and boost_count. Lag and rolling features contribute meaningfully but sit in the middle of the ranking.

I also generated a waterfall plot for the single worst prediction day. This is the day where the model’s prediction was furthest from the actual visitor count. The waterfall plot as shown below breaks down each feature’s contribution, showing which ones pushed the prediction up or down and by how much. This lets you diagnose why the model got it wrong: did it miss because something happened that day that no feature could capture (e.g., a surprise road closure), or because the feature values were an unusual combination it hadn’t seen during training (e.g., a sunny public holiday in winter)?

Prediction Intervals

A point prediction alone isn’t enough. Operations teams need to know the range of uncertainty. Therefore, I computed bootstrap prediction intervals using training residuals:

# Bootstrap prediction intervals
n_bootstrap = 500
bootstrap_preds = np.array([
    preds + np.random.choice(train_residuals, size=len(preds), replace=True)
    for _ in range(n_bootstrap)
])

lower_80 = np.percentile(bootstrap_preds, 10, axis=0)
upper_80 = np.percentile(bootstrap_preds, 90, axis=0)
lower_95 = np.percentile(bootstrap_preds, 2.5, axis=0)
upper_95 = np.percentile(bootstrap_preds, 97.5, axis=0)

The blue line tracks the model’s predictions against the actual visitor counts (black). The darker shaded band is the 80% interval; the lighter band is 95%. The actual values fall within these bands at close to the expected rate and this is what well-calibrated uncertainty looks like.

The intervals are well-calibrated whereby the 80% interval captures approximately 80% of actual values, and the 95% interval captures approximately 95%. This is the kind of result that builds trust with stakeholders: “We predict 380 visitors, with 80% confidence it’ll be between 310 and 450.”

It’s worth noting that this bootstrap method assumes residuals are identically distributed across all conditions. In practice, as we saw in the error breakdown, weekends and public holidays have higher error than quiet weekdays. A more refined approach would compute condition-specific intervals, but for an initial deployment this provides a useful and honest range.

6. Stakeholders Engagement

I know from experience that metrics like MAE and RMSE don’t resonate with stakeholders who may be the museum directors or the day-to-day operations people in this context. What resonates is business impact. I framed the model’s performance in two ways.

Traffic Tier Classification

Instead of reporting raw numbers, the model classifies each day into a traffic tier:

- Low: < 250 visitors
- Medium: 250–450 visitors
- High: > 450 visitors

Each tier could easily map to a staffing plan and security/safety deployment.

Left: the confusion matrix shows how often the model’s predicted tier matches reality. The diagonal dominance means operations teams can trust the signal. Right: revenue prediction errors (at $25/ticket) cluster tightly around zero, with most days falling within plus or minus $2,000 AUD.

The model’s tier accuracy gives the operations team a reliable signal: if the model says “High traffic,” they can confidently deploy the larger team.

Revenue Forecasting

At say $25 AUD per ticket, though most days the museum is free, every visitor prediction translates directly to revenue. The model’s mean daily revenue error gives the finance team a concrete number to work with for budgeting and forecasting. For a data scientist, it’s worth celebrating MAE = 69.9 visitors. However, a museum director will celebrate when they hear: “we can predict tomorrow’s ticket revenue within plus or minus $1,750 AUD.”

That wraps up Part 1 and I admit it was quite long. I promise to shorten Part 2. We’ve gone from the business problem through data generation, feature engineering, model training, evaluation, and business framing. The XGBoost model with 81 engineered features achieves a MAPE of 14.0%, production-ready for operations planning.

In Part 2, I’ll take this trained model and deploy it as a full-stack application: a FastAPI backend that serves predictions through a REST API, a React + Tailwind dashboard for the operations team, and a Supabase database for prediction logging. I’ll also cover deployment considerations and what I’d do differently with real data. This is something that I have never written about so I’m also very excited.

As always, all code is open-sourced here. Please feel free to clone it and adapt it for your own venue. If you found this useful, a clap or comment goes a long way. You can find my other articles on my profile, and I’m always happy to connect via LinkedIn.

Stop Guessing Staffing Needs: Predicting Daily Museum Visitors Before They Arrive (Part 1) was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stuck behind a firewall. Read the article for free here

It’s my fourth New Year here in Australia, a day I cherish greatly. Normally, I take this time to relax and reflect on the events of the previous year. However, this time around, I found myself reflecting deeply on my former professional life. Before working in the finance space, I actually worked in spatial analytics for a company that predicted visitor numbers in museums and exhibitions based on various factors. It was one of those roles where you spend half your time explaining to curators (at least the product owners did) why ‘the spot near the bathrooms’ in a mall or museum isn’t actually premium real estate, despite the foot traffic. The other half was spent building models that some clients didn’t fully trust until they saw the results in action. It was fascinating work, and I genuinely enjoyed it.

Fast forward to today, and I realized I hadn’t touched spatial analytics in such a long time. I’ll still attribute this to being busy riding the Generative AI (Gen AI) wave, plus the usual modeling work that fills my days. I managed to write a few Gen AI articles and some predictive modeling pieces to justify the time spent on it. Many of them can be found here. Please read them for free and share your feedback. As always, all code accompanying each article is open-sourced, so feel free to re-implement it for your own problems.

For this reason, I decided to revisit spatial analytics once again, but with a twist. For those who may not be familiar with this use case, it’s simply the process of analyzing foot traffic patterns to identify the best locations for premium advertisements or exhibitions in malls and museums. For example, a jewelry shop or museum wanting to place a high-value exhibition needs to understand which locations actually deliver value. To accomplish this, they must understand not just how many people pass through a location, but how they move, where they linger, and which paths they take. Therefore, this will always be venue-specific and cannot be generalized. Additionally, they should have a way of measuring the success of their placement decisions. Don’t worry if this sounds complicated. I’ll break it down for you in the simplest terms possible with a relevant example.

The Problem with “Premium” Space

Here’s the uncomfortable truth: most “premium advertising space” decisions in malls/museums/exhibitions are still made the old-fashioned way. Someone senior walks the venue, points at a corner, and says, “This feels busy.” And to be fair, sometimes that intuition is correct. But it’s also the reason why premium placements become political. The leasing team thinks one spot is best, the tenants think another spot is best, and the marketing team is somewhere in the middle trying to justify the budget.

Now, if your venue offers free WiFi and (anonymously) tracks movement, you already have what many businesses dream of: behavioral data at scale. The trick is using it in a way that is statistically defensible, explainable to non-technical stakeholders, and respectful of privacy.

Why This Approach is Different

I’ll be honest: when I worked in spatial analytics, most of the analysis I saw treated locations as independent silos. Basically, all that was done was to count visitors, measure dwell time and call it a day. But that approach misses something fundamental. The real value of a location depends on:

1. The network structure — the zones that connect to it
2. Flow patterns — whether it’s a destination or a pass-through
3. Movement dynamics — how people actually navigate the space
4. Temporal patterns — when traffic peaks occur

This is why I built my solution around a graph-inspired process rather than simple aggregation. Here, the venue e.g., museum or mall is modeled as a network of connected zones, and visitor movement is analyzed as paths through this network. This approach captures something that simple traffic counts miss: a location’s value isn’t just about how many people stop there, but also about being on high-traffic paths and serving as a hub.

Think of it this way: a corridor with low dwell time but massive throughflow might be more valuable than a quiet corner where people linger. Both matter, but for different reasons and to different advertisers.

Dataset Structure & The WiFi Pings Reality

I came up with the below approach that would generally be used by venues with WiFi tracking capabilities. However, the implementation is not exhaustive and should NOT be adopted as is in production environments. Its subdivided as follows:

a) Venue Graph Structure

The first step is modeling the physical space as a graph where nodes represent zones (entrances, corridors, food courts, galleries etc.) and edges represent walkable connections between the zones. Therefore, each zone has coordinates (x, y) for spatial visualization. This is fundamentally different from treating each location independently.

Below is the exact code I used to generate a mock venue graph (zones + edges) and simulate WiFi pings.

import numpy as np
import pandas as pd

def generate_mock_venue(seed=7):
    """
    Creates:
      - zones (a simplified mall/museum graph)
      - wifi pings: device_id, timestamp, zone_id, x, y, rssi(received signal strength indicator)
      - optional POIs / candidate ad locations
    """
    rng = np.random.default_rng(seed)

    # ----- Venue zones (nodes) -----
    # Think of each zone as a "Wi-Fi localization cell" or area (corridor section / gallery / food court).
    zones = pd.DataFrame([
        {"zone_id": "ENT_N", "name": "North Entrance", "type": "entrance", "x": 10, "y": 90},
        {"zone_id": "ENT_S", "name": "South Entrance", "type": "entrance", "x": 10, "y": 10},
        {"zone_id": "COR_1", "name": "Main Corridor 1", "type": "corridor", "x": 30, "y": 70},
        {"zone_id": "COR_2", "name": "Main Corridor 2", "type": "corridor", "x": 50, "y": 50},
        {"zone_id": "COR_3", "name": "Main Corridor 3", "type": "corridor", "x": 70, "y": 30},
        {"zone_id": "FOOD",  "name": "Food Court", "type": "food", "x": 80, "y": 80},
        {"zone_id": "ANCH_A","name": "Anchor Store A", "type": "anchor", "x": 90, "y": 60},
        {"zone_id": "ANCH_B","name": "Anchor Store B", "type": "anchor", "x": 90, "y": 20},
        {"zone_id": "GALL_1","name": "Gallery 1", "type": "gallery", "x": 40, "y": 85},
        {"zone_id": "GALL_2","name": "Gallery 2", "type": "gallery", "x": 60, "y": 85},
        {"zone_id": "REST",  "name": "Restrooms", "type": "utility", "x": 65, "y": 10},
        {"zone_id": "SEAT",  "name": "Seating / Atrium", "type": "amenity", "x": 55, "y": 55},
    ])

 
    # zone adjacency edges - think of this as your "walkable graph" approximation. Remember this is hypothetical!
    edges = [
        ("ENT_N", "COR_1"), ("ENT_S", "COR_3"),
        ("COR_1", "COR_2"), ("COR_2", "COR_3"),
        ("COR_2", "SEAT"),
        ("COR_1", "GALL_1"), ("GALL_1", "GALL_2"), ("GALL_2", "FOOD"),
        ("COR_2", "ANCH_A"), ("COR_3", "ANCH_B"),
        ("COR_3", "REST"),
        ("FOOD", "ANCH_A"),
    ]

    # Convert to adjacency map for simulation
    adj = {}
    for a, b in edges:
        adj.setdefault(a, []).append(b)
        adj.setdefault(b, []).append(a)

    # ----- Simulate device journeys -----
    n_devices = 1500
    # sessions per device (some repeat visitors)
    sessions_per_device = rng.choice([1, 2, 3], size=n_devices, p=[0.72, 0.22, 0.06])

    # Campaign day timeline
    start = pd.Timestamp("2025-12-01 09:00:00")
    end   = pd.Timestamp("2025-12-01 21:00:00")
    total_minutes = int((end - start).total_seconds() // 60)

    # Entry zone bias: more people use south entrance
    entry_zones = ["ENT_N", "ENT_S"]
    entry_p = [0.4, 0.6]

    # Zone "stickiness" (higher -> more dwell)
    stickiness = {
        "FOOD": 0.65, "SEAT": 0.55, "GALL_1": 0.45, "GALL_2": 0.45,
        "ANCH_A": 0.35, "ANCH_B": 0.35,
        "REST": 0.25,
        "COR_1": 0.15, "COR_2": 0.15, "COR_3": 0.15,
        "ENT_N": 0.10, "ENT_S": 0.10
    }

    def simulate_session(device_id: str):
        # start time
        t0 = start + pd.Timedelta(minutes=int(rng.integers(0, total_minutes)))
        # session length (minutes)
        duration = int(rng.normal(75, 25))
        duration = int(np.clip(duration, 20, 180))
        t1 = t0 + pd.Timedelta(minutes=duration)

        # pick entry
        zone = rng.choice(entry_zones, p=entry_p)

        # simulate movement in 1-minute steps, later we can downsample to "pings"
        times = pd.date_range(t0, t1, freq="1min")
        zones_path = []
        for _ in times:
            zones_path.append(zone)
            # decide whether to stay
            if rng.random() < stickiness.get(zone, 0.2):
                continue
            # else move to a neighbor
            nbrs = adj.get(zone, [])
            if not nbrs:
                continue
            # slight bias toward anchors/food places - chances are that stays are longer in such locations 
            weights = []
            for n in nbrs:
                w = 1.0
                if n in ["FOOD", "ANCH_A", "ANCH_B"]:
                    w *= 1.35
                if n in ["GALL_1", "GALL_2"]:
                    w *= 1.15
                weights.append(w)
            weights = np.array(weights) / np.sum(weights)
            zone = rng.choice(nbrs, p=weights)

        df = pd.DataFrame({"device_id": device_id, "timestamp": times, "zone_id": zones_path})

        # add (x,y) with noise around zone centers
        zmap = zones.set_index("zone_id")[["x", "y"]].to_dict("index")
        df["x"] = df["zone_id"].map(lambda z: zmap[z]["x"] + rng.normal(0, 2.5))
        df["y"] = df["zone_id"].map(lambda z: zmap[z]["y"] + rng.normal(0, 2.5))

        # add RSSI proxy (more negative is weaker)
        # assume closer to AP in zone -> stronger; we just simulate plausible values
        df["rssi"] = -35 - (np.abs(rng.normal(0, 7, size=len(df))) + rng.uniform(0, 10, size=len(df)))
        return df

    all_sessions = []
    for i in range(n_devices):
        device_id = f"d_{i:05d}"
        for s in range(sessions_per_device[i]):
            all_sessions.append(simulate_session(device_id))

    traces = pd.concat(all_sessions, ignore_index=True)

    # ----- Downsample to "wifi pings" -----
    # Real Wi-Fi localization is irregular; simulate that by sampling each device ~ every 2-5 minutes.
    traces["minute"] = traces["timestamp"].dt.floor("min")
    # keep ping with probability dependent on random interval
    keep = np.zeros(len(traces), dtype=bool)
    for dev, g in traces.groupby("device_id", sort=False):
        step = int(rng.integers(2, 6))
        keep_idx = g.iloc[::step].index
        keep[keep_idx] = True
    pings = traces.loc[keep].drop(columns=["minute"]).reset_index(drop=True)

    # Candidate ad/exhibit locations: pick zones that can host placements
    candidates = zones[zones["type"].isin(["corridor","food","amenity","gallery"])].copy()
    candidates["is_candidate"] = True

    return zones, pd.DataFrame(edges, columns=["from_zone","to_zone"]), pings, candidates

zones, edges, pings, candidates = generate_mock_venue(seed=7)

print("zones:", zones.shape)
print("edges:", edges.shape)
print("pings:", pings.shape)
print("candidates:", candidates.shape)

pings.head()

By encoding the graph structure, we can measure connectivity and flow, not just presence. This is crucial for understanding how zones relate to each other.

b) WiFi Ping Data

Real WiFi tracking data is irregular and sparse, making it quite messy to deal with. Devices don’t ping at consistent intervals. Connection quality varies. People also turn WiFi on and off.

In the code above, I simulated movement in 1-minute steps and then downsampled it into irregular pings (every ~2–5 minutes per device). That last part is important because it mimics what WiFi data is like under normal circumstances.

The “stickiness” is another crucial parameter. Food courts naturally have higher stickiness (people sit and eat). On the other hand, places like corridors have low stickiness (people pass through). This creates realistic heterogeneity in the data and yes, it’s based on patterns I observed in museums and exhibition spaces.

c) Session Reconstruction from Irregular Pings

As mentioned above, WiFi pings are irregular, so I needed to reconstruct them to coherent sessions. I used time gaps to identify when a device has left and returned. This is how actual WiFi analytics systems work.

Here’s the exact sessionisation function I used:

def build_sessions(pings: pd.DataFrame, inactivity_gap_minutes=15):
    """
    Convert irregular pings into sessions per device based on time gaps.
    """
    df = pings.sort_values(["device_id","timestamp"]).copy()
    df["prev_ts"] = df.groupby("device_id")["timestamp"].shift(1)
    df["gap_min"] = (df["timestamp"] - df["prev_ts"]).dt.total_seconds() / 60.0
    # new session if first ping or big inactivity gap
    df["new_sess"] = (df["prev_ts"].isna()) | (df["gap_min"] > inactivity_gap_minutes)
    df["session_id"] = df.groupby("device_id")["new_sess"].cumsum()
    df["session_key"] = df["device_id"] + "_s" + df["session_id"].astype(str)
    return df.drop(columns=["prev_ts","gap_min","new_sess","session_id"])

You can’t assume continuous tracking and sessions need to be inferred from sparse observations. Therefore, I added a 15-minute inactivity threshold as a reasonable default. However, you should tune this value based on your venue’s typical visit duration.

The Metrics That Actually Matter

This is where this approach gets a bit interesting. I put together four categories of metrics that overall paint a complete picture of each location’s advertising value. These aren’t random choices but what advertisers actually care about when evaluating placement. To keep it clean, I calculate most zone metrics in one function, then calculate flow metrics separately (because that depends on transitions).

1. Reach Metrics

This is basic but essential. More eyeballs equals more potential impressions. This is the foundation of any advertising value calculation.

Here’s the exact metric function I used (reach + dwell + repeat + peakiness):

def compute_zone_metrics(sessionized: pd.DataFrame, zones: pd.DataFrame, time_bucket="15min"):
    """
    Computes:
      - unique_reach: unique devices per zone
      - dwell_minutes: approx dwell per device-zone using time deltas
      - repeat_rate: how often devices return to the zone within a session
      - peakiness: how concentrated the traffic is in time (good for timed campaigns)
    """
    df = sessionized.sort_values(["session_key","timestamp"]).copy()

    # time delta to next ping within session; cap to avoid huge dwell from sparse pings
    df["next_ts"] = df.groupby("session_key")["timestamp"].shift(-1)
    df["dt_min"] = (df["next_ts"] - df["timestamp"]).dt.total_seconds() / 60.0
    df["dt_min"] = df["dt_min"].clip(lower=0, upper=8)  # cap dwell contribution

    # reach
    reach = df.groupby("zone_id")["device_id"].nunique().rename("unique_reach")

    # dwell time per zone (sum of dt_min)
    dwell = df.groupby("zone_id")["dt_min"].sum().rename("dwell_minutes")

    # average dwell per device (attention proxy)
    dwell_per_device = (df.groupby(["zone_id","device_id"])["dt_min"].sum()
                          .groupby("zone_id").mean()
                          .rename("avg_dwell_per_device_min"))

    # repeat exposure in-session: count visits to zone within a session
    visits = (df.groupby(["session_key","zone_id"]).size()
                .rename("pings_in_zone")
                .reset_index())
    repeat = visits.groupby("zone_id")["pings_in_zone"].apply(lambda x: (x > 1).mean()).rename("repeat_rate")

    # peakiness: bucket traffic in time and compute coefficient of variation
    df["bucket"] = df["timestamp"].dt.floor(time_bucket)
    bucket_counts = df.groupby(["zone_id","bucket"])["device_id"].nunique().reset_index(name="u")
    peakiness = bucket_counts.groupby("zone_id")["u"].agg(["mean","std"])
    peakiness["peakiness_cv"] = (peakiness["std"] / peakiness["mean"]).replace([np.inf, -np.inf], np.nan).fillna(0)
    peakiness = peakiness["peakiness_cv"]

    out = pd.concat([reach, dwell, dwell_per_device, repeat, peakiness], axis=1).fillna(0).reset_index()
    out = out.merge(zones[["zone_id","name","type"]], on="zone_id", how="left")
    return out

2. Engagement Metrics (Dwell Time)

The 8-minute cap is important. Sparse pings can create artificially long dwell times. Someone sitting in the food court for 45 minutes might only appear in the data twice, creating a huge apparent gap. The cap prevents these outliers from dominating the analysis. This is one of those things you learn the hard way when working with real WiFi data. As you can see in the code above, dwell is derived from the dt_min deltas and clipped to 8 minutes per ping interval.

3. Flow Metrics

This is where the graph-inspired approach pays off. I measured throughflow which is just how many transitions involve each zone. This captures something simple traffic counts is likely to miss.

Below is the exact flow function I used. It returns both the transition table T and per-zone flow metrics:

def compute_flow_metrics(sessionized: pd.DataFrame, edges: pd.DataFrame):
    """
    Build directed transitions between zones and compute:
      - throughflow: total transitions in + out for each zone
      - connectivity: number of distinct zones people come from and go to (observed)
    Returns:
      - T: transition table (from_zone, to_zone, n)
      - flow: per-zone flow metrics (zone_id, inflow, outflow, throughflow, connectivity)
    """
    df = sessionized.sort_values(["session_key", "timestamp"]).copy()
    df["prev_zone"] = df.groupby("session_key")["zone_id"].shift(1)

    trans = df.dropna(subset=["prev_zone"]).copy()
    trans = trans[trans["prev_zone"] != trans["zone_id"]]

    # Transition counts
    T = (
        trans.groupby(["prev_zone", "zone_id"]).size()
        .rename("n")
        .reset_index()
        .rename(columns={"prev_zone": "from_zone", "zone_id": "to_zone"})
    )

    # Inflow/outflow/throughflow
    outflow = T.groupby("from_zone")["n"].sum().reset_index().rename(
        columns={"from_zone": "zone_id", "n": "outflow"}
    )
    inflow = T.groupby("to_zone")["n"].sum().reset_index().rename(
        columns={"to_zone": "zone_id", "n": "inflow"}
    )

    flow = pd.merge(inflow, outflow, on="zone_id", how="outer").fillna(0)
    flow["throughflow"] = flow["inflow"] + flow["outflow"]

    # Connectivity proxy: distinct previous + distinct next (observed)
    distinct_next = T.groupby("from_zone")["to_zone"].nunique().reset_index().rename(
        columns={"from_zone": "zone_id", "to_zone": "distinct_next_zones"}
    )
    distinct_prev = T.groupby("to_zone")["from_zone"].nunique().reset_index().rename(
        columns={"to_zone": "zone_id", "from_zone": "distinct_prev_zones"}
    )

    conn = pd.merge(distinct_prev, distinct_next, on="zone_id", how="outer").fillna(0)
    conn["connectivity"] = conn["distinct_prev_zones"] + conn["distinct_next_zones"]

    flow = pd.merge(flow, conn[["zone_id", "connectivity"]], on="zone_id", how="left").fillna(0)

    return T, flow

Why this matters: A corridor might have low dwell time but massive throughflow. For instance, if 5,000 people walk through it daily, that’s 5,000 advertising impressions, even if each person only spends 30 seconds there. This is the difference between a destination and a pathway, and both have value.

4. Repeat Exposure

High repeat rate means it’s a hub people return to, or it’s unavoidable (like a main corridor). This is good for brand reinforcement. If someone passes your ad three times during their visit, that’s more valuable than one pass as they are likely to have an idea of what is being advertised. Repeat rate is also computed in compute_zone_metrics() above via “pings per session per zone”.

5. Temporal Peakiness

Coefficient of variation tells you if traffic is concentrated (high CV) or spread out (low CV). High peakiness is great for timed campaigns or events. Low peakiness means consistent exposure throughout the day. This is also included in compute_zone_metrics() above via peakiness_cv.

The Scoring Formula

With all these metrics, I combine them into a single “premium score” (for lack of a better term) using weighted aggregation. This is where art meets science, because the weights reflect what different stakeholders care about.

Before scoring, I first combine everything into one dataset (sessionize → compute metrics → merge flow). Here’s exactly how I do that:

sessionized = build_sessions(pings, inactivity_gap_minutes=15)
zone_metrics = compute_zone_metrics(sessionized, zones)
T, flow_metrics = compute_flow_metrics(sessionized, edges)

zone_metrics = zone_metrics.merge(flow_metrics[["zone_id","throughflow","connectivity"]], on="zone_id", how="left").fillna(0)

zone_metrics.sort_values("unique_reach", ascending=False).head(10)

Now for the scoring.

Min-Max Normalization

Without normalization, raw numbers would completely dominate dwell times (in minutes). Normalization brings everything to the same scale [0, 1], so the weights actually mean something.

Here’s the exact normalization function I used:

def minmax(s: pd.Series):
    if s.max() == s.min():
        return pd.Series(np.zeros(len(s)), index=s.index)
    return (s - s.min()) / (s.max() - s.min())

Weighted Scoring + Ranking

I used this ranking function that includes the justification string so you can defend the output with stakeholders.

def rank_candidates(zone_metrics: pd.DataFrame, candidates: pd.DataFrame,
                    weights=None):
    if weights is None:
        weights = {
            "unique_reach": 0.35,
            "avg_dwell_per_device_min": 0.30,
            "throughflow": 0.25,
            "repeat_rate": 0.10
        }

    df = zone_metrics.merge(candidates[["zone_id","is_candidate"]], on="zone_id", how="left")
    df = df[df["is_candidate"] == True].copy()

    # Normalize each component so scales don't dominate
    for k in weights.keys():
        df[f"{k}_norm"] = minmax(df[k].astype(float))

    df["premium_score"] = 0.0
    for k, w in weights.items():
        df["premium_score"] += w * df[f"{k}_norm"]

    # Justification strings for each zone (top drivers)
    def explain_row(r):
        parts = []
        parts.append(f"Reach={int(r['unique_reach'])} devices")
        parts.append(f"Avg dwell={r['avg_dwell_per_device_min']:.2f} min/device")
        parts.append(f"Flow={int(r['throughflow'])} transitions")
        parts.append(f"Repeat={r['repeat_rate']:.2%}")
        return " | ".join(parts)

    df["justification"] = df.apply(explain_row, axis=1)
    return df.sort_values("premium_score", ascending=False).reset_index(drop=True), weights

ranked, used_weights = rank_candidates(zone_metrics, candidates)
ranked[["zone_id","name","type","premium_score","justification"]].head(10)

Why these weights?

• Reach (35%): More people equals more value. This is the foundation.
• Dwell time (30%): Longer exposure equals better ad absorption. Quality matters.
• Throughflow (25%): Being on high-traffic paths matters even without long dwell.
• Repeat (10%): Nice bonus for reinforcement, but secondary to the above.

These weights are not arbitrary. They reflect what advertisers actually care about: impressions (reach), attention (dwell), and position (flow). You can adjust them based on your specific use case. For a quick-impact ad campaign (think movie posters), you might increase throughflow weight. For detailed product displays, increase dwell time weight.

Results and Justifications

Running this on the simulated dataset, the top locations emerge with clear justifications.

Typical outputs are likely to look like this:

zone_id  name              premium_score  justification
COR_2    Main Corridor 2   0.89          Reach=1247 devices | Avg dwell=2.30 min/device | Flow=8234 transitions | Repeat=45%
FOOD     Food Court        0.85          Reach=1089 devices | Avg dwell=5.80 min/device | Flow=6421 transitions | Repeat=38%
SEAT     Seating/Atrium    0.82          Reach=1156 devices | Avg dwell=4.20 min/device | Flow=5892 transitions | Repeat=52%
COR_1    Main Corridor 1   0.79          Reach=1198 devices | Avg dwell=1.80 min/device | Flow=7456 transitions | Repeat=41%
GALL_1   Gallery 1         0.74          Reach=892 devices | Avg dwell=3.90 min/device | Flow=4231 transitions | Repeat=29%

What this tells us:

1. Main corridors dominate despite low dwell time. Throughflow is massive. These are your “billboard on the highway” locations.
2. Food Court balances reach and engagement. People linger here. This is perfect for detailed product displays or interactive kiosks.
3. Atrium/Seating is a hub with good repeat exposure. People return to rest, wait for friends, etc.
4. Galleries have engaged audiences but lower overall traffic. They could be great for niche products or high-value exhibitions.

This is the kind of insight you can bring to a leasing meeting. Instead of “this corner feels busy,” you can say something like “Main Corridor 2 has 8,234 daily transitions with 1,247 unique visitors and 45% repeat exposure within sessions.”

Visualization: Making It Tangible

One thing I learned from my museum analytics days is that business stakeholders love spatial visualizations. Numbers are abstract. Maps are the real deal to them.

You can adopt this simple function for the plots:

import matplotlib.pyplot as plt

def plot_zone_scores(zones, ranked, top_n=8):
    df = zones.merge(ranked[["zone_id","premium_score"]], on="zone_id", how="left").fillna(0)

    plt.figure(figsize=(8,6))
    plt.scatter(df["x"], df["y"], s=200, alpha=0.6)
    for _, r in df.iterrows():
        plt.text(r["x"]+0.8, r["y"]+0.8, r["zone_id"], fontsize=9)

    # highlight top
    top = ranked.head(top_n).merge(zones, on="zone_id", how="left")
    plt.scatter(top["x"], top["y"], s=600, alpha=0.35)

    plt.title("Venue zones + highlighted top premium placement candidates")
    plt.xlabel("x")
    plt.ylabel("y")
    plt.show()

plot_zone_scores(zones, ranked, top_n=6)

The below spatial plot immediately shows which zones are physically clustered and which are isolated.

Combined with the scores, it helps identify strategic placement patterns. For example, you might notice that all your top zones are in one wing of the mall. This is useful information for planning advertising campaigns or tenant mix.

Privacy and Ethics

Before we go further, let me be very clear about privacy. This is not optional. Do it wrong and you’ll have lawsuits, not insights.

What you must do:

• Only track anonymous device IDs (MAC addresses, hashed)
• Aggregate data before analysis (no individual tracking)
• Clear signage about WiFi analytics
• Compliance with GDPR/local privacy laws
• Data retention policies (delete raw pings after aggregation)

What you MUST NEVER do:

• Track specific individuals
• Link WiFi data to payment data or personal info
• Share raw movement data with third parties
• Use data for purposes beyond what was disclosed

The general rule: if you can’t explain your data collection to a concerned parent or privacy advocate, you shouldn’t be doing it.

Future Directions

This approach is solid, but there’s always room for improvement. Here are extensions I’d consider in future iterations:

1. Demographic Proxies

Device types can serve as demographic indicators. For example:

• > iPhone 15 Pro → likely affluent (I could be biased on this).
• Older Android models → budget-conscious
• Tablets → families with children

You can’t know for certain, but probabilistic inference adds value.

2. Path Analysis

Instead of just zone-level metrics, analyze common journeys:

• Entry → Corridor → Food Court → Exit
• Entry → Gallery 1 → Gallery 2 → Exit

This reveals which advertising sequences work (billboard in corridor → detailed display in gallery).

3. Time-of-Day Patterns

Different hours have different audiences:

• Morning: commuters, coffee runs
• Midday: lunch crowd, families
• Evening: diners, entertainment seekers

Adjust placement recommendations by time of day.

4. A/B Testing Zones

Actually place ads and measure conversion:

• QR code scans
• App downloads
• Coupon redemptions

Feed this back into your model. Real performance beats all predictions.

5. Causal Modeling

What if an ad was placed here? This is uplift thinking applied to spatial analytics. Estimate the causal effect of placement on engagement, not just correlation. May be its time you revisit my earlier uplift modelling article here.

Conclusion

That’s it for now. I’ve taken you through the entire process of building a graph-inspired WiFi analytics system for identifying premium advertising locations in any space. We’ve moved from simple traffic counting to sophisticated network analysis, capturing not just where people are, but how they move, why it matters, and what it’s worth.

The key insight: location value is about more than traffic volume. It’s about the intersection of reach, engagement, flow, and connectivity. A low-traffic zone on a critical path can be more valuable than a high-traffic dead end. Both matter, but for different reasons and to different advertisers.

I hope this walkthrough was also useful. Don’t forget to follow me, clap for me, and leave a comment. Let me know if you’d like me to extend this to causal analysis (what-if scenarios for different layouts) or multi-venue comparisons. If you want to check out my other articles, you can find them on my profile. I’m also happy to connect via LinkedIn.

Technical Note: All data is simulated for educational purposes. Real deployments require proper WiFi infrastructure, privacy compliance, and ground-truth validation.

Finding the Perfect Spot: WiFi Analytics for “Premium” Space Advertising was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

From predicting who might leave to targeting who you can actually save. Using uplift to focus the budget on “persuadables,” not “sure things” or “lost causes.”

In my previous article, I explored propensity-to-churn modelling in a simulated banking setting, a small love letter to classical Machine Learning (ML) after a year of GenAI. The piece is long (I know 😅), but worth your time, so I recommend reading it first. For the “stubborn ones,” here’s the gist: I showed how a bank could simulate customer data, train logistic regression and gradient boosting models, and rank customers by churn risk. If you’d like the full story, you can read it here. It’s a free link and no subscription required

A friend (a lawyer, working hard to understand AI) told me to simplify my writing for a wider reach. Fair point. So in this article, I’ll keep things straightforward and simple, as I’m genuinely trying to make technical concepts accessible to everyone. Back to customer churn.

Prediction is normally just half the story. Yes, we can estimate who is likely to leave. But the real product question is:

👉 “If I intervene, will it actually make a difference?”

That’s where an interesting concept called uplift modelling/analysis comes in, moving us from predictions to actions.

What is Uplift Analysis?

Uplift analysis is a measure of the causal impact of an intervention. Basically, instead of asking, “Who will churn?” we ask, “Who can we save because we intervened?”

Why uplift analysis matters: Hypothetically, if your churn model flags 100 “high-risk” customers and you blast all of them with discounts, there is a very high chance that you’ll waste budget on many who would stay anyway. In addition, there is also a very high chance that you’ll still miss those who only stay if you actually help them.

In uplift and with specificity to the banking scenario in my previous article, we think of these classes of customers in four plain-English segments:

• Persuadables who will churn (leave the bank in our case) if we do nothing but stay if we intervene. (Sweet spot.)
• Sure Things will stay regardless. (No need to spend anything here.)
• Lost Causes will leave regardless of the treatment. (Acknowledge and move on.)
• Sleeping Dogs—would have stayed, but intervention pushes them away. I sometimes feel like I’m a “Sleeping Dog” with a number of my utility providers. I’m always waiting for the trigger to drop them. Basically, they contact me, and I’ll “remember” the bad experience I had with them and, of course, leave. Seems like they know it, as no “annoying” deals have come through just yet.

This framing prevents “blanket offers” and actually focuses the budget where it actually changes outcomes.

Why Uplift Makes More Sense Than Just Propensity

Typically, a propensity score would say something like, “Customer A has an 80% chance of churning (leaving).” I acknowledge that this is helpful but also incomplete. It usually doesn’t say whether an SMS, fee waiver, email, or phone call is likely to change that outcome.

Therefore, uplift models capture the difference between:

1. Churn probability if treated (sent an SMS, emailed or apply a loyalty credit) and
2. Churn probability if not treated (nothing is done)

That difference is the uplift score. The conclusion would be something like:

• Positive uplift → treatment helps (good to target).
• Negative uplift → treatment hurts (avoid!).

Reusing the Churn Dataset (and of course Extending It Slightly)

In the previous churn article, I simulated a realistic-ish retail-bank dataset with features covering demographics, digital engagement, balances, loans, etc. To keep things simple here, I’ll extend the same CSV by adding a treatment flag that represents a retention offer (e.g., fee waiver, loyalty bonus, branch call).

In real life, you’d randomize who gets the offer during a small pilot. Randomization makes uplift measurement honest and simple.

I’ll add a few essential code snippets here for you to run and replicate the additional columns and uplift computation. I’ll keep the code minimal and readable for everyone. As usual, the full notebook with more extensive steps is on my GitHub.

1. Addition of a Treatment Flag

Assuming the dataset is the same as the one generated when the main.py code is run here, then we’ll simulate a 50/50 randomized offer and a mild “it helps” effect for illustration purposes. This will be different based on different business requirements in real-life scenarios. Therefore, do not use this approach on real production data.

import numpy as np
import pandas as pd

df = pd.read_csv("equity_bank_churn_dataset.csv")
np.random.seed(42)

# 1) Randomized treatment - remember this is for demonstration purposes 
df["treatment"] = np.random.binomial(1, 0.5, len(df))

# 2) Simple treated-world probability (reduce churn prob by ~10 points if treated)
df["churn_probability"] = pd.to_numeric(df["churn_probability"], errors="coerce").fillna(0.0)
TREATMENT_EFFECT = -0.10  # negative because churn is "bad"
df["adj_churn_prob"] = (df["churn_probability"] + TREATMENT_EFFECT * df["treatment"]).clip(0, 1)

# 3) Synthetic treated outcome - for learning purposes; in reality you observe only one outcome)
df["churned_treated"] = np.random.binomial(1, df["adj_churn_prob"]).astype(int)

# Observed outcome for this tutorial:
# - control customers keep their original 'churned' label
# - treated customers use the synthetic treated outcome
label_col = "churned" if "churned" in df.columns else "churn"
df["outcome"] = np.where(df["treatment"]==1, df["churned_treated"], df[label_col]).astype(int)

⚠ Please don’t fabricate outcomes in production settings. Run a small randomized pilot so your treated vs. control outcomes are real.

2. Two Small Models (Classical + Explainable)

Computing the uplift involves training one model to predict churn for treated customers and another for control customers. Then, a prediction of both scenarios is made for everyone, and the difference is the uplift. As simple as it sounds, that’s the whole idea behind uplift analysis. The code snippet below helps illustrate this:

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

# Keep it simple: auto-detect features from theCSV
DROP = {"customer_id","account_open_date","churn_date",
        "churn_probability","churned","churn","churn_flag",
        "treatment","adj_churn_prob","churned_treated","outcome"}

cat_cols, num_cols = [], []
for c in df.columns:
    if c in DROP: 
        continue
    (cat_cols if df[c].dtype=="object" else num_cols).append(c)

pre = ColumnTransformer([
    ("cat", Pipeline([("imp", SimpleImputer(strategy="most_frequent")),
                      ("ohe", OneHotEncoder(handle_unknown="ignore", sparse_output=False))]), cat_cols),
    ("num", Pipeline([("imp", SimpleImputer(strategy="median")),
                      ("sc", StandardScaler())]), num_cols),
])

# Split into treated/control
df_treat = df[df["treatment"]==1].copy()
df_ctrl  = df[df["treatment"]==0].copy()

X_treat, y_treat = df_treat[cat_cols+num_cols], df_treat["outcome"]
X_ctrl,  y_ctrl  = df_ctrl[cat_cols+num_cols],  df_ctrl["outcome"]

m_treat = Pipeline([("pre", pre), ("clf", LogisticRegression(max_iter=1000, class_weight="balanced"))]).fit(X_treat, y_treat)
m_ctrl  = Pipeline([("pre", pre), ("clf", LogisticRegression(max_iter=1000, class_weight="balanced"))]).fit(X_ctrl,  y_ctrl)

# Predict both worlds for everyone
X_all = df[cat_cols+num_cols]
df["p_treat"] = m_treat.predict_proba(X_all)[:,1]   # P(churn | treat)
df["p_ctrl"]  = m_ctrl.predict_proba(X_all)[:,1]    # P(churn | no treat)
df["uplift"]  = df["p_ctrl"] - df["p_treat"]        # positive => good to treat

This is the core: uplift = p(no-treat) − p(treat).

Higher uplift → bigger expected benefit from intervening.

3. Turn Scores Into Actions (The Four Segments)

We’ll bucket customers with simple, explainable rules that can, of course, be tweaked, as well as thresholds changed to match your budget and risk appetite.

# Simple quantile thresholds 
up_pos = df["uplift"].quantile(0.70)   # clearly positive uplift
up_neg = df["uplift"].quantile(0.10)   # clearly negative uplift
r_hi   = df["p_ctrl"].quantile(0.60)   # genuinely at risk without treatment
r_lo   = df["p_ctrl"].quantile(0.20)   # pretty safe already

def segment(u, r):
    if u <= up_neg: return "Sleeping Dog"          # treatment backfires
    if (u >= up_pos) and (r >= r_hi): return "Persuadable"
    if (r <= r_lo) and (up_neg < u < up_pos): return "Sure Thing"
    if (r >= r_hi) and (up_neg < u < up_pos): return "Lost Cause"
    return "Gray Zone" # pilot group that you can learn from 

df["segment"] = [segment(u, r) for u, r in zip(df["uplift"], df["p_ctrl"])]

df["action"] = df["segment"].map({
    "Persuadable": "Target (intervene)",
    "Sure Thing": "Do not treat",
    "Lost Cause": "Do not treat (low ROI)",
    "Sleeping Dog": "Avoid (may backfire)",
    "Gray Zone": "Test small / learn"
})

Reading this as a product owner, and I know some of you are:

• Treat persuadables first (money well spent).
• Skip Sure Things (they stay anyway).
• Skip Lost Causes (hard to move).
• Exclude Sleeping Dogs (don’t poke the bear!).
• Gray Zone—run tiny tests and learn from their behaviour. You can then decide what to do with them.

4. “Does This Pay?”

This is a fundamental question that the business is likely to ask you about. I’d answer it this way. If the average Customer Lifetime Value (CLV) is CLV and the treatment cost per customer is COST, an expected profit can be formulated as follows:

Profitᵢ ≈ upliftᵢ × CLV − COST

What this actually means:

• upliftᵢ​: the estimated drop in churn probability for customer i if treated (e.g., P_ctrl − P_treat​). This will be a number between 0 and 1.
• CLV: is the net customer lifetime value (in money) you gain by keeping a customer. This could be defined differently depending on the business case.
• COST: the per-customer cost of the intervention (SMS, call, discount, emails, etc.)

Because upliftᵢ​ is a probability, upliftᵢ × CLV is the expected monetary value of saving that customer. The product minus the COST gives the expected profit of keeping the customer.

• If Profitᵢ > 0, then it's worth treating.
• If Profitᵢ<0, then there is no need of keeping them to be treated.

I’ll use an example here to illustrate this:

Suppose the Upliftᵢ = 0.25 and the CLV = $250 and the cost is $5, then:

Profitᵢ ≈ (0.25 * 250) – 5 = 57.5

So, on average, treating this customer is likely to yield $57.5 in profit. I’ve thresholded treatment for any customer with any profit. Depending on your budget, you can change this and, for example, target the ones that are likely to yield higher profits.

The below code illustrates this. Simply pick the top customers by this number until you hit your budget.

CLV, COST, BUDGET = 250.0, 5.0, 5000  # this can be changed to fit your context/goals

df["expected_profit"] = df["uplift"] * CLV - COST
campaign = df.sort_values("expected_profit", ascending=False).head(BUDGET)

print("Selected:", len(campaign), "Expected total profit ~", int(campaign["expected_profit"].sum()))
campaign[["customer_id","uplift","p_ctrl","p_treat","segment","action","expected_profit"]].head(10)

This to a large extent makes the decision “honest” in that treatment is offered where the incremental benefit outweighs the cost.

Practical Notes (and please read)

• Randomize where you can. Even a small A/B pilot (e.g., 10% holdout) will make your uplift estimates far more trustworthy.
• Be careful with non-random treatment. If teams hand-pick who to call, then your data is biased by all definitions. In this case, consider causal ML or run a clean experiment.
• Define "churn" very clearly, as it's very tricky. In banking, it could be account closure, 90-day inactivity, or a balance near zero. Some customers churn more than once, so consider whether it's best to keep them. In short, your label drives behaviour.
• Watch Sleeping Dogs. Aggressive “save” campaigns can create complaints and unsubscribes. As I mentioned earlier, I’m in this segment with a few of my utility providers.
• Keep the experiments explainable. Starting with classical models helps get business buy-in. You can always upgrade to fancier meta-learners later (T-learner, X-learner, causal forests).

Conclusion

That’s it for now. In Part 1, we stopped guessing who will leave. In this article, we stop guessing what works to keep them.

Uplift analysis bridges the gap between prediction and intervention. With a small, classical setup, you can rank customers by incremental impact, segment them into Persuadables / Sure Things / Lost Causes / Sleeping Dogs, and spend your budget where it actually moves the needle. As usual, my full code is here, and I hope it's easy to follow.

I hope this walkthrough was also useful. Don’t forget to follow me, clap for me, and leave a comment. If you want to check out my other articles:

• STOP Guessing Who Will Leave — How I Would Predict Customer Churn Before It Happens
• How to Create an Entire RAG System as a Newbie
• AI-Powered Equipment Maintenance Planning: Leveraging DeepSeek LLM and CrewAI for Smarter Decisions
• Capacity Optimization in Freight Trains — Part 1
• 6 Kgs Lost in 31 Days of COVID-19 Lockdown: A Data Analytics Perspective

Stop Guessing What Works — How I’d Use Uplift Modelling to Target Churn Interventions without fancy… was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A simple, step-by-step approach to classical churn modelling that works — no black-box AI required

Stuck behind a paywall? Read for free!

In my discussions with a fellow data scientist the other day, I realized that I hadn’t done much in classical Machine Learning(ML) for the last year or so. This is something I really enjoyed as a product data scientist. I attribute this to being busy riding the Generative AI (Gen AI) wave that is ever-changing. I managed to write a few Gen AI articles to justify the time spent on it. Many of them can be found here. Please read them and give some feedback. As always, all my codes are open-sourced here.

It's for this reason that I decided to relook at propensity modelling once again. For those who may not have heard of this term, propensity modelling is just the process of predicting the likelihood of a certain behaviour happening. For example, a jewelry shop predicting the likelihood of customers aged between 21 and 50 purchasing a certain ring etc. For this to happen, the shop has to understand the target outcome (whether that set of customers will purchase the ring or not). Therefore, this will always be use-case specific. In addition, the shop should also have a way of measuring the success of the modelling process. Don’t worry if this sounds complicated. I’ll break it down for you in the simplest terms possible with a relevant example.

Propensity to Churn in finance

Allow me to use a hypothetical example in the finance world. People often join banks for the purpose of accomplishing certain financial goals. However, some customers realise that the very banks may not be meeting their objectives. They then decide to leave after some time. This is a classical example of propensity-to-churn (leave the bank) modelling if the bank tries to statistically gauge whether they'll leave. By modelling this problem, the banks are likely to understand why certain customers left, thereby mitigating the same to counter the exit.

I’ll now walk you through a simplified example of a hypothetical Kenyan bank that is trying to predict the likelihood of some of its customers leaving. In this example, I’ll demonstrate to you how customer data spanning demographics, digital engagement, financial behavior, and product usage can be structured to model and predict customer churn. As mentioned above, this is a hypothetical example, meaning the data is also simulated. While the data is synthetic, I’ll make sure that it closely reflects real-world banking behavior in a Kenyan context. For those who’ve banked in Kenya, the incorporated data points will ring a bell, especially mobile money usage, etc., and other region-specific features .

Dataset Structure & Feature List

I came up with the below features that would generally be collected by financial institutions. However, the list is not exhaustive and should not be adopted as is in production environments. The features are subdivided as follows.

a) Customer Identity & Demographics

- customer_id (unique), age (years), gender
- region (e.g. Nairobi, Rift Valley, Coast, Western, etc.), urban (1=urban, 0=rural)
- education_level (none/primary/secondary/tertiary)
- employment_status (employed, self-employed, informal, unemployed)
- KYC_verified (Yes/No)

b) Account & Tenure Details

- account_open_date
- tenure_days (computed)
- customer_age_of_money (days since first deposit)
- account_type (savings, current, ecobank Eco-Savings, etc.)
- linked_mobile_number (SIM registered same person? flag for third-party SIM

c) Digital Engagement & Mobile Banking Usage

- uses_equity_mobile_app (binary)
- equity_mobile_sessions_last_30d (count)
- equity_mobile_trans_volume_last_90d (KES)
- equity_mobile_txn_count_last_90d
- uses_equitel (binary, Equity Group SIM) 
- equitel_txn_count_last_90d
- equity_online_login_count
- pesalink_inbound_count 
- pesalink_outbound_count (interbank transfers) 
- atm_withdrawals_count 
- atm_deposits_count

d) Mobile Money Ecosystem

- mpesa_linked_to_bank (Yes/No)
- mpesa_cash_in_count, cash_out_count (last 90d)
- mpesa_balance_avg_last_30d
- uses_mshwari (Yes/No)
- mshwari_savings_balance, 
- mshwari_loans_count
- fuliza_overdraft_count, 
- fuliza_overdraft_amt

e) Transaction Behavior & Financial Usage

- avg_balance_last_6m
- min_balance_last_6m
- num_deposits_last_90d, 
- num_withdrawals_last_90d
- avg_monthly_inflow, 
- avg_monthly_outflow
- loan_applications_count, 
- loans_disbursed_amt 
- loan_repayment_rate (ratio repaid vs due)
- credit_card_prepaid_spend / usage

f) Savings & Spending Patterns

- savings_rate (deposit volume / inflow)
- spend_rate (withdrawal volume / inflow)
- recency_deposit_days (days since last deposit)
- frequency_deposit (days between deposits)
- recency_digital_txn_days
- frequency_digital_access_days

g) Behavioral Indicators & Support Interactions

- complaints_count_last_year (e.g. service, fraud)
- branch_visits_count_last_year
- customer_support_calls_last_6m
- biometric_enabled_atm_user (Yes/No) 
- security_alerts_triggered (password resets, blocked login)

h) Loyalty related features

- num_products_held (e.g. account, card, loan, insurance)
- loyalty_program_member (Yes/No)
- rewards_redeem_count
- referrals_count (how many new customers referred)

i) Churn/Outcome Variables

- churned (1=no activity or account closed within last 90 days)
- churn_date
- last_active_date
- reason_code (survey-captured reason: e.g. switched bank, poor service, moved regions, digital usability issues, trust/security concerns)This feature list is not exhaustive by any means. Internally, banks are likely to be capturing many more features especially in relation to the number of branch visits as well as financial well-being of customers. They are normally good indicators of churn especially if customers visit branches to complain.

FYI, I’m likely not to use the entire list of the above features in the modeling process, so don’t be worried if you see some features missing.

Data Simulation

Coming up with a list of features like I did above is the easy part. However, the tricky bit is actually tweaking the simulated data to be statistically as close as possible to what the bank would have. In addition, real data is usually messy and, most times, provides a realistic distribution of features, complete with missing values, unusual spikes etc., compared to what I have here.

The code I put together generates a simulated dataset of about 10,000 customers, with account tenures spanning roughly three years. The numbers are not set in stone and can be changed. Remember, customers can churn more than once in reality, but I’ll just focus on one churn event in this simulation within the observation period. Please try modify the code to simulate more than one churn if you have the time.

The code is split three files config.py , generator.py and main.py for easier debugging. FYI this a good software engineering approach to adopt as data scientists. config.py holds all the variables where say the sample size, dates, region mix, urban shares, churn target, and distribution settings can be defined. This helps tune assumptions without touching logic.generator.py contains small, readable functions that build the dataset in stages (generation of the population size → adoption → transactions → amounts → credit → churn), with deterministic randomness for reproducibility. main.py is just a wrapper that loads the config, assembles the dataset, prints a quick churn sanity check, and writes to a equity_bank_churn_dataset.csvfile. The data folder is here.

Try playing with parameters in main.py and config.pyand you’re likely to get a fresh dataset each time. One thing to note is that it might not be easy to hit a certain percentage of churn in the data e.g., 30% if that’s your target. Try adjusting the churn_target and reducing the weibull_scale_days if you run into this issue. You’re likely to get to your target percentage of churn faster and after a few iterations.

Statistical Distribution in the Data

As mentioned in the introduction, the distribution in the data has to mimic the diversity you’d see in an actual Kenyan retail bank. This meant that I had to introduce a few nuances in the data as follows:

1. Urban and rural customers are modeled differently, whereby customers in Nairobi are treated as being in an urban area, while other provinces have both rural and urban population. They are in proportions that reflect real-life patterns.
2. Digital engagement varies accordingly, with urban customers more likely to use the the mobile app, transact via Equitel, or link their M-Pesa wallets to their bank accounts, etc. This may not be overly factual, but I maintained it this way for simplicity.
3. Continuous variables like transaction counts and monetary amounts are generated using statistical distributions that fit the nature of each feature. For example:

• Poisson and Negative Binomial distributions for counts like mobile sessions, deposit and withdrawal transactions, or branch visits. These capture the fact that most customers make only a handful of transactions, but a few transact heavily.
• Lognormal and Gamma distributions for balances, loan amounts, and savings etc. The idea here is to create a realistic skew where most customers hold small amounts, while a few hold very large sums.
• Beta distribution for loan_repayment_rate where all values are kept between 0 and 1, but skewing towards high repayment for employed or self-employed customers. This is also an assumption that might not be overly true if proper analysis on a real dataset was done.

All these are defined in thegenerator.py file in the data here.

Below is a summary of the train and test set features. It was a time-based split where the training data is between 2022–01–01 → 2023–12–31.

=== TRAIN DATA ===
Rows: 6,694 | Columns: 25
Date range: 2022-01-01 → 2023-12-31
Churn rate: 33.21%

Numeric feature ranges:
                                           min          max          mean
age                                    18.0000      69.0000     40.545414
urban                                   0.0000       1.0000      0.592620
uses_equity_mobile_app                  0.0000       1.0000      0.723185
uses_equitel                            0.0000       1.0000      0.300717
mpesa_linked_to_bank                    0.0000       1.0000      0.841649
equity_mobile_sessions_last_30d         0.0000      45.0000      4.785181
equity_mobile_txn_count_last_90d        0.0000      93.0000     13.161189
mpesa_cash_in_count                     0.0000      64.0000      6.577980
mpesa_cash_out_count                    0.0000      42.0000      5.221691
num_deposits_last_90d                   0.0000      21.0000      3.943681
num_withdrawals_last_90d                0.0000      23.0000      3.593666
equity_mobile_trans_volume_last_90d  4532.4100  411993.9300  61774.712601
avg_balance_last_6m                  2132.8600  380194.2900  51316.757398
mshwari_savings_balance                 0.0000   32067.2900   3071.699382
mshwari_loans_count                     0.0000       9.0000      1.675530
fuliza_overdraft_amt                   32.2200   18405.4300   3077.626280
loan_applications_count                 0.0000       6.0000      1.231999
loan_repayment_rate                     0.0987       0.9993      0.740785
branch_visits_count_last_year           0.0000       8.0000      1.610397
complaints_count_last_year              0.0000       4.0000      0.204661

Most frequent categories:
  gender: Female (51.7%)
  region: Rift Valley (23.8%)
  education_level: Secondary (47.9%)
  employment_status: Employed (45.3%)
  KYC_verified: Yes (91.8%)

On the other hand, the tests are done on one year of data that doesn't overlap with the training data (2024–01–01 → 2024–12–31)

=== TEST DATA ===
Rows: 3,306 | Columns: 25
Date range: 2024-01-01 → 2024-12-31
Churn rate: 7.59%

Numeric feature ranges:
                                           min          max          mean
age                                    18.0000      69.0000     40.555354
urban                                   0.0000       1.0000      0.593164
uses_equity_mobile_app                  0.0000       1.0000      0.734725
uses_equitel                            0.0000       1.0000      0.306413
mpesa_linked_to_bank                    0.0000       1.0000      0.847550
equity_mobile_sessions_last_30d         0.0000      37.0000      4.920145
equity_mobile_txn_count_last_90d        0.0000      79.0000     13.261948
mpesa_cash_in_count                     0.0000      50.0000      6.626134
mpesa_cash_out_count                    0.0000      37.0000      5.165759
num_deposits_last_90d                   0.0000      22.0000      3.941016
num_withdrawals_last_90d                0.0000      23.0000      3.537205
equity_mobile_trans_volume_last_90d  3471.6600  324709.7100  60852.307114
avg_balance_last_6m                  2967.5200  511006.9400  50976.922426
mshwari_savings_balance                 0.0000   28821.7300   3090.220230
mshwari_loans_count                     0.0000       9.0000      1.702057
fuliza_overdraft_amt                   18.8900   15597.9200   3011.771385
loan_applications_count                 0.0000       6.0000      1.212341
loan_repayment_rate                     0.1345       0.9995      0.740829
branch_visits_count_last_year           0.0000       9.0000      1.615245
complaints_count_last_year              0.0000       4.0000      0.209316

Most frequent categories:
  gender: Female (50.2%)
  region: Rift Valley (24.6%)
  education_level: Secondary (46.8%)
  employment_status: Employed (44.5%)
  KYC_verified: Yes (92.0%)

Churn as the target outcome is not random. The assumption is that its influenced by many factors such as low digital engagement, rural location, higher complaint counts etc. Therefore, the time until churn is generated using a Weibull survival model which lets the likelihood of churn increase over time, especially for higher-risk customers. I’ve tuned the simulation so that around 30% of customers churn within the three-year window, similar to what many banks are likely to experience in practice.

While this is still a simplification of the reality, the dataset is statistically rich enough to demonstrate churn prediction techniques, explore the relationship between customer attributes and retention, and test how targeting strategies might work before deploying them on real, messy banking data. Remember, this is a simulated dataset and should not be used as a replacement for real data.

Propensity to Churn Modeling

I’ve defined the features and explained the rationale behind the data simulation. This data is not messy, so not much cleaning is needed. However, this will not be the case in a real dataset, as it will have so many missing and and sometimes very skewed values.

With the dataset preparation out of the way, the next step is to build predictive models that estimate each customer’s propensity to churn. For this project, I applied two complementary algorithms.

1. Logistic Regression—it's a simple and easily interpretable algorithm,, that is widely used in banking for its transparency. This means that it allows business teams to see exactly how each feature influences churn risk.
2. Gradient Boosting—This is a powerful ensemble method that can model complex, non-linear relationships and interactions between features. This often achieves higher predictive accuracy at the cost of interpretability.

Feel free to test a few more algorithms if you have the time. FYI, please have a look at Random Forest algorithm and how its performance differs from say Gradient Boosting one. This might just save you in your next interview.

By training and evaluating both models on the same data, we can compare their performance using metrics like ROC_AUC, Precision-Recall AUC, Lift, and Gains curves, enabling data-driven decisions on which model is best suited for different campaign targeting strategies.

The below code is part of the modelling workflow. It details the data splits (train and test) and preprocessing steps for categorical and continuous variables. The code in section 5, 6 is the training one, and the evaluation (testing model performance) one is in part 7. Simply run the code here to get the entire modelling run after generating the sample data in activated environment. To run the the below, simply run the command python model.py in a Python terminal while inside py folder.

df = pd.read_csv(DATA_PATH)
df = parse_dates(df, DATE_COLS)

y = df[TARGET].astype(int).values
X = df.drop(columns=[c for c in DROP_COLS if c in df.columns] + [TARGET], errors="ignore")

# Only add columns that are not already in X
cols_to_add = [col for col in DATE_COLS + [TARGET] if col not in X.columns]
df_for_split = pd.concat([X, df[cols_to_add]], axis=1)
train_df, test_df = temporal_split(df_for_split, AS_OF_CUTOFF)

X_train = train_df.drop(columns=DATE_COLS + [TARGET])
y_train = train_df[TARGET].astype(int).values
X_test  = test_df.drop(columns=DATE_COLS + [TARGET])
y_test  = test_df[TARGET].astype(int).values

print(f"Train: {len(X_train):,} rows | Test: {len(X_test):,} rows")

# Describe training and test sets
describe_dataset(X_train, y_train, train_df["account_open_date"], "Train")
describe_dataset(X_test, y_test, test_df["account_open_date"], "Test")

# -----------------------------
# 4) Preprocessing
# -----------------------------
cat_cols = X_train.select_dtypes(include=["object", "category"]).columns.tolist()
num_cols = [c for c in X_train.columns if c not in cat_cols]

cat_pipe = Pipeline([
    ("imputer", SimpleImputer(strategy="most_frequent")),
    ("ohe", OneHotEncoder(handle_unknown="ignore", sparse_output=False)),
])

num_pipe = Pipeline([
    ("imputer", SimpleImputer(strategy="median")),
    ("scaler", StandardScaler()),
])

pre = ColumnTransformer([
    ("cat", cat_pipe, cat_cols),
    ("num", num_pipe, num_cols),
])

# -----------------------------
# 5) Models
# -----------------------------
logreg = LogisticRegression(max_iter=1000, class_weight="balanced", random_state=SEED)
gbdt   = GradientBoostingClassifier(
    random_state=SEED,
    learning_rate=0.05,
    n_estimators=300,
    max_depth=3,
    subsample=0.9
)

pipe_lr  = Pipeline([("pre", pre), ("clf", logreg)])
pipe_gbdt = Pipeline([("pre", pre), ("clf", gbdt)])

# -----------------------------
# 6) Train
# -----------------------------
pipe_lr.fit(X_train, y_train)
pipe_gbdt.fit(X_train, y_train)

# -----------------------------
# 7) Evaluate
# -----------------------------
results = []
results.append(evaluate_model("Logistic Regression", pipe_lr, X_test, y_test))
results.append(evaluate_model("Gradient Boosting", pipe_gbdt, X_test, y_test))

print("\n=== MODEL COMPARISON (Test Set) ===")
for r in results:
    print(f"{r['name']}:")
    print(f"  ROC AUC:        {r['roc_auc']:.4f}")
    print(f"  PR  AUC:        {r['pr_auc']:.4f}")
    print(f"  Top 10% Prec.:  {r['top10_precision']:.4f}")
    print(f"  Top 20% Prec.:  {r['top20_precision']:.4f}")
    print(f"  Confusion Matrix @0.5 [TN FP; FN TP]:\n{r['conf_matrix']}")
    print("-" * 50)

Model Evaluation Results

To evaluate the predictive performance of the two models, I tested both the Logistic Regression and Gradient Boosting on the held-out (data that is left out of the training phase) test dataset. As mentioned above, performance was assessed using a mix of threshold-independent metrics (ROC AUC, PR AUC) and business-relevant targeting metrics (Top-k Precision, Confusion Matrix). The results are as follows and they indicate the following:

| Metric               | Logistic Regression | Gradient Boosting |
| -------------------- | ------------------- | ----------------- |
| ROC AUC              | 0.7872              | 0.7814            |
| Precision-Recall AUC | 0.1739              | 0.1804            |
| Top 10% Precision    | 0.1909              | 0.1970            |
| Top 20% Precision    | 0.1785              | 0.1710            |

1. Both models achieve similar ROC_AUC scores (~0.78), indicating good ranking ability in their predictions.
2. Gradient Boosting edges out Logistic Regression in Top 10% Precision. What this means is that it's slightly better at identifying the highest-risk customers making it ideal for identifying potential churners for small, focused retention campaigns. If I was in charge of the results implementation, then I’ll use gradient boosting results if budget is an issue.
3. Logistic Regression slightly outperforms in the Top 20% Precision. This makes it competitive for broader targeting strategies.

Confusion Matrices @ 0.5 Threshold

A confusion matrix is just a table with side by side model evaluation results in classification models like in the above example. The Logistic Regression results are below:

True Negatives(TN) = 1880  | False Positives(FP) = 1175 
False Negatives(FN) =  39  | True Positives (TP) = 212

This just means the model identified 212 true churners correctly, missed 39, and had 1,175 false positives.

For the Gradient Boosting one, the results were as follows:

True Negatives(TN) = 2112 | False Positives(FP) = 943  
False Negatives(FN) = 67  | True Positives (TP) = 184

Unlike the logistics regression one, GBM had fewer false positives but missed more actual churners than Logistic Regression.

Key Drivers of Churn

I also identified key drivers of churn following feature importance values and direction of coefficients in the two models. Logistic Regression coefficients just reveal the direction and magnitude of influence as shown in the below plot. The interpretation is as follows:

• Negative coefficients (reduce churn risk): Secondary education, Primary education, Coast region, Higher branch visits etc.
• Positive coefficients (increase churn risk) like KYC verified.

The bars were color-coded so that red features are deemed to be the one to push churn risk up and blue features lower churn risk, making interpretability easier.

For example, customers with secondary education or from the coast region are likely to be better integrated in the banking system that they don’t want to leave. Its the opposite of KYC verified where the likelihood of customers with this positive feature are likely to leave the bank or already on the way out.

On the other hand, Gradient Boosting feature importances show predictive power as in the following feature importance plot:

Findings from the above plot show that:

• The most influential features just like in the logistics regression example are still the Education level (Secondary, Primary), Branch visits, Region (Coast), Complaints count in that order.
• Other factors: Employment status, Loan repayment rate, KYC verification status.

Its worth noting that education_level_Secondary is ranked as by far the most important predictor (importance 0.416) by GBM. However, it doesn’t really tell if it’s protective or risky. Please have a look at SHAP values here as they are a good answer to this shortfall in using plain feature importance values.

Business Implications

Beyond the above metrics, churn prediction models need to demonstrate business value and specifically, how well they help the bank target retention campaigns. Two types of charts make sense in this aspect :

• Cumulative Gains Chart — This shows the proportion of actual churners captured in the customer base, ranked by predicted churn probability. These ones are shown in the left part of the below plots.

Their interpretation is quite simple. The diagonal dashed line represents a random targeting strategy i.e., in case no model is used. From the visualisations, the curves rise steeply above this diagonal line. This is an indicator that the model is effectively identifying churn-prone customers early in the targeting process which is the case in the two algorithms. For example, when we target 40% of customers most likely to churn, there is a high chance that we’ll capture over 80% of actual churners. This is significantly better than random targeting, which would only capture ~40% of churners with the same effort.

• Lift Charts are on the right side of the gains chart above. They basically just tells us how much better the model is at identifying churners compared to random selection. A lift of 3 means we are three times better than random at that targeting level.

From the above results, Gradient Boosting delivers the highest lift in the very first deciles, reaching ~4.8× improvement over random when targeting the top 1% of customers. It then stabilizes around a lift of ~2.0–2.5 for the next 20–40% of the population. On the ther hand, Logistic Regression starts with a maximum lift of ~3× in the top 1% and maintains a lift above 2 for a large portion of the ranked list, indicating consistent value.

Business Takeaway from This Analysis:

For such a bank, these charts highlight the practical advantage of using a predictive churn models. Instead of spreading retention efforts across the entire customer base, targeting just the top 20% most at-risk customers could capture the majority of potential churners. This translates into lower intervention costs (fewer SMS, calls, or retention offers) and higher ROI for customer retention campaigns. The interpretation is as follows:

• For highly targeted campaigns (e.g., top 10% of customers at risk), Gradient Boosting is slightly stronger and captures more churners early.
• For broader campaigns (top 20%+ of customers), Logistic Regression is competitive and has the advantage of interpretability, a very crucial factor to consider for explaining risk drivers to non-technical business stakeholders.
• Both models highlight similar churn drivers, with education level, branch visit patterns, and regional differences being strong predictors, which aligns with observed customer behavior patterns in the Kenyan banking context. Remember this is likely to be very different when a real dataset is used.

Conclusion

I introduced propensity modelling, a vital skill for product data scientists and analysts. In addition, I deep-dived in the process of simulating realistic customer data for a propensity-to-churn use case. I was able to demonstrate strategies around customer data simulation, engineering of relevant features, and finally the process of building predictive models to estimate the propensity to churn. While simulated data lacks the messiness and unpredictability of real-world datasets, it allowed us to focus on the core workflow detailing the data preparation, model training, evaluation, and interpretation steps.

The results show that both Logistic Regression and Gradient Boosting can effectively identify high-risk customers, with Gradient Boosting having a slight edge in early-decile lift. Importantly, the evaluation metrics, cumulative gains, and lift charts make it clear that targeted retention strategies can capture a large proportion of churners while engaging only a small portion of the customer base.

In a real banking context, such models can help allocate retention budgets more efficiently, reduce customer attrition, and ultimately protect revenue. However, it’s worth noting that model performance will depend heavily on data quality, feature richness, and continuous retraining to adapt to changing customer behavior.

This approach can be extended beyond churn to other key business challenges such as cross-selling etc., all driven by the same principle: using data-driven insights to focus efforts where they matter most. Please let me know in the comments whether I should write related articles. As usual, my code is always open-sourced and is here. Please feel free to clone it and re-run. I hope you’ll remember to re-look at the random forest algorithm.

I hope this walkthrough was also useful. Don’t forget to follow me, clap for me, and leave a comment. If you want to check out my other articles:

• Building a RAG System with MMR for Safaricom’s Smart Assistant
• Drive Customer Success: Supercharging Safaricom’s Product FAQs with Llama 2 Model
• How to Create an Entire RAG System as a Newbie
• AI-Powered Equipment Maintenance Planning: Leveraging DeepSeek LLM and CrewAI for Smarter Decisions

STOP Guessing Who Will Leave — How I Would Predict Customer Churn Before It Happens was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Stuck behind a paywall? Read for free!

I took a short break from writing to focus on some pressing AI-related work commitments. This gave me time to think about fresh ideas I could explore in the AI space, especially ones that can have a real impact on everyday people.

So, last week, I spent some time thinking deeply about how AI can support mental health therapy for teenagers, specifically in Kenya and other developing countries. Teenagers in these regions face growing mental health challenges due to academic pressure, fears around unemployment, and the emotional toll of poverty and family instability. Yet, access to professional help remains limited.

Normally, there are very few trained counselors and healthcare centers to handle such cases. In addition, some cultural barriers and cost of therapy make it even harder for the teens to reach out. In fact, I spoke with a counseling professional recently who was concerned about the high cost of proper therapy in Kenya. This is made worse when compared to the median wage in a country where about 39.8% of people live below the poverty line.

This got me really interested in exploring how AI can be used to tackle this issue. I thought of how I can leverage agentic AI as a promising solution to this. AI Agents are scalable, always available, and can provide immediate support. My goal with this mini-project is simple: create a safe, accessible, AI-powered space for Kenyan teens to explore their feelings and learn coping strategies

Let me be clear — this is NOT meant to replace human therapists. It’s just a supportive tool for young people who might otherwise go unheard.

Foundations of AI and Counselling Psychology

To ensure the AI system I built is not only technically sound but also clinically meaningful, I grounded its design in evidence-based practices from adolescent mental health research.

Effective therapy for teens isn’t just about giving advice. Its about building trust and respecting the cultural context they are accustomed to. In addition, teaching emotional and cognitive skills and empowering them to ensure safety during crises are other considerations to be factored.

Here are the key principles that guided the development of each agent in the AI system:

✅ Emotional validation — Teenagers need to feel seen, heard, and accepted without judgment. Research shows that the therapeutic alliance (bond between therapist and client) is one of the strongest predictors of positive outcomes (Shirk & Karver, 2003). That’s why I created the Empathy Agent to simulate this connection by offering warm, validating responses to their concerns.

✅ Cultural relevance — Mental health interventions are most effective when they align with the cultural background of the person receiving them (Sue et al., 2009). In Kenya, this means recognizing local languages that are closer to teens like Sheng and Kiswahili. In addition, understanding family roles, spiritual beliefs, and common stressors such as academic pressure and unemployment etc., are to be considered. Here, I added the Cultural Agent that ensures responses resonate well with Kenyan teens. This agent can be adapted for different demographics.

✅ Psychoeducation and cognitive-behavioral skills — Cognitive Behavioral Therapy (CBT) is one of the most evidence-based treatments for adolescent depression and anxiety (Weisz et al., 2017). Teaching teens how thoughts, emotions, and behaviors interact builds resilience and self-awareness. The CBT and Coping agents are meant to deliver bite-sized psychoeducation tools to help with this.

✅ Empowerment and connection to resources — Adolescents are more likely to thrive when they feel in control of their lives and are connected to support networks (Zimmerman, 1995). The Goal Agent in the framework helps teens clarify and break down personal goals, while the Resource Agent links them to real-world services in Kenya, bridging the gap between AI and human care.

✅ Crisis management — Safety comes first. Following global suicide prevention guidelines (WHO, 2014), I included a Crisis Agent that prioritizes risk assessment and provides grounding techniques and emergency contact info when high-risk language is detected. High-risk keywords include:

  risk_keywords = [
        "suicide", "kill myself", "end my life", "harm myself",
        "hurt myself", "die", "death", "no point", "give up", "hopeless", '
        "abuse", "hitting me", "beating me", "harming me",
        "scared", "terrified", "trapped", "emergency"
    ]

AI Agents

Each of the above psychological foundations informed the design of the agentic AI framework. Each agent was mapped directly to a counselling psychology principle as follows:

1. The Empathy Agent provides the emotional validation and builds trust with the teen.
2. The Cultural Agent — Ensures relevance and respect for the Kenyan teen’s lived experience. This could always be customised for other cultures and demographics.
3. Cognitive Behavioral Therapy(CBT) and Coping Agents — These agents deliver psycho-education and teach practical coping strategies based on the problem at hand.
4. Goal Agent encourages empowerment by helping teens set and track personal goals.
5. Resource Agent — Connects teens with real-life support services in Kenya.
6. The Crisis Agent — Activates in emergencies, providing immediate crisis contacts like Childline Kenya (116 or +254 722 116116) and Befrienders Kenya (+254 722 178 177). This agent’s main role is to express concern and guide teens toward human help that is of essence in times like this.

Agentic AI Pipeline

By combining these agents, the system mimics a layered and integrative approach that a human therapist would use. Here’s how the flow works:

When a user inputs a message, the system scans for risk keywords to assign a risk score. If the message indicates a potential crisis (e.g., mentions of suicide or abuse), the system immediately switches to crisis mode. There is a risk score that is computed before the change to crisis mode in a very simple way. This why the risk score in the code below is just the minimum of a weighted score per crisis word hit and is capped at 1. Basically, any “hint” of a crisis word will immediately trigger the system to follow the crisis workflow. This in return triggers specific agents designed for crisis response.

hits = sum([1 for word in risk_keywords if word in text_lower])
if hits == 0:
    return 0.0
# Simple scoring: base score + increment per hit, capped at 1.0
# Ensures any hit gives a noticeable score.
risk_score = min(0.1 + 0.1 * hits, 1.0)

If no high-risk words are detected, the system follows the standard workflow as follows:

1. First, it applies cultural context analysis to make sure the crafted message is culturally relevant.
2. Secondly, it offers empathetic listening.
3. Finally, it branches into cognitive-behavioral guidance, coping strategies, goal setting, or resource recommendations based on the words in the teen’s message.

All agentic outputs are then combined into a single, coherent response using this function like this:

def combine_agent_responses(step_responses, risk_score):
    """
    Combine multiple agent responses into a coherent chat message.
    Prioritizes crisis response if risk is high.
    Otherwise, attempts to weave together empathy, advice/resources, and a supportive closing.
    Args:
        step_responses (list[str]): A list of strings, where each string is the output
                                    of a sequential LLM call simulating an agent's task.
        risk_score (float): The calculated risk score for the user's message.
    """
    responses = []
    final_response = "I'm here to listen. How can I help you today?" # Default fallback

    # --- 1. Extract Raw Outputs ---
    # Ensure step_responses is a list of strings
    if isinstance(step_responses, list) and all(isinstance(r, str) for r in step_responses):
        responses = [r.strip() for r in step_responses if r and r.strip()]
    else:
        print(f"Warning: Invalid step_responses format received: {step_responses}")
        # Attempt to use the input directly if it's a string, otherwise return default
        return str(step_responses) if isinstance(step_responses, str) else final_response

    # --- 2. Handle High-Risk (Crisis) Scenario ---
    # Use the risk score threshold defined for routing (0.15)
    if risk_score >= 0.15:
        print("Combine Responses: High risk detected, prioritizing crisis output.")
        # In the crisis flow (Cultural -> Empathy -> Crisis -> Coping), the Crisis response is likely the 3rd output.
        # The Coping response (4th) might offer grounding, but the Crisis message is paramount.
        crisis_response = "It sounds like you're going through a very difficult time. Please reach out for immediate support by calling Childline Kenya at 116 or Befrienders Kenya at +254 722 178 177. They are available to help you right now." # Safer default crisis message
        if len(responses) >= 3:
            # Assume the third response (index 2) is from the Crisis step
            # Check if it contains hotline numbers; if so, use it directly.
            if "116" in responses[2] or "Befrienders" in responses[2].lower():
                 crisis_response = responses[2]
            # If not, maybe the 4th (coping) response (index 3) has them? Less likely but check.
            elif len(responses) >= 4 and ("116" in responses[3] or "Befrienders" in responses[3].lower()):
                 crisis_response = responses[3]
            # If neither contains hotlines, stick to the default crisis message.

        # Ensure essential hotline info is present in the final crisis response
        if "116" not in crisis_response:
            crisis_response += "\nEmergency Help: Call 116 (Childline)"
        if "Befrienders" not in crisis_response:
             crisis_response += "\nOr call +254 722 178 177 (Befrienders Kenya)"

        return crisis_response.strip()

    # --- 3. Handle Standard (Non-Crisis) Scenario ---
    print(f"Combine Responses: Standard flow detected. Responses received: {len(responses)}")
    if not responses:
         return final_response # Return default if no responses somehow

    # In standard flow (Cultural -> Empathy -> [CBT/Coping/Goal/Resource]), expect at least 3 responses.
    # The first (index 0) is cultural context (often internal note), second (index 1) is empathy, third (index 2) is the specialized response.

    # Start with Empathy (usually the second response)
    empathy_part = ""
    if len(responses) >= 2:
        # Take the empathy response (index 1), usually short and validating.
        empathy_part = responses[1]
        # Basic check if it looks like validation
        if not ("understand" in empathy_part.lower() or "hear you" in empathy_part.lower() or "sounds like" in empathy_part.lower() or "okay to feel" in empathy_part.lower()):
             empathy_part = f"I hear you. {empathy_part}" # Add a generic validation if needed

    # Get the main content (from CBT, Coping, Goal, or Resource agent - usually the last response)
    main_content_part = ""
    if len(responses) >= 3:
        main_content_part = responses[-1] # Assume the last step provides the core advice/resource/prompt
        # Remove potential redundancy if it repeats the empathy part
        if empathy_part and main_content_part.startswith(empathy_part.split('.')[0]): # Check if starts similarly
             pass # Keep it as is, agent might have synthesized well
        elif empathy_part:
             main_content_part = "\n\n" + main_content_part # Add spacing if distinct

    # Construct the final response
    if empathy_part and main_content_part:
        final_response = empathy_part + main_content_part
    elif main_content_part: # Only got main content
        final_response = main_content_part
    elif empathy_part: # Only got empathy
        final_response = empathy_part + "\n\nWhat else is on your mind?" # Add a prompt
    elif responses: # Fallback to last response if logic failed
         final_response = responses[-1]

    # Add a general supportive closing, avoiding redundancy if already present
    closing_statement = "\n\nRemember, taking care of yourself is important, and you don't have to figure everything out alone. I'm here to support you."
    if not ("remember" in final_response.lower() or "alone" in final_response.lower() or "support you" in final_response.lower()):
         final_response += closing_statement

    return final_response.strip()

The above function ensures the final message is emotionally validating, culturally appropriate, and actionable — whether the user is in crisis or just needs support.

Sample workflows Outputs

Crisis Workflow Outputs:

Below is an example of a high-risk message that triggered the crisis workflow. From the response, the message was culturally sensitive, empathetic and followed by emergency contact details.

In another test, the same user sent a follow-up message in Sheng — a local slang blending Swahili and English (Githiora, C. 2018) that is commonly spoken by Kenyan youth. The system understood the message and responded in the same language appropriately.

Sample Standard Workflow Output

For non-crisis messages, the system follows the standard path whereby:

1. It starts with a culturally relevant and empathetic response.
2. Thereafter , it uses keyword routing to determine which agent should respond.
3. Finally, it combines and refines the agent outputs into a final, supportive message.

A sample output of this workflow type is as follows:-

From the above outputs, the system adapts well to both crisis and non-crisis situations while staying grounded in psychological principles.

Its really that simple. I didn’t share many code snippets as I’ve always done in my previous articles as the bigger chunk is just the prompting and re-routing. However, as usual, my code is always open and can be accessed here. Instructions to run the same are in the repo’s Readme file. Just remember to use your Open AI key for the code to run.

TLDR;Below are the key aspects of the project :

1. I presented a culturally-tailored AI Support app built specifically for Kenyan teens. The chatbot understands local languages like Sheng and Kiswahili, and addresses therapy related issues factoring in the cultural context of most Kenyan youth.
2. The app is made up of several agents simulating different therapeutic roles i.e., empathy, CBT, coping, goal-setting, resources, and crisis management — through sequential LLM calls.
3. The app is also able to detect crisis in the query of the person interacting with it. It does this by identifying high-risk language in the wordings and is able to redirect users to emergency hotlines in Kenya.

Summary and future work

Blending AI with cultural knowledge is a step forward in the dissemination of mental health support in especially under-resourced areas. It has the potential to transform therapy into a continuous, personalized journey of self-discovery and resilience as demonstrated in this mini-project.

In conclusion, I’m still of the opinion that AI should NEVER be a replacement of trained professionals. It should only augment the work of therapists. Humans deserve more than just algorithmic responses — they deserve genuine empathy and real human connection.

Looking ahead, here are a few features I’d like to add to this project if time allows:

1. Personalized Long-Term Memory & Goal Tracking — With consent and anonymization, I’m kene on adding a feature to allow the chatbot to remember themes, preferred coping strategies, or past goals etc. This is likely to personalise the experiences of users on the app.
2. Integration of AI-driven interactive therapeutic tools like mood journaling and CBT-style ones could be done in the future.

Such features would make the chatbot more engaging and give teens hands-on tools to manage their mental health.

I hope this walkthrough was also useful. Don’t forget to follow me, clap for me, and leave a comment. If you want to check out my other articles:

• Building a RAG System with MMR for Safaricom’s Smart Assistant
• How to Create an Entire RAG System as a Newbie
• Drive Customer Success: Supercharging Safaricom’s Product FAQs with Llama 2 Model

References

• Shirk, S. R., & Karver, M. (2003). Prediction of treatment outcome from- relationship variables in child and adolescent therapy. Journal of Consulting and Clinical Psychology , 71(3), 452–464. https://doi.org/10.1037/0022-006X.71.3.452
• Sue, S., Cheng, J. K., Saad, C. S., & Chu, J. P. (2012). Asian American mental health: A call to action. American Psychologist , 67(7), 532–544. https://doi.org/10.1037/a0028900
• Weisz, J. R., Kuppens, S., Ng, M. Y., Eckshtain, D., Ugueto, A. M., Vaughn-Coaxum, R., & Fordwood, S. R. (2017). What five decades of research tells us about the effects of youth psychological therapy. American Psychologist , 72(2), 79–117. https://doi.org/10.1037/a0040360
• Zimmerman, M. A. (1995). Psychological empowerment: Issues and illustrations. American Journal of Community Psychology , 23(5), 581–599. https://doi.org/10.1007/BF02506983
• World Health Organization. (2014). Preventing Suicide: A Global Imperative . https://apps.who.int/iris/handle/10665/131056
• Githiora, C. (2018). Sheng: Rise of a Kenyan Swahili Vernacular . Boydell & Brewer. https://doi.org/10.2307/j.ctv1ntfvm

From Code to Coping: Inside an AI Chatbot Designed for Teen Well-being was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

I was talking to one of my friends the other day about the advancements in Text To Speech(TTS) and Speech To Text(STT) systems in the ever-evolving AI world. I realized this is one area that I haven’t really explored much. For those who may not be aware, TTS is just the technology that helps convert text on a digital interface into natural-sounding audio.

I’ve in the past written about optimisation, Retrieval-Augmented Generation (RAG), agents, as well as data analytics-driven stories. Therefore, I wanted to look at something slightly different, and this led me to TTS advancements and especially their interplay with agentic AI. How could agentic AI work be combined with TTS?

After much thought, I settled on coming up with a customized Agenti AI-powered safari guide application. Local safari guides are usually great, but as humans, we tend to repeat a lot of what we’ve memorized over time. Therefore, this resulted in the thought of developing an AI-driven tour guide. Just think of a local guide that you meet in a museum, gallery, or safari but one who has a massive knowledge base that’s driven by you. All you need to do is prompt the AI tour guide, and you get very relevant and up-to-the-minute audio tour information. This could include updates fused with live weather data that’s specific to the location. Enough of stories. Let's go to the interesting parts.

1. Agentic-AI Powered Architecture

As mentioned in the introduction, agents were my choice in developing this application. For those who may not be aware, agents are mini computer programs that can interact with the environment to collect data and perform certain tasks to meet a predetermined goal. A good example is, say, a robot barista that’s been instructed to serve the perfect cup of coffee. It doesn’t just follow a script. It checks the weather (maybe you’d prefer something iced on a hot day) and adjusts the brew. If you ask for something unusual — say, a lavender oat milk cortado with half the sweetness — it searches its recipes, adapts, and serves it up. If it doesn’t meet your requirements, then it can ask you for help/suggestions on the alternatives. That’s how AI agents work: they’re not just following instructions, but they think through the process of reaching the goal, step by step, with whatever tools and info they’ve got.

This app is orchestrated around a core controller called SafariManager. This class in itself doesn’t generate any content but delegates the tasks to the domain-specific agents. The data generation process starts with a user inputting their safari preferences based on the location they are visiting. This then forms the basis of the extracts from the agents. The primary agents work as follows, and this is in the context of a tour. For example, imagine you are at the Maasai Mara National Park in Kenya; the agents would pull information as follows:

1. Biodiversity agent — describes the local Mara ecosystems, historical wildlife data, and other engaging ecological narratives.
2. Environmental agent — provides up-to-date environmental conditions, including highlights on the sustainable practices and local environmental challenges in the Mara area.
3. Culture agent — describes in detail the cultural narratives, including local Maasai traditions, keeping the conversation warm and respectful.
4. Safety agent — this is where safety guidelines, including navigation tips, weather alerts, and general safety measures, are curated. It could include safety tips when navigating, say, muddy roads in the park. This agent will do a web search to get the up-to-date safety data.
5. Planner Agent — this is a vital agent as it plans for the audio tour. It does this by analysing the user’s location, interests, and tour duration to determine the optimal time allocations for each of the specialist agents above. In addition, introduction and conclusion section timing allocations are also added here.
6. Orchestrator — this agent stitches together the content from the other agents into a natural, flowing narrative. It adds an introduction and a brief conclusion.

Remember each agent is prompted with structured instructions and given a specific role. The following diagram visualizes this simplified flow:

2. User Input to Agent Execution

I used a Streamlit interface to capture most inputs from users. It's a simple and Python-native framework that makes it easy to integrate modules such as the ones in the above architecture. The following details are captured in the interface:

• Safari location (e.g., Amboseli, Tsavo, Maasai Mara ((for my case))
• Topics of interest (biodiversity, environment, etc., representative of the agents)
• Preferred language (English or Swahili)
• Duration of the tour (in minutes).
• Guide voice tone (e.g., friendly , professional, etc.)

The interface looks like the below. Remember to input your OpenAI key for the generation to happen. However, this can be customised for open-source models.

Once all fields are set and the user hits the “Generate Safari Tour” button, the backend calls this code:

final_tour = run_async(
    mgr.run, location, interests, duration, st.session_state.get("LANGUAGE")
)

In the above code, mgr is a variable assigned to the classSafariManager . Triggering mgr.run that generates an OpenAI trace ID for observability. The planner agent is first invoked to allocate minutes across sections based on word counts. The speech rate is set at 150 words per minute. You can play around with words per minute if you want the speech a bit slow or fast.

Each agent is then triggered using the runner.runabstraction. All prompts include the language, location, tone guidance, and word limit choices. Their outputs are then assembled by the orchestrator agent whose job, as mentioned above, is to put together the final narrative with smooth transitions between the sections. The output is quite detailed and interesting. A sample run output in English is captured below:

The Swahili narrative below is also actually good. I have actually learnt a number of new Swahili words in this output.

3. Generating the Audio Tour

The last part of the setup is to convert the above outputs to audio and that’s where TTS comes in. The below tts()function handles this part. I used OpenAI's speech endpoint for this conversion. The endpoint requires specification of the model to be used (gpt-4o-mini-tts) in this case, the text to be turned into audio (generated output), and the voice to speak the output (nova). All these configuration options can be found here.

response = client.audio.speech.create(
    model="gpt-4o-mini-tts",
    voice="nova",
    input=text,
    instructions="""
    You are a friendly, knowledgeable safari guide. Speak naturally and conversationally...
    """
)
response.stream_to_file(speech_file_path)

You can download and listen to the generated audio I generated here. They are in Swahili and English. Let me know whether the audio quality matches your expectations in the comments. That’s actually it for today.

4. Future Directions

The above pipeline demonstrates how practical agentic AI and TTS systems could be built. Beyond safaris, similar setups could be used in museum walkthroughs, classroom storytelling, and cultural heritage apps.

The fact that there are independent agents means that you can add new modules or simply update the current ones without entirely changing the whole app. A few notable improvements could be made here. I actually suggest you look at this in your free time:

• Human-in-the-loop feedback: Here, you can enable users to refine the agentic outputs before final assembly. This way, users will be able to get very personalised outputs.
• Personalization memory: This is a feature that could help remember user preferences across sessions or based on their tour history, thus hyper-personalised outputs.
• Multimodal input: This is ambitious, but letting users upload images of landscapes or animals around them and receive guided commentary will be golden. Let me know in the comments whether you’d like such an article.

5. Conclusion

AI Safari Guide is more than just a cool TTS and agentic AI demo — it’s a fully functional and extensible AI system. The app combines conversational generation and audio synthesis to deliver a personalised audio guide to users and is specific for Maasai Mara. However, this can be adapted for any other location if need be.

As usual, my code is open sourced in this repo. Clone it, set up your environment, and install the required packages in the file requirements.txt.

If you enjoyed this write-up or found it useful, please leave a clap, follow me, or drop a comment with your thoughts and feedback. If you want to check out my other articles:

• Uncovering Patterns and Trends in Ausgrid Power Outage Data
• How to Create an Entire RAG System as a Newbie
• Building a RAG System with MMR for Safaricom’s Smart Assistant

Creating a Personalized Safari Guide with Agentic AI and Text-to-Speech (TTS) was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.