Aivinn Challenge System: A Complete Redesign — Case Study

Table of Contents

1. Background & Problem Statement
2. System Architecture Decisions
3. AI Tool Layer: AssignChallengeTool & IncrementChallengeTool
4. Transaction-Triggered Evaluation: evaluateChallengeProgress
5. Cron Jobs: The Backbone of Automation
6. Shared Logic & Deduplication
7. Testing & Verification
8. Lessons Learned

Background & Problem Statement

Aivinn's Challenges feature lets users opt into system-generated financial habit challenges (e.g. "Avoid Coffee & Snacks," "Track Expenses Daily"). Unlike Goals, Challenges are not user-editable — they're suggested by the system, accepted by the user, and tracked automatically.

The original implementation had two tools (AssignChallengeTool, IncrementChallengeTool) wired into an AI agent, plus a partial evaluation function (evaluateChallengeProgress) that only handled 2 of 5 challenge types. The system had three concrete problems:

1. No proactive suggestion path. Challenges were only ever suggested reactively, when a user's chat message happened to match a category keyword. A user (including our own CEO) who never typed something like "I want to save on coffee" would simply never receive a challenge.
2. Three of five challenge types were dead code. AVOID, STREAK, and SAVE types existed in the schema and had seed data, but nothing ever evaluated their progress — they'd sit at 0% forever.
3. No lifecycle management. Challenges never expired, never got cleaned up, and stale SUGGESTED rows could accumulate indefinitely.

This case study documents the full redesign: five cron jobs, a rewritten evaluation pipeline, and tool-layer fixes — built incrementally, with each architectural decision made deliberately rather than assumed.

System Architecture Decisions

Before writing any code, several foundational questions needed answers. Getting these wrong early would have meant rebuilding later.

Decision 1: User-centric, not wallet-centric

The question: Aivinn wallets can be shared between multiple users. Should a challenge belong to a wallet (shared) or a user (individual)?

The answer: Challenges belong to the wallet owner only — never to wallet members. This was already implicit in the schema (user_challenges has both user_id and wallet_id, with the uniqueness constraint keyed on user_id), but needed to be made explicit as a rule: wallet_id exists purely to know which wallet's transactions to evaluate against, not to determine who the challenge belongs to.

Why this matters: if two people share a wallet, only the owner receives suggestions and notifications. This avoids ambiguity around "whose challenge is this" and "who gets credit" — decisions that would otherwise need their own arbitration logic.

Decision 2: Concurrency cap — 3 total, 2 suggested

A user can have at most 3 challenges in play at once (combined SUGGESTED + ACTIVE), with a sub-limit of 2 in SUGGESTED state. This means:

Why: an earlier draft capped it at just 1 in-flight suggestion, which felt too restrictive — offering almost nothing to engage with. But no cap at all risks overwhelming the user with a wall of suggestions. 3/2 was chosen as a middle ground: enough variety to feel alive, tight enough to stay uncluttered.

Decision 3: Expiry over deletion where possible

Two different cleanup philosophies were needed for two different states:

• Unaccepted SUGGESTED challenges → hard-deleted after 2 weeks. There's no analytical value in a suggestion nobody engaged with, and no field was added to track "why" — it's just gone.
• Resolved EXPIRED/COMPLETED challenges → kept for 3 months before deletion. This preserves a window for analytics (completion rates, reward history) before the row is purged.

Why the asymmetry: unaccepted suggestions carry no signal — a user never even looked at it. Resolved challenges carry real signal (user engaged, succeeded or failed) and are worth retaining briefly for product insight.

Decision 4: rules.evaluation_window_days over challenges.duration_days as the source of truth

The challenges table has a duration_days column, and the JSON rules field (personalized per user at assignment time) has its own evaluation_window_days. These can diverge — the assignment tools only ever read/write evaluation_window_days, never touching duration_days. Using the wrong one would mean expiry timing silently drifting from what the user was actually told.

Resolution: every expiry/deadline calculation in the system reads user_challenges.rules.evaluation_window_days, falling back to a default (30 days) if absent. duration_days is effectively legacy and unused going forward.

Decision 5: Type-specific failure semantics

Not all challenge types fail the same way. This took the most back-and-forth to settle:

• AVOID ("don't spend on X for N days"): a single violating transaction should fail the challenge immediately — there's no partial credit for "mostly avoiding" something you were told to avoid entirely.
• STREAK ("do X daily for N days"): a missed day should reset progress to 0, not fail the challenge outright — the user still has the remaining window to build a fresh streak. This was a deliberate UX choice: a punitive "one missed day = game over" felt discouraging, whereas a reset-and-encourage approach ("Streak Broken 💔 — log a transaction today to start again!") keeps the user engaged rather than abandoning ship.
• LIMIT / SAVE: these are threshold-based, not event-based — progress is a continuous ratio (spend vs. limit, or savings vs. goal), recalculated daily/on-transaction until the window closes.

AI Tool Layer

AssignChallengeTool

This is the entry point when a user expresses challenge-relevant intent through chat (e.g. "I want to spend less on coffee").

Original design flaw: the tool only matched challenges of type LIMIT or FREQUENCY (an MVP-era restriction), always picked the first matching challenge deterministically (matchedChallenges[0]), and had no concept of the concurrency cap — a chat-triggered assignment could push a user past the 3/2 limit that the cron jobs otherwise respect.

Final implementation:

Loading code snippet…

Why personalization matters: min_spend_threshold is adjusted to 80% of the user's actual recent spend in that category (never below the template's floor). A flat, un-personalized limit would be either trivially easy (irrelevant to a low spender) or impossibly hard (unfair to a high spender). Personalizing it against real getUserMetrics data makes the challenge meaningfully calibrated per user.

Why strict category matching, no fallback: the tool description explicitly forbids guessing. Financial suggestions carry real weight — assigning the wrong challenge based on a loose keyword match (e.g. matching "car" to "Transportation" when the user meant something else) erodes trust in the AI's judgment. If nothing matches cleanly, the tool declines and redirects to Goals instead of guessing.

IncrementChallengeTool

Called when a user explicitly reports progress in chat (e.g. "I avoided coffee today").

Original design flaw: the schema required a userId parameter that the function body never used — dead input, and a missed opportunity for a safety check.

Final implementation:

Loading code snippet…

Why add the ownership check: challengeId is AI-supplied from conversation context, not deterministically fetched. If the model ever hallucinates or carries over a stale ID from a different user's context, incrementing progress on the wrong person's challenge would be a real data integrity bug. The check costs one indexed query and closes that gap outright — using data (userId) the tool was already receiving but silently discarding.

Transaction-Triggered Evaluation

evaluateChallengeProgress runs on every transaction create, update, and delete. It's the reactive half of the system — the half that responds instantly to user behavior, as opposed to the cron jobs, which handle everything time-based or passive.

Why some logic lives here and not in cron: anything that needs to react the moment a transaction happens — like an AVOID challenge failing the instant a disqualifying purchase is logged — can't wait for a daily cron run. A day's delay between the violation and the notification would feel broken; the user would keep thinking their challenge was still alive.

Loading code snippet…

Why AVOID lives here but STREAK and SAVE don't: AVOID's success condition is defined by the absence of a matching transaction — its failure condition is the presence of one, which is inherently event-driven. STREAK ("did the user log something today") and SAVE ("what's the net income-minus-expenses so far") are both aggregate, time-window questions that only make sense to answer once per day, not on every individual transaction. Trying to evaluate a streak or savings target on every transaction would be both wasteful (recomputing the same daily aggregate repeatedly) and semantically wrong (a single transaction doesn't determine a streak).

Cron Jobs

Five daily jobs were built, each with a distinct, non-overlapping responsibility. All run via node-cron against config.cron_timezone (Europe/Stockholm).

1. expiredSuggestionsCleanupJob

Purpose: deletes SUGGESTED challenges nobody ever accepted, after 2 weeks.

Loading code snippet…

Why hard delete, no status transition: unlike resolved challenges (which carry analytical value), an unaccepted suggestion represents a decision the user never made. There's nothing to learn from it that the raw suggestion count elsewhere in the system doesn't already tell you. Keeping the row around indefinitely just accumulates clutter blocking the suggestion cap slot it was occupying.

2. expiredChallengesJob — the core resolution engine

Purpose: the single daily pass that resolves every ACTIVE challenge type. This is the most complex job in the system, because each challenge type has fundamentally different resolution logic.

Loading code snippet…

Why one job handles five different resolution strategies instead of five separate jobs: all five types share the same iteration surface (ACTIVE challenges) and the same daily cadence. Splitting them into separate jobs would mean five separate queries fetching largely overlapping data, and — more importantly — five places where the deadline-calculation logic (rules.evaluation_window_days, pastDeadline) could drift out of sync with each other if one got updated and the others didn't. One job, branching by type, keeps the shared primitives (deadline math, expireChallenge) genuinely shared.

A real bug caught during this design (worth documenting): an early version of the STREAK/SAVE branches checked the stale uc.progress (fetched before the loop's mutations) when deciding whether to also expire the challenge. This meant a challenge that had just been marked COMPLETED in the same pass could immediately be marked EXPIRED right after — because the deadline check was comparing against the old, pre-update progress value. Fixed by tracking newProgress locally within each branch and using that for the expiry decision instead.

The LIMIT safety-net, explained: evaluateChallengeProgress normally resolves a LIMIT challenge's final outcome only when a transaction triggers the check near the window's end. But if the user simply doesn't transact again in that category after their last purchase, nothing ever fires that final check — the challenge would sit at, say, 40% progress forever, even if the user genuinely succeeded (stayed under threshold). The safety-net closes this: the cron independently re-checks total spend against the threshold once the window has closed, regardless of whether a transaction happened to trigger it.

3. oldChallengesCleanupJob

Purpose: purges resolved challenge history after 3 months.

Loading code snippet…

Why completed_at for COMPLETED but last_updated_at for EXPIRED: EXPIRED challenges never set completed_at (only successful ones do) — last_updated_at is the only timestamp reliably set at the moment either outcome occurs, since expireChallenge explicitly sets it.

4. proactiveSuggestionJob — solving the "no regular challenges" problem

Purpose: this is the direct fix for the original complaint (the CEO never receiving challenges). It runs daily, scans every wallet, and — for users under the concurrency cap — assigns a new challenge based on real spending metrics, without requiring any chat interaction.

Loading code snippet…

Why daily cron and not per-transaction: proactive suggestion is fundamentally a discovery problem — "here's something you haven't been doing" — evaluated against a 30-day rolling view of spending (getUserMetrics), not a single event. Running this per-transaction would mean redundant recomputation on every logged expense and risk firing multiple suggestion attempts in rapid succession if a user logs several transactions in one sitting. A daily cadence naturally throttles this to one meaningful check per day.

Why global (non-category) challenges are included as candidates: initially the job only considered category-matched challenges (LIMIT/AVOID/FREQUENCY tied to actual spending categories). But this meant users with weak or scattered spending patterns — or ones who'd exhaust their obvious category matches — would never see habit-building challenges like "Track Expenses Daily." Including global challenges in the same candidate pool, subject to the same tier-priority picking, ensures variety without requiring a special-case fallback path.

5. reactivationJob — the inactivity-driven flows

Purpose: re-engage users who've gone quiet, through two independent signals.

Loading code snippet…

Why two independent checks instead of one: "hasn't logged a transaction" and "hasn't opened the app" are different failure modes with different fixes. A user might open the app daily to check balances but never log anything (transaction-lapse without app-lapse), or vice versa. Conflating them into one signal would mean sending the wrong message to the wrong situation — e.g. offering a "log for 10 days" challenge to someone who hasn't even opened the app to see it.

Why the transaction-lapse gets a challenge but the app-lapse only gets a notification: a challenge needs the user to be present to accept it — pre-assigning one to someone who isn't opening the app creates ambiguous state (when does the challenge's clock start if they're not around to start it?). The app-open nudge is a simpler, lower-commitment ask: just come back. Once they do, the proactive suggestion job or the transaction-lapse check can take over naturally.

A known limitation, documented but deliberately deferred: both checks use exact-day matching (daysSinceLastTx !== 14, daysSinceActive === 14). If the cron job doesn't run on the exact day a user crosses the threshold (server downtime, deploy window, etc.), that user is silently skipped forever for that cycle — there's no catch-up mechanism, unlike expiredChallengesJob's 2-day window check. This mirrors a bug class we fixed elsewhere in the system, and is flagged here as a known follow-up rather than solved immediately, to avoid scope creep mid-build.

Shared Logic & Deduplication

Two pieces of logic were duplicated across AssignChallengeTool and proactiveSuggestionJob early in the build — the concurrency cap check, and the "which candidate to pick" tier logic. Left duplicated, any future change to the cap rule or picking algorithm would require remembering to update both places, risking silent drift between the chat-triggered and cron-triggered assignment paths.

Loading code snippet…

Why "most recent status per challenge" rather than "ever succeeded/failed": a user could theoretically fail a challenge once, then succeed at it later on a subsequent attempt. Ranking by the most recent outcome (rather than "has this ever happened") reflects their current relationship with that specific challenge, not a permanent historical record. A past failure shouldn't permanently deprioritize a challenge the user has since mastered — likewise, a past success shouldn't stay marked as "highest priority to avoid repeating" if they later failed it in a subsequent attempt.

Why successful challenges stay eligible (just lowest priority) instead of being excluded forever: habits regress. A user who beat "Limit Coffee Spending" three months ago might benefit from being offered it again if their spending drifts back up. Permanently excluding successes would mean the pool of assignable challenges shrinks monotonically over time for engaged users — eventually leaving nothing to suggest.

Generic typing (<T extends { id: string }>): pickBestCandidate is called with two different shapes — a lean { id: string } list in early drafts, and the full challenges row (with title, description, rules) in the final version. Making it generic means the function preserves whatever fields the caller passed in, avoiding a TypeScript error where properties like .rules or .title would otherwise be typed away.

Testing & Verification

Rather than deploying to trigger these on their real daily schedule, verification was done directly against the running dev server — cron jobs execute identically regardless of environment, so no deployment was needed.

Method: each job's cron.schedule(...) expression was temporarily set a few minutes ahead of the current server time (in config.cron_timezone), combined with direct SQL manipulation of the relevant timestamps to simulate the conditions each job checks for:

Loading code snippet…

Each job also received a start-of-execution console.log (e.g. [Reactivation] Job started at ...), separate from its end-of-run summary log, so a run could be confirmed as having fired even before checking database state.

Bugs caught during this process:

1. A null as any cast used to work around a Prisma type error actually still evaluated to a real null at runtime, which Prisma rejects outright for not: null filters — resolved by discovering last_active_at was non-nullable in the schema, making the filter unnecessary entirely.
2. The STREAK/SAVE stale-progress expiry bug described earlier (a challenge could be marked COMPLETED and EXPIRED in the same pass) — caught during code review before it ever reached production testing.

Verified working end-to-end:

• reactivationJob: both the no-transaction (challenge + notification) and no-app-open (notification only) paths.
• proactiveSuggestionJob: correctly assigned a second challenge to a user already holding one, respecting the concurrency cap and picking from real spending-category matches.
• expiredChallengesJob: confirmed both an AVOID challenge resolving to COMPLETED (survived its window with no violation) and a STREAK challenge resolving to EXPIRED (missed days, reset, then ran out its window) in the same test run.

Lessons Learned

1. Schema fields that "could" diverge eventually will. The duration_days vs. rules.evaluation_window_days mismatch wasn't hypothetical — the personalization logic already only ever touched one of them, meaning any challenge template without an explicit evaluation_window_days in its rules would silently disagree with its own duration_days the moment it got personalized.

2. "Simple" defaults (e.g. skip personalization) often aren't actually simpler in outcome — they just move the complexity into user experience inconsistency instead of code. The decision to add personalization to proactiveSuggestionJob even though it complicated the code slightly was the right call, because the alternative (identical challenge terms for every user regardless of spending) would have undermined the point of proactive suggestion.

3. Deferred scope should be written down, not just remembered. Sub-category granularity (e.g. distinguishing Uber from bus within "Transportation"), sentiment-triggered challenge assignment, and the exact-day-match fragility in reactivationJob were all identified mid-build and explicitly parked rather than solved immediately — kept out of scope creep, but captured for a planned full-system review rather than lost.

4. Cap and picking logic belonging in two places instead of one is a latent bug waiting to happen. Extracting isUnderChallengeCap and pickBestCandidate into shared functions wasn't just a cleanliness pass — it closed a real risk where the chat-triggered and cron-triggered assignment paths could silently diverge in behavior after any future tweak to either.

5. Test by simulating time, not by waiting for it. Manipulating timestamps directly and temporarily shifting cron schedules a few minutes ahead made it possible to verify five distinct time-dependent behaviors (14-day inactivity, 21-day inactivity, 7-day AVOID window, 10-day STREAK window, deadline-crossing expiry) all within a single working session, without needing to wait real days between each test.