artifacts/toby-frontend-doctor/c1bf20e9-d112-429a-817a-986e7a08ce2f/finding.mdFrontend finding — `retention_offers` silent
TL;DR
The retention save-offer UX does exist, lives only in apps/extension, and the in-app cancel path does call the retention endpoints when the user clicks through it. Hypothesis A from the ticket ("UX never reaches the save-offer step") is only partially correct — the modal step exists, but there are three concrete reasons it can stop producing rows in retention_offers, all of them frontend-attributable in part:
retention_offersis append-only on ACCEPT by design. Every "0/0 in 30 days" data point is consistent with "0 users clicked CLAIM DISCOUNT in 30 days" — which doesn't necessarily mean the modal never showed. The ticket framing conflates "offers issued" with "rows inretention_offers". The DB schema only stores acceptances. (This is a re-read of Hypothesis A.)- There are at least two bypass paths from the same in-app Subscription panel that route the user to the Stripe Customer Portal and let them cancel without ever touching the retention modal. The portal cancellation never records a cancellation reason and never creates a retention_offers row.
- The cancel button is hidden entirely for
team_legacyandteam_basicplans (hasSubscriptiongate atSubscription.tsx:41-43). The biggest churn cohort in Feb 2026 was ThankYouLegacy renewals (perproduct/metrics/surveys/churn-survey-analysis.md) — those users have no in-app cancel CTA, so they cancel via Stripe email links and never see retention.
Backend should verify (a) whether enableRetentionOffer flag is still on in production for last 30d, (b) whether getRetentionOffer is returning eligible:false for everyone (cooldown / age / DB error), and (c) what the recent Stripe webhook volume vs. recorded cancellation_reasons rows looks like — that ratio quantifies the bypass.
Surface inventory
| Surface | Has cancel UX? | Goes through retention modal? |
|---|---|---|
apps/extension (Chrome new-tab) | Yes — only entry point | Yes, IF user clicks the explicit "Cancel subscription" link AND is not on team_legacy/team_basic AND backend returns eligible:true AND user does not also click the "Billing & Invoices → View" link first |
apps/landing (gettoby.com) | No — marketing static site (Astro). Verified live: only a "Pricing" link, no auth / account / billing surfaces. | n/a |
apps/mobile (Expo) | No cancellation code present (only Settings/Updates) | n/a |
app.gettoby.com | "Service Unavailable" — no web app deployed | n/a |
| Stripe Customer Portal (off-app) | Yes — directly cancellable from Stripe email links, Toby's "View invoices" link, or anyone bookmarked URL | No — bypasses Toby entirely |
So the extension is the only Toby-controlled cancellation surface.
The in-app cancel flow (what does exist)
Entry → modal → retention API call → conditional retention modal:
apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:228-240
└── <Link onClick={handleCancelPlan}>Cancel subscription</Link>
└── handleCancelPlan() → openModal(CANCEL_PLAN) (line 80-83)
apps/extension/app/components/ProfileEntitySidebar/Spaces.tsx:596-605
└── {currentOpenModal === CANCEL_PLAN && <CancelSubscription ... />}
apps/extension/app/components/Modal/Downgrade/CancelSubscription.tsx
└── Step1 (informational, line 119-208)
└── Step2 (informational "Are you sure?", line 210-282)
└── Step3 (cancellation reason survey, line 284-441)
└── handleSubmit / handleSkip → onNext(getSurveyOptions())
└── handleCheckRetentionOffer(surveyOptions) (line 643-709)
├── recordReasonMutation.mutateAsync(...) ← writes cancellation_reasons
├── if (!isRetentionOfferEnabled) → cancel ← FEATURE FLAG GATE
├── api.subscriptions.getRetentionOffer(team.id) ← BE ELIGIBILITY GATE
├── if (eligible && offer)
│ └── setShowRetentionOffer(true)
│ └── trackEvent(RETENTION_OFFER_SHOWN, ...) ← Amplitude only
└── else → handleCancelSubscription() → Stripe portal redirect
When the user lands on the retention modal:
apps/extension/app/components/Modal/Downgrade/RetentionOffer.tsx:644-669
├── <Button onClick={onClaimDiscount}>CLAIM DISCOUNT / SWITCH / KEEP MY PLAN</Button>
│ └── handleClaimDiscount() (CancelSubscription.tsx:555-597)
│ └── trackEvent(RETENTION_OFFER_ACCEPTED, ...) ← Amplitude
│ └── acceptOfferMutation.mutateAsync(team.id) ← THIS is what writes retention_offers
└── <Button onClick={onContinueToCancel}>CANCEL ANYWAY</Button>
└── trackEvent(RETENTION_OFFER_DECLINED, ...) ← Amplitude only — NOT written to retention_offers
└── handleCancelSubscription() → Stripe portal
So a retention_offers row only ever appears when the user reaches the modal and clicks the discount button. Decline never writes. Show never writes (only Amplitude).
This corroborates a note in product/metrics/surveys/churn-survey-analysis.md:128: "The retention_offers database table only records accepted offers (14 rows)... The real accept rate is ~14%."
The "16 rows all-time" in the ticket is consistent with this schema interpretation — these are 16 historical acceptances, not 16 offers ever shown.
Three frontend-dimension contributors to the silence
1. Stripe Customer Portal bypass — preloaded in the same Subscription panel
apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:51-62, 192-208 always pre-fetches a Stripe Customer Portal URL on render and exposes it as a <Link href={stripeUrl} target="_blank">View</Link> next to "Billing & Invoices":
// Subscription.tsx:51-62
useEffect(() => {
if (!hasSubscription) return;
api.subscriptions
.patch(team.id) // → PATCH /teams/{id}/admin/subscriptions, returns {url}
.then((res) => setStripeUrl(res.url))
...
}, [team?.id, hasSubscription]);
api.subscriptions.patch (endpoints.ts:692-697) returns a Stripe Customer Portal session URL. Stripe Customer Portal sessions can be configured to allow self-serve cancellation, and the resulting cancel does NOT call back into Toby's retention flow. The Stripe webhook will subsequently mark the subscription as canceled, but cancellation_reasons and retention_offers stay untouched.
Any user who clicks "View" (perhaps to download an invoice, change card, or check renewal date) and then clicks "Cancel plan" inside Stripe's portal bypasses retention entirely. I cannot quantify this from the FE side — the backend doctor can compare:
- count(subscriptions canceled in last 30d) vs. count(cancellation_reasons rows in last 30d)
If the gap is large, this bypass is real and significant.
2. Stripe email / external-link bypass (off-product)
Stripe sends renewal-notice and receipt emails that contain a "Manage subscription" link → Customer Portal. Users clicking from email never load the extension, never see the retention modal. Same outcome: cancellation_reasons stays empty, retention_offers stays empty.
This is the same Customer Portal as path (1) — the surface is just an email instead of the in-app "View" link.
3. team_legacy users have no in-app cancel CTA at all
apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:41-43:
const hasSubscription =
!!team?.paymentCustomerID &&
!['team_legacy', 'team_basic'].includes(team.accessRole);
The "Cancel subscription" link only renders inside {hasSubscription && (...)} blocks (line 217 and line 186). Legacy users see neither the Billing & Invoices View link nor the Cancel button. The only way they can cancel is to:
- click a Stripe email link → Customer Portal → cancel there, OR
- contact support, who probably does it manually via Stripe dashboard.
Both paths skip the retention modal.
Per product/metrics/surveys/churn-survey-analysis.md:22: "February 2026 was the peak churn month — coincides with ThankYouLegacy renewals (3x price increase from $0.99/mo to $3.01/mo)." So the cohort with the highest churn in the recent window is exactly the cohort that has no FE retention path. This alone is a plausible explanation for why offer creation skews even more sharply toward zero in recent windows: legacy renewals create cancellation pressure that is structurally invisible to the FE retention modal.
What likely caused the recent drop to 0 (last 30 days)
Hypotheses, ranked by my confidence:
| # | Hypothesis | Confidence | Resolving evidence |
|---|---|---|---|
| H1 | The 0 figure represents 0 accepts, not 0 shows. Offers may still be shown in Amplitude but nobody clicked claim. Decline rate is normally ~86% per the churn analysis. | medium-high | Backend / analytics doctor can pull Amplitude retention_offer_shown count for last 30d. If > 0, this hypothesis stands. |
| H2 | Most recent cancellations were team_legacy users (Apr-May 2026 likely had another renewal wave) and they have no in-app cancel CTA. | medium-high | Backend can run: SELECT access_role, COUNT(*) FROM teams JOIN subscriptions WHERE canceled_at >= now() - 30 days. |
| H3 | The recently-deployed AuthWrapper hydration gate (commit d68726b29, prior incident blank-extension-page) is keeping a meaningful fraction of users out of the new-tab page entirely. Those users still get to Stripe to cancel via email. | medium | Cross-reference with the blank-extension-page incident impact estimate. |
| H4 | enableRetentionOffer feature flag silently flipped to false or its CoreAPI endpoint started returning empty. Frontend treats undefined as false (featureFlagsV2?.enableRetentionOffer === true — CancelSubscription.tsx:471), so any flag-loading regression skips retention universally. | medium | Backend should verify the flag in CoreAPI and check whether the Experiments query returns non-empty. |
| H5 | Backend getRetentionOffer started returning eligible:false universally (e.g., new bug introduced by cbc92a78d on 2026-03-31, or a DB error path being swallowed). Frontend's catch block (CancelSubscription.tsx:702-708) routes any error directly to handleCancelSubscription, bypassing retention. | medium | Backend can check getRetentionOffer HTTP 200 rate and ratio of eligible:true responses in the last 30 days vs. before. |
H1 and H2 together likely account for most of the 30-day silence; H3-H5 are amplifiers I cannot rule out without backend telemetry.
Reproduction
Could not perform a live end-to-end reproduction: the extension build is not pre-compiled (apps/extension/.output/ does not exist, node_modules is empty), and a live cancel requires a signed-in user with an active paid subscription. Playwright cannot load an unpacked Chrome extension without a persistent context, and I am not authorized to write to the codebase or run pnpm install && pnpm dev. (Anti-pattern from prior runs.)
Source-level reproduction is dispositive. All file paths and line ranges cited above were read directly from the codebase at HEAD as of this run.
Public-web verification (live):
- Navigated Playwright to
https://www.gettoby.com/. Page contains only a "Pricing" link. No login, no billing, no cancel flow. Confirms there is no web-app surface that could route around the extension. - Navigated Playwright to
https://app.gettoby.com/. Response:Service Unavailable. No deployed web app.
Evidence — key file paths and line ranges
| What | Path |
|---|---|
| Cancel CTA (only entry point) | apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:228-240 |
hasSubscription gate (hides CTA for legacy/basic) | apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:41-43 |
| Stripe portal URL preload (bypass surface) | apps/extension/app/components/Modal/OrgSettings/Subscription.tsx:51-62, 192-208 |
| Modal mount | apps/extension/app/components/ProfileEntitySidebar/Spaces.tsx:596-605 |
| 3-step modal flow + retention dispatch | apps/extension/app/components/Modal/Downgrade/CancelSubscription.tsx:455-769 |
| Feature flag gate | apps/extension/app/components/Modal/Downgrade/CancelSubscription.tsx:469-471, 668-672 |
recordCancellationReason mutation | apps/extension/app/state/routers/subscription.ts:54-60 |
getRetentionOffer query | apps/extension/app/state/routers/subscription.ts:61-65 |
acceptRetentionOffer mutation (the only path that writes to retention_offers) | apps/extension/app/state/routers/subscription.ts:66-73 |
| HTTP endpoints | apps/extension/app/api/endpoints.ts:692-781 |
| RetentionOffer modal UI | apps/extension/app/components/Modal/Downgrade/RetentionOffer.tsx |
| Amplitude event names | apps/extension/app/constants/App.ts:387-393 |
Evidence — recent commits
| Commit | Date | Effect |
|---|---|---|
cbc92a78d Make retention discount eligible for all cancellation reasons | 2026-03-31 | Removed RETENTION_ELIGIBLE_REASONS array and isReasonEligibleForRetention gate from FE. After this commit, FE always calls getRetentionOffer when reason is recorded. Should have increased offer volume, not decreased it. |
d68726b29 fix: gate AuthWrapper on user hydration | 2026-04-09 | Widened blank-extension-page hang surface (see prior incident blank-extension-page). Users stuck on the skeleton cannot reach Org Settings → Cancel. |
No FE changes to Subscription.tsx, Spaces.tsx, CancelSubscription.tsx since 2026-03-31 | — | The cancel modal entry flow itself has not changed in the 30-day window where retention_offers went to zero. The cause is not a recent UI regression in the cancel path. |
Evidence — Amplitude event semantics
These FE events fire (CancelSubscription.tsx):
RETENTION_CANCEL_INITIATEDon step 0 → step 1 (line 539-547)RETENTION_CANCEL_REASON_SELECTEDafter the reason POST succeeds (line 656-661)RETENTION_OFFER_SHOWNwhenofferResponse.eligible && offerResponse.offerand modal mounts (line 690-696)RETENTION_OFFER_ACCEPTEDon claim discount click (line 559-565) — fires before the BE writeRETENTION_OFFER_DECLINEDon "cancel anyway" click (line 622-627)RETENTION_SUBSCRIPTION_CANCELLEDonhandleCancelSubscriptionstart (line 603-606)
RETENTION_OFFER_SHOWN count vs. RETENTION_CANCEL_REASON_SELECTED count is the cleanest FE-side telemetry to disambiguate hypotheses H1-H5. Backend/analytics doctor: please pull these two events for last 30d and the 12-week prior window.
Root-cause hypothesis (overall)
There is no single FE bug causing the silence. The retention pipeline is architecturally:
[in-app modal path] → records cancellation_reason → maybe writes retention_offers
↑↑↑
(this is the only path that can write)
[Stripe portal path] → Stripe webhook → updates subscriptions only
↑↑↑
(this path is invisible to retention)
The Stripe portal path is reachable from inside the same in-app Subscription panel (via "View") AND from every Stripe email, AND it is the only path available to legacy users. The proportion of cancellations going through each path is invisible to the FE — the FE only ever sees its own modal traffic.
The 30-day zero strongly suggests the proportion shifted toward the Stripe-portal path (driven by legacy renewals, blank-page incident eating into modal traffic, or both), and/or users who saw the offer didn't accept (decline-but-cancel is the modal flow that writes zero to retention_offers).
The fix is not a frontend bug fix. It is a combination of:
- Treat the
retention_offerstable as the wrong place to measure offer issuance — add aretention_offer_shownserver-side write when BE returnseligible:trueand the FE-sideRETENTION_OFFER_SHOWNevent fires (or, simpler, changeretention_offersschema to insert a row ongetRetentionOffersuccess, not on accept). This makes the funnel visible to ops. - Either disable Stripe Customer Portal self-serve cancellation, OR add a Stripe portal
flow_dataconfiguration that redirects "Cancel plan" back into the Toby cancel flow (Stripe supports this). - Surface a Cancel CTA for
team_legacyusers (currentlySubscription.tsx:41-43excludes them). Their churn pressure is structurally invisible to the retention path.
Confidence in this finding: high on the architectural shape (multi-path cancellation, append-on-accept schema, legacy invisibility); medium on which specific hypothesis (H1-H5) accounts for the 30-day zero — that needs backend telemetry.
What I want from the backend doctor
defer_to: backend on these specific questions:
- Stripe portal cancel traffic vs. modal traffic. Compare
count(subscriptions canceled in last 30d)vs.count(cancellation_reasons inserted in last 30d). The delta is the lower bound of Stripe-portal bypass. getRetentionOffereligibility outcomes. In the last 30 days, what fraction ofgetRetentionOffercalls returnedeligible:true? WhichReasonstrings (subscription_too_new/cooldown_active/invalid_reason) dominated wheneligible:false?- Cancellations by
access_rolein last 30 days. Areteam_legacyusers an outsized share? If yes, hypothesis H2 stands. enableRetentionOfferflag state. Confirm the CoreAPI feature flag is still returningenableRetentionOffer: true(or whatever the expected payload is) for all users, includingteam_legacy. The FE coercesundefinedtofalse.- Amplitude RETENTION_OFFER_SHOWN events in the last 30d vs. the prior 12 weeks. If
RETENTION_OFFER_SHOWN > 0in last 30d, hypothesis H1 (decline-only) is confirmed and we know the modal is reaching users.
I expect the backend findings to resolve H1 vs. H2-H5 cleanly.