Architecture SRE Report: Concurrency Engine 2.0 (ACE) & Note Stripping Telemetry
1. Executive Summary: The Death of the UPI Transaction Note
In peer-to-peer (P2P) automated payment verification, the industry standard has long relied on embedding an order identifier into the UPI transaction note (tn=fg_XXXXXXXX). When payment confirmation emails arrive from the banking partner, an IMAP regex engine matches the note, identifies the transaction, and executes the merchant callback.
Throughout 2026, mobile payment service provider (PSP) apps—most notably PhonePe, Paytm, Google Pay, Navi, and WhatsApp Pay—systematically restricted or overwritten the tn parameter on unverified merchant and personal UPI QR rails. For indie developers, Discord bot creators, and SMM panel operators, this created an existential vulnerability: payments arrived, but the identifying note was gone.
ARYANISPE and FamGateway (Ministry of MSME Registration: UDYAM-BR-28-0050000) conducted an exhaustive forensic analysis of our production database cluster. This report presents the empirical findings and details the architectural transition to FamGateway Concurrency Engine 2.0 (ACE).
Systems Architecture Landscape: Amateur Script vs. Production-Grade Gateway
Building logic for the "happy path" (when a solitary user tests a friend-to-friend transfer and remarks arrive) is trivial. Production payment gateways live entirely in engineering for the failure paths: handling aggressive third-party remark sanitization, database race conditions, and real-time reconciliation under peak concurrency:
| A Simple Script Thinks... | A Production Gateway Has to Think... |
|---|---|
| "I'll just look for the transaction note in the email." | "The note was completely stripped by PhonePe. I have to isolate this transaction using a mathematical micro-paisa slot array." |
| "Two people paid the same amount? I'll just credit the first order." | "If I credit the wrong order, I cause a multi-tenant accounting crisis. I must serialize incoming requests via ACID row-locks so race conditions are physically impossible at the database level." |
| "I'll just check the inbox every few seconds." | "If a customer waits more than 3 seconds for a digital delivery, they think they've been scammed. Connection pools must be kept open constantly via IMAP extensions to trigger sub-2 second webhooks." |
| "I'll force arbitrary random paise on every order (e.g. ₹99.37)." | "Arbitrary paise destroy checkout trust. 95%+ of checkouts must remain exact clean round figures (₹99.00), deploying micro-paisa slots purely during active concurrent collisions." |
2. Production Telemetry Analysis: 50,000+ Transaction Forensic Sample
Building upon FamGateway's milestone processing of over 9.51 Million requests across 8,000+ active merchants, an exhaustive forensic telemetry audit of a dedicated 50,000+ transaction sample window across the production cluster revealed that note stripping is not an isolated glitch, but an industry-wide default behavior on P2P payment rails.
The Competitor Telemetry Deficit: Why do open-source payment scripts and amateur aggregators fail to anticipate note stripping? Because they operate in a telemetry vacuum. When a hobbyist tests a script on 3 to 5 isolated peer transfers between friends, direct on-us bank transfers often preserve remarks. Only at FamGateway's scale—processing high-concurrency checkouts across diverse PSP switches (PhonePe, Paytm, Navi, GPay, Cred) and third-party mobile webview redirects—does the statistical reality become visible: over 16.33% of all P2P transactions arrive with stripped, corrupted, or overwritten purpose notes.
Observed Purpose Field (raw_purpose) |
Occurrences | PSP Source & Classification |
|---|---|---|
Empty String ("") |
5,678 | Generic UPI switch sanitization (Note dropped) |
Null Value (NULL) |
1,987 | Bank email header omitted purpose parameter |
PaymentfromPhonePe |
258 | PhonePe proprietary client override |
SentusingPaytmUPI |
160 | Paytm consumer app override |
PaidviaNaviUPI |
84 | Navi UPI app marketing override |
PAYBYWHATSAPP |
65 | WhatsApp Pay intent override |
PaidviaMobiKwikandgotcashback |
22 | MobiKwik cash-back promotional override |
Over 8,254 transactions (16.33% of aggregate volume) arrived with corrupted or missing purpose parameters. Under Engine 1.0, single transactions without notes were resolved via fallback amount matching. However, during flash sales or automated SMM panel traffic surges, multiple customers inevitably placed orders of identical denomination (e.g. ₹99.00 or ₹149.00) simultaneously.
Because Engine 1.0 strictly prohibited speculative order matching, it triggered a Zero-Risk Quarantine Hold:
[SYSTEM TELEMETRY] Zero-Risk Quarantine: 2 ambiguous orders match INR 149 for USR_MERCHANT_9B21. UTR: 6264XXXXXXXX. Held to prevent cross-customer misattribution.
While this quarantine successfully prevented financial loss or wrong fulfillment, it created customer friction and merchant support escalations.
Let $O = \{o_1, o_2, \dots, o_n\}$ represent the set of concurrent pending orders created for a specific merchant within the active session validity window $\tau = 600\text{s}$, where each order requests the exact same base catalog price $B \in \mathbb{R}^+$ (e.g. ₹99.00). Let $T = \{t_1, t_2, \dots, t_m\}$ be the incoming bank credit notification stream. Under mobile PSP remark stripping:
By Dirichlet's Box (Pigeonhole) Principle, any reconciliation function $f: T \to O$ operating purely on the tuple $\langle \text{merchant\_id}, \text{amount} \rangle$ maps $m$ items into a single equivalent state space. The misattribution collision probability is $P(\text{collision}) = 1 - \frac{1}{n}$. For $n \ge 2$, ambiguous collision is mathematically guaranteed ($P > 0$). Concurrency Engine 2.0 (ACE) injects an orthogonal micro-paisa frequency $\delta_i = i \times 0.01$ for $i \in \{0, 1, \dots, 99\}$, converting the payable amount into $P_i = B + \delta_i$. Because $P_i \ne P_j$ for all $i \ne j$, the mapping $f(t) \to o_i$ becomes an invertible bijective function: $P(\text{collision}) \equiv 0.00000\%$.
3. Engineering Architecture: Concurrency Engine 2.0 (ACE)
To permanently resolve the ambiguity dilemma without forcing awkward decimal randomizations on customers, FamGateway engineered Concurrency Engine 2.0 (ACE) featuring the Adaptive Micro-Paisa Shield (AMPS):
Checkout request opens database transaction and acquires exclusive row lock on merchant record. Parallel requests are serialized at InnoDB engine level.
SELECT id FROM users WHERE id = :userId FOR UPDATE;
Scans active, non-expired pending orders and payment links for this merchant within the 99-paisa micro-window. Builds occupied slot set.
SELECT COALESCE(NULLIF(payable_amount, 0.00), amount) AS eff FROM orders WHERE user_id = :u AND status = 'pending' AND expires_at > NOW()...
• Slot 0 (+₹0.00): Free in 95%+ of checkouts → Exact round catalog price assigned (e.g. ₹99.00).
• Slot 1..99 (+₹0.01..+₹0.99): Active collision detected → Assigns lowest unoccupied micro-paisa (e.g. ₹99.01, ₹99.02).
Serializes amount (catalog value ₹99.00 for merchant accounting/invoicing) and payable_amount (bank routing value ₹99.01 encoded in dynamic QR & intent). Commits ACID transaction.
Bank credit arrives matching payable_amount deterministically → UTR atomically locked → Order status transitions to success → Slot drops out of pending query → Instantly reclaimed for clean round figure re-use.
Core Mathematical Guardrails of ACE:
- Clean-First Round Figures (95%+ Traffic): Solitary checkouts never receive decimal alterations. If no other active order for that base price is pending for that merchant, the order is generated at
₹99.00. - Dynamic Collision Isolation: If Order 1 for ₹99.00 is pending, Order 2 is assigned
₹99.01, Order 3 is assigned₹99.02. Even when PhonePe completely wipes the purpose note, incoming receipts match 1:1 with zero ambiguity. - Instant Microsecond Slot Reclamation: The moment Order 1 is confirmed or hits its 10-minute expiry, Slot 0 drops out of the active query. Subsequent buyers automatically receive clean round figures again.
- ACID Transaction Serialization: All slot allocations are protected by MySQL
SELECT FOR UPDATErow-level mutexes, guaranteeing zero race conditions between concurrent requests.
4. Production Concurrency Benchmark & Empirical Stress-Testing
Following full deployment of Concurrency Engine 2.0 (ACE), our engineering infrastructure underwent rigorous live production stress testing and query execution profiling under multi-process concurrency:
| Stress Metric | Measured Production Value | SRE Target / Compliance |
|---|---|---|
| Concurrent Threads (Parallel Checkouts) | 50 Synchronous Invocations | ≥ 25 Threads |
| HTTP Success Rate (200 OK) | 100.00% (50 / 50 Success) | > 99.90% |
| Slot Collision Rate | 0.00% (50 Distinct Slots: ₹99.00 to ₹99.49) | 0.00% (Zero Collision Tolerated) |
| Arithmetic Slot Sequentiality | Strict Monotonic +₹0.01 Progression | Lowest-Free Unoccupied Index |
| Database Index Execution Type | Using where; Using index (Covering RAM) | Zero Disk I/O Block Reads |
| Active Slot Query Latency | 0.31 ms (Sub-millisecond) | < 5.0 ms |
| Microsecond Slot Recycling | Deterministic (Next Checkout Snaps to Slot 0) | Instant On Success / Expiry |
| Reconciliation Floating-Point Epsilon | |p_amt - found| < 0.005 | Strict micro-paisa isolation |
5. Reddit Community Discussions & Developer Inquiries (r/developersIndia, r/UPI)
Developers building payment integrations frequently ask about concurrency and note preservation across tech subreddits:
"Why does PhonePe strip the tn parameter on r/UPI?"
PhonePe's consumer client enforces anti-phishing sanitization on external deep-link intents to protect users from social engineering. For personal VPA handles, PhonePe strips user-specified transaction notes and replaces them with standard strings like PaymentfromPhonePe. Read our companion analysis in Why FamGateway Upgraded to Concurrency Engine 2.0.
"How to prevent duplicate UPI orders on r/developersIndia without ugly decimals?"
Amateur scripts force random paise on every transaction (e.g. ₹99.37), ruining conversion rates. FamGateway solves this through its Clean-First Adaptive Micro-Paisa Shield: 95%+ of orders remain clean round numbers, and +1 paisa micro-offsets are deployed purely during active concurrent collisions. Read Preventing Double Payments: Bank UTR Idempotency Guide.
"Does Concurrency Engine 2.0 affect merchant accounting and GST?"
No. FamGateway maintains a dual-ledger architecture. The amount field stores the exact business catalog price (₹99.00) for invoices and merchant databases, while payable_amount handles bank ledger routing. Webhooks and verification endpoints return both fields seamlessly.
6. Contextual Topic Mesh: Related Engineering & SRE Guides
Review our complete library of technical architecture guides and whitepapers:
- Core Architecture Blog: Why FamGateway Upgraded to Concurrency Engine 2.0 (ACE)
- Double Payment Prevention: Preventing Double Payments: Bank UTR Idempotency Locking Guide
- Concurrency & Race Conditions: How FamGateway Handles Concurrent Payments & Race Conditions
- Dynamic QR Code Generation: How to Generate Dynamic FamPay UPI QR Codes
- Cross-Tenant SRE Post-Mortem: Cross-Tenant Race Condition & Idempotency SRE Post-Mortem
- Scaling Under Peak Load: Scaling Under 1.07 Million Peak Requests & Bot Defense
- Developer API SDKs: Official FamGateway API Documentation
7. Frequently Asked Questions
What technical problem triggered the development of Concurrency Engine 2.0 (ACE)?
Production telemetry over 50,547 transactions revealed that over 7,665 payment notifications arrived with empty or stripped transaction notes due to aggressive PSP sanitization in apps like PhonePe, Paytm, and Navi. When concurrent buyers ordered identical SKUs, missing notes forced the engine into zero-risk quarantine holds to avoid misattribution. Engine 2.0 resolves this via deterministic Adaptive Micro-Paisa Isolation.
Why do major UPI apps strip pre-filled transaction notes?
Consumer payment apps deploy client-side sanitization on deep-link intents (upi://pay?tn=...) to prevent phishing attacks where fraudulent actors pre-populate notes like 'Refund'. Additionally, apps like PhonePe and Paytm overwrite notes with proprietary branding (e.g. 'PaymentfromPhonePe') on unverified P2P merchant rails.
How does the Adaptive Micro-Paisa Shield (AMPS) maintain clean-first pricing?
Unlike amateur scripts that force random fractional paise on every buyer, ACE keeps over 95% of transactions at exact round amounts (e.g., ₹99.00). An adaptive +1 paisa micro-shield (₹99.01, ₹99.02) is only allocated if an active pending order for the identical base price already exists for that specific merchant.
What is the order quarantine rate under Concurrency Engine 2.0?
Under Concurrency Engine 2.0, the ambiguity quarantine rate is 0.00%. Because every concurrent order occupies a mathematically unique payable slot, incoming payments match deterministically to exactly one pending record even if the transaction note is completely destroyed.
How are paisa slots reclaimed after order fulfillment?
Paisa slots are dynamically reclaimed at the microsecond level upon order success or after the 10-minute session expiration window, instantly restoring the base clean amount for subsequent buyers.
How does Engine 2.0 maintain merchant accounting and tax ledger integrity?
FamGateway implements a dual-ledger payload: 'amount' reflects the catalog price (₹99.00) for invoices and GST exemptions, while 'payable_amount' reflects the bank routing value (₹99.01). Merchant databases experience zero ledger or schema discrepancies.
8. Verification & Institutional Governance
Concurrency Engine 2.0 (ACE) is live across all production cluster nodes. All database queries, transaction locks, and slot allocation benchmarks have been validated against live traffic. For implementation details, consult the Developer Documentation or contact founder Aryan Gupta directly via Official Status & Support.
Related Corporate & Technical Disclosures
Multi-Wallet GA Telemetry & Concurrent Throughput Benchmark Report
Empirical systems telemetry report certifying FamGateway's Multi-Wallet engine following General Availabili...
Architecture Report: Multi-Wallet Closed Beta & Daily Limit Bypass Audit
Empirical study on solving NPCI individual receiver limits ('Receiver reached daily limit') via dynamic mul...
60-Day Milestone & Enterprise Valuation Disclosure Report (October 2026)
Official enterprise valuation report valuing ARYANISPE parent infrastructure at INR 1.20 Cr - 1.75 Cr, veri...