Test Before You Launch: Why Credit Unions Can't Skip Product and Conversion Testing
Oct 23, 2026 · @John Warner III
Written by JGW Advisory Group
A credit union's most valuable asset isn't on its balance sheet. It's the quiet confidence members have that their paycheck will post on Friday, their debit card will work at the grocery store, and the loan rate they were quoted is the rate they'll actually pay.
That confidence is built over decades and can be damaged in a single weekend. The most common way it happens isn't fraud or a market shock. It's a launch that went live before it was ready: a core conversion, a new digital banking platform, a new card program or a new loan product that was never properly tested.
The argument of this article is simple. For a credit union, thorough pre-launch testing is not an IT nicety or a line item to trim when the schedule slips. It is member service, risk management and regulatory compliance all at once.
What's really at stake
When a launch fails at a credit union, the damage lands in four places at once.
Members' money. A misconfigured dividend rate, a fee table loaded wrong, or a loan that accrues interest on the wrong basis is not a cosmetic bug. It takes or withholds real dollars from real people, often thousands of them before anyone notices.
Access. Members who can't log in, can't move money between accounts, or can't make a loan payment end up with late fees, overdrafts and missed bills they did nothing to cause.
Trust. Credit unions compete on the promise of being on the member's side. A failed rollout breaks that promise in the most visible way possible, and frustrated members talk to local media, post online and move their direct deposits.
Regulatory exposure. Examiners and consumer protection regulators treat a preventable failure as a governance failure, not bad luck. That can mean findings, consent orders, restitution and civil money penalties.
Credit unions are also less able to absorb these hits than large banks. Smaller teams, thinner margins and a membership concentrated in one community or employer group mean a single bad weekend can shape the institution's reputation for years.
Core conversions: the highest-risk event a credit union undertakes
A core conversion moves every account, loan, card, transaction history and interface the credit union has onto a new system, usually over a single weekend. Nearly everything a member touches depends on the core, so nearly anything can break.
The risk isn't only in moving the data correctly. It's in everything connected to it: online and mobile banking, bill pay, card processing, ACH and wires, statements, GL posting, notices, and dozens of third-party integrations. A record can convert perfectly and still fail the member if the downstream system reads it differently.
Case study: VyStar Credit Union
In May 2022, VyStar Credit Union, a Jacksonville, Florida credit union with about 850,000 members, launched a new online and mobile banking platform with a new provider. The platform was pulled offline soon after launch. When it came back, key functions like transfers and loan payments were missing, and some weren't restored for months.
The CFPB's October 2024 consent order found the conversion was rushed to meet an unrealistic deadline despite warning signs from VyStar's own team. According to American Banker, the head of quality assurance refused to sign off on the launch, and it went ahead anyway. The ABA Banking Journal summarized the CFPB's allegations: call volume rose about 250%, with 31% to 44% of calls dropped for months.
The result was a $1.5 million civil penalty, a required governance committee for consumer-facing system projects, and refunds of fees members incurred. VyStar neither admitted nor denied the findings. The CFPB's central conclusion is the one every credit union board should remember: the harm was foreseeable and preventable.
Case study: TSB Bank (UK)
TSB is a bank, not a credit union, but it is the textbook example of a migration gone wrong. In April 2018, TSB moved customer data to a new platform. The data itself migrated, but the platform then failed, disrupting branch, phone, online and mobile banking for a large share of its 5.2 million customers, with problems lasting until December.
UK regulators fined TSB £48.65 million in 2022, on top of £32.7 million already paid to customers. Regulators cited poor planning, weak governance and inadequate oversight of its critical IT supplier.
The lesson
Both failures share the same root causes: a fixed date that outranked readiness, not enough end-to-end testing, and too much trust placed in a vendor. Neither was caused by a single exotic bug. Both were caused by launching before the institution knew the system worked.
Digital platform switches: where members feel every mistake
If a core conversion is the riskiest project behind the scenes, a digital banking switch is the riskiest one in front of members. A core problem may be caught and fixed by staff before anyone notices. A digital banking problem is discovered by members themselves, on their phones, often at the moment they need to pay a bill or check whether their paycheck arrived.
Digital conversions are also uniquely demanding of members. Unlike a core conversion, which members may barely notice, a new online and mobile platform usually asks every member to take action, and each action is a chance for something to fail.
A typical digital platform switch asks members to:
Re-enroll, often with a new password and sometimes a new username
Download a new app when the old one stops working on a set date
Re-create scheduled transfers, external accounts and alerts that don't migrate
Work around a bill pay freeze in the days before cutover
Reconnect Quicken, QuickBooks or other personal finance tools
Each step is reasonable on its own. Together, they send nearly every active member to enrollment screens, identity checks and support lines in the same few days. When recurring payments fail to migrate, two-factor login stalls under load, or the call center can't keep up, members miss bills and lose access to their money. That is exactly the pattern the VyStar case shows.
What to test before a digital switch
Enrollment at scale. Test re-enrollment for every member type: joint owners, minors, trusts, business sub-users, members without email or a current phone on file. Simulate thousands of enrollments in an hour, not a handful.
Authentication and recovery. Multi-factor setup, passkeys, password resets and locked-account recovery must work on old and new phones, across carriers and for members who rarely log in.
Data that doesn't migrate. Know exactly what members will lose (transfers, payees, alerts, external accounts) and test how they'll rebuild it. Anything a member must recreate is a likely missed payment.
Integrations. Bill pay, person-to-person payments, remote deposit, card controls, statements and personal finance tools like Quicken and QuickBooks each need end-to-end tests with real accounts.
Balances and history. Confirm that every account type displays the right available balance, pending items and history, including loans, certificates and business accounts.
Support capacity. Model the call, chat and branch volume for the first two weeks. Staff and train for it, and give front-line employees a tested script and an escalation path.
Old-to-new handoff. Test the cutover itself: what members see when they open the old app, how they find the new one, and how scam-awareness messaging keeps fraudsters from exploiting the confusion.
A digital platform is the credit union, as far as most members are concerned. Testing it as if every member will use it on day one, because many will, is the only standard that protects them.
Why the stakes are rising: AI agents, fintechs and mergers
Three trends are multiplying the number of launches credit unions make, and each one adds risk that only testing can catch.
AI agents
Credit unions are adding AI to lending decisions, fraud detection, member chat and back-office work, and the next wave is agents that take actions on a member's behalf: moving money, answering account questions, opening disputes. Unlike traditional software, AI can give different answers to the same question and can be confidently wrong.
The NCUA has not written AI-specific rules; its position is that existing regulations are technology-neutral and apply to AI. That means fair lending, UDAAP, privacy and error-resolution rules all apply to what an AI agent says or does. A May 2025 GAO report also found the NCUA's model risk guidance too limited and noted it still can't examine the technology providers credit unions rely on for AI, so the credit union carries that risk itself.
Before an AI tool touches members, test it:
Against hundreds of real member scenarios, including edge cases and adversarial prompts, not a vendor demo
For accuracy on rates, fees, balances and policy answers, checked against the source of truth
For fair lending and bias, with outcomes compared across member groups
For hard limits on what the agent can do without a human, especially moving money
For clean handoff to staff and a full audit trail of every action
Fintech partnerships
Fintech partners let credit unions launch new products quickly, but every integration is another place for money and data to fall out of sync. The 2024 collapse of Synapse, a middleman between fintech apps and banks, locked more than 100,000 customers out of their savings, and a court-appointed trustee found up to $96 million of customer funds missing. At the heart of it were ledgers that didn't reconcile.
Before going live with a fintech, test:
Daily reconciliation between the fintech's records and the credit union's core, to the penny
Every money movement end to end: funding, withdrawals, returns, reversals and disputes
Failure scenarios: what happens if the fintech's API goes down, sends duplicates or fails mid-transaction
Disclosures, Reg E error resolution and complaint handling for members who use the partner's app
An exit plan, including how members' data and money come back if the partner fails
Mergers and acquisitions
Consolidation is accelerating. NCUA approved 319 credit union mergers in 2024 and 2025, and credit unions are also buying banks. Every merger ends in a conversion: the merging institution's members, accounts, loans, cards and digital banking users are moved onto the continuing credit union's systems.
Merger conversions carry extra risk. The data comes from a different core, often with different product structures, fee schedules and account numbering. Members didn't choose the change, so their patience is thin. And the two staffs are learning to work together while the conversion happens.
Before a merger conversion, test:
Product mapping: every legacy product matched to a new one, with rates, fees and terms honored as promised in the merger disclosures
Account and routing number changes, and every direct deposit, ACH debit and card that depends on them
Loan conversions, including payment amounts, due dates, escrow and payoff figures
Digital banking enrollment for members who have never used the continuing credit union's platform
Staff readiness on both sides, so the merged members get answers on day one
The common thread: AI agents, fintechs and mergers all increase how much a credit union depends on systems and partners it didn't build. Testing is how the credit union confirms those systems actually serve its members before members find out the hard way.
New products deserve the same discipline
Conversions get the headlines, but everyday product launches cause their share of member harm too. They're often treated as a configuration task rather than a project, so they get less testing than they need.
The places where untested products most often go wrong:
Rates and dividends. A certificate special set up with the wrong compounding method, rate tier or maturity rule can over- or under-pay every member who opens one. The error compounds quietly until a member or auditor catches it.
Fees. A new overdraft, courtesy pay or service fee configured to trigger in the wrong conditions can create regulatory issues under Regulation E and UDAAP standards, plus refunds.
Loan products. Payment calculations, escrow, grace periods, late fees and payoff quotes all have to match the note and the disclosures exactly. Truth in Lending errors are expensive to remediate.
Disclosures and notices. The system has to produce the right disclosure, with the right figures, at the right time. A product that works but generates inaccurate disclosures is still a compliance failure.
Card programs. Card conversions and new card products touch networks, processors, fraud tools and digital wallets. Vendors often call card programs the hardest part of a conversion, because most credit unions support several networks and card types.
Digital banking features. New features in online or mobile banking need to work across devices, browsers, accessibility tools and every account type, not just the test account a vendor demoed.
The rule of thumb: if a product changes how money is calculated, moved or disclosed, it needs a test plan, documented results and a sign-off before members see it.
Regulators expect it, and they're watching
Testing isn't just good practice; examiners assess it. In August 2024, the FFIEC, whose members include the NCUA and CFPB, issued its Development, Acquisition, and Maintenance booklet. It sets examiner expectations for how institutions plan and execute system development and acquisition, govern the risks, manage change, and oversee third-party providers.
In practice, examiners will want to see:
A documented project plan with realistic timelines and clear decision points
Test plans, test results and evidence that defects were tracked and resolved
Board and management oversight, including who approved go-live and on what basis
Due diligence on vendors, especially new or unproven ones, and the credit union's own validation of their work
Change management controls for products and configuration changes after launch
The VyStar order also shows that consumer protection law applies. The CFPB treated poor planning and vendor oversight as an unfair practice because members were harmed. A credit union can't hand accountability to its vendor: if members are hurt, the credit union answers for it.
What good testing looks like
Strong testing programs at credit unions tend to follow the same progression, with each stage building confidence before the next.
Define success up front. Write down what "working" means before testing starts: balances that must reconcile to the penny, transactions that must post correctly, functions that must be available on day one. These become the go-live criteria.
Map the data and the edge cases. Your staff know your data better than any vendor. Identify the unusual accounts: joint and trust accounts, dormant accounts, accounts with holds or garnishments, loans with modified terms, legacy products no longer offered. These are where conversions break.
Run multiple mock conversions. Convert real (properly secured) production data into the test environment at least two or three times. Each run should be faster and cleaner than the last.
Reconcile everything. Compare share and loan balances, accrued dividends and interest, GL totals, account counts and card counts between the old and new systems. Investigate every variance, however small.
Test end to end. Follow real transactions through every system: a direct deposit posting, a debit card purchase, a bill payment, an ACH return, a loan payment, a month-end statement. Integrations are where hidden problems live.
User acceptance testing by front-line staff. Tellers, call center agents and lenders should run their daily tasks in the new system. They'll find problems a project team won't.
Test the member experience. Log in as members would, on real devices. Test password resets, enrollment, alerts and accessibility. Plan for the support load of the first week, not just the system.
Load and performance testing. Make sure online banking and the call center can handle the spike when every member logs in on Monday morning.
Hold a real go/no-go decision. Review open defects against the criteria from step 1. If the criteria aren't met, the date moves. Give quality assurance a genuine veto, and document who decided and why.
Plan for day one and beyond. Have a rollback plan, staffed command center, fee-waiver and refund policy, and member communications ready before launch, not after something goes wrong.
The same steps scale down for a new product: define the expected results, test the edge cases, reconcile the calculations, check the disclosures, and get a sign-off before launch.
The objections, and why they don't hold up
"We don't have time." The launch date usually feels fixed because of a contract expiration, a marketing campaign or a board commitment. But a delayed launch costs weeks; a failed launch costs months of remediation, plus refunds, penalties and member attrition. VyStar's restricted functionality lasted months.
"Testing is too expensive." Compare the cost of an extra mock conversion or a few more weeks of UAT against VyStar's $1.5 million penalty and the overtime, fee refunds and lost members that came with it. Testing is the cheapest place to find a defect.
"The vendor has done hundreds of these." Vendors know their system; they don't know your data, your products or your members. One core provider's operations director put it plainly in a 2008 Credit Union Journal piece: the conversions with the most problems are the ones where the credit union expects the vendor to do it for them.
"It's just a small product change." Small configuration changes touch calculations and disclosures that apply to every member who uses the product. Small changes deserve small test plans, not no test plan.
Testing is member service
Members will never see a test plan, a reconciliation report or a go/no-go meeting. What they'll see is that their paycheck posted, their card worked and their app opened on Monday morning. That invisible result is what testing buys.
For credit unions, the people affected by a failed launch aren't customers in the abstract; they're member-owners. Protecting them from preventable harm is the whole point of the cooperative model. So build time for testing into every plan, give quality assurance real authority, verify your vendors' work with your own eyes, and be willing to move the date. The launch that slips a month is forgotten quickly. The launch that fails is remembered for years.
Sources
American Banker: CFPB penalizes credit union over digital banking upgrade
Gordon Feinblatt: CFPB faults financial institution for rushed core conversion
Mortgage Solutions: TSB hit with £48m fine over operational failings
Morgan Lewis: UK regulators fine TSB for operational resiliency failures
NCUA: FFIEC Development, Acquisition, and Maintenance booklet
Credit Union Journal (2008): 4 CUs, 3-day weekend, 1 DP provider
GAO: Artificial Intelligence — Use and Oversight in Financial Services (GAO-25-107197)
NBC News: Thousands of Americans see their savings vanish in Synapse fintech crisis
Comments