Imagine an NBFC at 4:57 on a Friday afternoon.

A finance employee receives a video call that appears to come from a senior executive. The voice is familiar. The request sounds reasonable: a supplier payment needs urgent approval. An email arrives with supporting documents. An internal AI assistant reads the attachment and produces a clean, confident summary.

Everything looks ready to proceed.

But the executive on the call is synthetic. The supplier account has been changed. The attachment also contains instructions intended to manipulate the assistant reviewing it.

In this hypothetical incident, an attacker has used AI to impersonate a person and targeted AI inside the organisation. A rushed employee becomes the connection between those two attacks.

For a bank or non-banking financial company, this is the security problem worth understanding: AI can influence who we trust, what we believe, and which actions our systems perform.

Financial institutions can also use AI to investigate suspicious behaviour, find weaknesses and help employees make better decisions. Capturing that value requires controls over the entire workflow, including its data, permissions, human approvals and recovery arrangements.

The practical question is how to give AI useful work while keeping responsibility and authority in the right places.

The scenarios in this article are illustrative. The implementation patterns and rollout plan are proposed approaches, supported by the linked guidance rather than presented as measured results.

Understand the three fronts of AI security

AI security covers three related areas:

A fourth concern runs through all three: ordinary failure. An inaccurate explanation, biased prediction or unavailable service can harm a customer without an attacker being involved.

The technology also matters. A machine-learning model might estimate transaction risk from patterns in historical data. A large language model, or LLM, might summarise a case or draft a response. An AI agent combines a model with tools and a workflow so it can take actions, such as opening a ticket or requesting a change.

These systems need different controls. A model that writes a paragraph and an agent that can release a payment should never receive the same risk assessment merely because both use AI.

This distinction is reflected in the separate resources OWASP publishes for LLM applications and agentic applications. Use those resources to structure review, then test the actual financial workflow.

Give AI work that improves the evidence

Financial institutions have used AI in cybersecurity and fraud operations for years. The US Treasury’s 2024 sector report discusses those uses and the importance of managing the associated data, model and third-party risks. That experience provides a useful starting point for selecting new applications. US Treasury report

The following are practical candidates to evaluate, with a clear boundary around each one:

Consider a security operations centre investigating an unusual login, a mailbox rule change and a large document export. An assistant could assemble these events into a timeline and identify questions for the analyst. Its value is easier to assess when each statement links to a real event. An elegant narrative without traceable evidence can send an investigation in the wrong direction.

Fraud detection offers a different example. A graph-based model could identify connections among applications that reuse devices, addresses or beneficiary accounts. Those connections are investigation leads. A shared address might also belong to a family or shared accommodation. The review process must account for that ambiguity before a customer loses access to a service.

For penetration testers and application-security teams, AI can help generate test ideas, understand unfamiliar code and explain remediation. It still needs executable evidence. An AI statement that an API is vulnerable does not prove exploitation; an AI statement that a scanner finding is a false positive does not justify closing it.

Start with work where a reviewer can inspect the inputs, challenge the conclusion and correct the result. Expand authority only after measuring performance on the organisation’s own cases.

Prepare for attackers who can sound like anyone

Deepfake fraud is a documented financial-sector concern. In November 2024, FinCEN reported increased suspicious-activity reporting involving suspected deepfake media, including fraudulent identity documents used to circumvent verification and authentication. FinCEN alert announcement

The practical implication is that a recognisable face or voice should not, by itself, authorise a sensitive action.

In the opening scenario, a better payment process would send the employee to an independently authenticated approval workflow. It would show the amount, beneficiary and any recent account change. Verification would use an established contact route, obtained independently of the suspicious message. A changed beneficiary would trigger the organisation’s defined verification process.

Where separation of duties applies, the person requesting the payment and the person approving it should remain distinct. An AI assistant must not collapse those roles for convenience.

Deepfake detection can supply another signal, but the workflow must remain safe when that signal is wrong. A low detector score is insufficient evidence to release funds.

For digital onboarding, combine document checks, appropriately evaluated liveness measures, device information and independent validation. Test the whole journey, including account recovery and exception handling. A strong onboarding control loses value if a persuasive caller can later persuade support to reset access through a weaker route.

Training should rehearse these decisions. Employees need to practise declining an urgent request that bypasses the established process, even when the apparent sender is senior. Customers need a simple, known way to verify suspicious requests through the institution’s official channels.

Keep document content from becoming an instruction

Prompt injection occurs when supplied content redirects a model’s behaviour. It can arrive directly through a prompt or indirectly through a document, email, webpage or other material the model processes. Retrieval-augmented generation and fine-tuning do not eliminate this risk. OWASP guidance on prompt injection

Imagine a loan-review assistant reading an applicant’s uploaded document. Alongside financial information, the file contains a direction to disregard discrepancies and recommend approval. The document is evidence supplied by an applicant; it has no authority to change the institution’s review policy.

The dangerous step occurs if the application allows those two categories to blur.

Design the workflow so that document contents cannot grant permissions, change approval rules or authorise tools. Label external content clearly, test instruction-following failures and minimise the data and tools available during document analysis. Treat detection and filtering as additional protection whose effectiveness must be measured.

Enforce consequential decisions in application controls outside the model.

If an assistant proposes a tool call, the receiving service should independently check the caller, customer, action and permitted parameters. A convincing explanation does not create permission.

Generated output also needs normal software-security treatment. Encode it before displaying it in a browser, validate structured fields and use parameterised database operations. Do not execute model-generated commands merely because they arrived from an internal assistant. OWASP identifies insufficient validation and handling of model output as a distinct application risk. OWASP guidance on output handling

Control what enters the model and what leaves it

A customer-data leak can start with an employee trying to finish a legitimate task. They paste a statement, identity document, source-code fragment or incident log into an unapproved assistant because it is convenient.

Provide an approved route that is useful enough for employees to adopt. Specify permitted data categories and configure identity, access, retention and data-loss-prevention controls around that route. Where identifiers are unnecessary, remove or replace them before submission. Keep any re-identification mapping separately protected.

Check provider settings and contractual terms for training use, retention, human access, subprocessors and deletion. A promise that submitted data will not train a model does not by itself establish that the data is never stored or accessible. OWASP’s sensitive-information guidance covers exposure through both models and application context. OWASP guidance on sensitive information

Internal deployments need the same care. Hosting a model inside the organisation does not automatically fix excessive privileges, exposed APIs or insecure logs.

Retrieval-augmented generation, or RAG, adds another boundary. It retrieves relevant material and supplies it to the model. A bank might use it to search policies and case records. Relevance, however, does not establish a user’s right to see a record.

Apply authorisation before restricted content reaches the model.

Carry customer and tenant permissions through retrieval, caches, conversation history and exports. Recheck access when permissions change. An output filter cannot undo disclosure of a document that was already sent to an unauthorised processing context. OWASP explicitly discusses access partitioning and leakage risks in vector and embedding systems. OWASP guidance on retrieval security

Embeddings are numerical representations used for tasks such as semantic search. Treat them as potentially sensitive derived data rather than assuming the conversion anonymises the original material.

Map the complete data journey, including optical character recognition services, prompt traces, feedback records, evaluation datasets and backups. Each additional copy creates another place where access and retention must work correctly.

Protect what AI learns and what it treats as evidence

Traditional machine-learning systems can face evasion, poisoned data and privacy attacks. NIST’s adversarial-machine-learning taxonomy provides a broader vocabulary for these risks than a chatbot-only assessment. NIST adversarial machine learning report

For a lender, this suggests testing whether manipulated input features can distort a risk score and whether unreliable labels can contaminate future training. Review the provenance of training data and protect the processes used to correct labels. Keep an independently controlled evaluation set, and investigate material changes in performance before deployment.

An assistant can also be misled without retraining its model. Suppose an obsolete collection policy remains highly ranked in a search index. The model may accurately summarise a document that should no longer govern the case. Publishing access, document status, version history and effective dates therefore belong in the security design.

Treat supplier models, software libraries, model adapters and deployment images as a supply chain. Maintain an inventory, verify origins and integrity, review dependencies and assess changes before release. Integrity checks can establish that an artifact matches an approved version; they do not prove that the version behaves safely. OWASP guidance on AI supply-chain risk

Ordinary model error needs separate attention. A language model can invent a policy clause or produce an unsupported explanation with fluent wording. NIST’s generative-AI profile discusses confabulation and related information-integrity risks. NIST generative AI profile

For customer-facing amounts, use authoritative account systems and tested calculations. For lending explanations, start from the actual decision record and validated reason codes. Asking a model to explain a rejection after the event can produce a plausible story that does not describe why the decision occurred.

Give agents precise and limited authority

An agent that can call tools needs an identity and permission model as carefully designed as any other service account. OWASP’s agentic-security work addresses the risks that arise when models can plan and act across workflows. OWASP guidance for agentic applications

A practical policy should specify which agent may perform which action, for which customer or case, under which limits. Reading a balance, drafting a refund request and executing a refund are separate permissions.

Use narrowly scoped, short-lived credentials where supported. Restrict tools and outbound destinations. Keep secrets in controlled services instead of embedding them in prompts. Apply the same assessment to tools exposed through the Model Context Protocol, or MCP; a standard connection format does not establish that a connected tool is trustworthy.

For a consequential action, approval should cover the exact operation. The reviewer needs the beneficiary, amount, customer, destination and important changes. The backend should reject execution if those details change after approval. A generic permission to continue a conversation is inadequate for authorising a financial transaction.

Make retries safe. If an agent repeats a refund request after a timeout, the backend should recognise the duplicate rather than issue another refund. This requires transaction controls such as idempotency, independent of the model’s intentions.

Limit runtime, tool calls, retries and expenditure. A looping agent can consume resources and disrupt service even without malicious intent. OWASP describes excessive resource use as an availability and cost risk. OWASP guidance on unbounded consumption

Human review also needs design. Give reviewers underlying evidence, time to inspect it and a usable rejection path. Audit a sample of accepted recommendations to find cases where fluent explanations are encouraging automatic approval.

Build a loan assistant around verifiable steps

Consider an NBFC introducing an assistant for application review. The proposed goal is to help staff understand documents and identify inconsistencies. Success would mean less review effort with acceptable accuracy, privacy and customer outcomes.

The workflow can be structured around six stages:

  1. Intake: Receive documents for a specific application. Require an authenticated session, appropriate file checks and verified case ownership.
  2. Extraction: Read relevant fields from the submitted documents. Preserve source references and route uncertain values for review.
  3. Verification: Compare claims with permitted authoritative sources. Record completed checks and unresolved discrepancies.
  4. Assistance: Produce a concise case summary. Clearly distinguish verified facts, applicant claims and model inferences.
  5. Decision: Apply the approved credit process. Use validated rules or models and obtain the required approvals.
  6. Execution: Perform approved downstream actions. Enforce independent authorisation and prevent duplicate execution.

The summary should make uncertainty visible. A statement that an applicant reported a particular income is materially different from a statement that the income was independently verified.

Preserve that distinction in the interface and audit record. A useful summary might identify two inconsistent salary entries, link to their source pages and request review. It should not silently choose one value and present it as settled fact.

Test local operating conditions as well. An Indian deployment may encounter Hindi, Marathi, mixed-language messages, transliteration, poor scans and regional document formats. Evaluate the actual languages and document types the institution expects to receive.

Apply the same discipline to collections and customer service. An assistant should retrieve current case facts and approved policy before drafting a response. Any proposed fee waiver, settlement or account change should go through the institution’s controlled process.

This design creates a measurable pilot. Reviewers can compare source accuracy, missed inconsistencies, correction effort and handling time before expanding the assistant’s role.

Make governance specific to the institution

Assign a business owner to each AI use case. Record its purpose, affected customers, data sources, model provider, integrations, permissions, evaluation results and fallback arrangements. Include AI embedded in purchased products as well as systems built internally.

Risk classification should consider what happens if the system is wrong. A public-information assistant, an internal case-summary tool and a credit-decision model expose the institution to different consequences.

NIST’s AI Risk Management Framework provides a voluntary structure for organising AI risk management across the lifecycle. It can support an institution’s governance process, but adopting it is not a substitute for meeting applicable obligations. NIST AI Risk Management Framework

For Indian banks and NBFCs, compliance should map each use case to the requirements that apply to that entity and activity. Review relevant RBI directions, privacy and confidentiality obligations, outsourcing arrangements, customer-treatment requirements and incident-reporting duties. Verify applicability and effective dates against current official texts. A committee recommendation, industry framework and binding direction have different status.

A partner-heavy lending workflow needs clear ownership at each handoff. Establish which party collects data, which service processes it, who makes the decision and who responds to a customer complaint. Confirm that partner permissions and subcontractor access match those responsibilities.

Security, fraud, model risk, privacy, operations and compliance teams should review high-impact use cases together. An internal-audit team should be able to inspect the resulting evidence independently.

Evaluate customer outcomes as part of that evidence. Check whether error and escalation rates differ across relevant languages, products and customer segments, using lawful and appropriate evaluation methods. Removing an explicit sensitive attribute does not establish that other inputs cannot act as proxies.

Customers affected by a consequential error need a usable way to challenge it. Record who can review the case and correct the outcome.

Ask vendors questions that expose the real deployment

A vendor demonstration shows what a product can do under selected conditions. Due diligence should establish what happens with the institution’s data and permissions.

Ask where content is processed and retained, whether it can be used for training, who can access it and which subprocessors participate. Determine whether model updates can be controlled, how changes are communicated and what happens when a version is retired.

Request evidence of isolation testing and relevant security evaluations. Agree on incident-notification arrangements, deletion procedures and support for investigation. Test the ability to export necessary records and continue essential operations if the service becomes unavailable.

Consider correlated dependencies. Two applications may rely on the same underlying model or cloud service despite having different vendors. The Financial Stability Board identifies third-party dependence and provider concentration among AI-related financial-sector vulnerabilities. FSB report on AI and financial stability

For a smaller NBFC, procurement should include the staff effort required to validate, monitor and recover the service. A lower licence price can be misleading if the organisation cannot operate the controls the product requires.

Test the harm that the workflow must prevent

AI security testing should combine ordinary application and infrastructure testing with model- and workflow-specific scenarios. The backend still needs protection against access-control failures, injection flaws, exposed secrets and vulnerable components.

Add tests that exercise the institution’s actual trust boundaries. The examples below are proposed tests, not a complete certification standard:

Run adversarial tests alongside ordinary customer requests. A system that refuses everything might block many attacks while failing its business purpose. Repeat representative tests because a single successful response does not establish consistent behaviour.

Record the tested model version, configuration, retrieval data, user role and tool permissions. Keep failed examples for regression testing. Reassess after changes to the model, prompt, knowledge base or integration, since any of these can alter behaviour.

Treat generated security findings with the same care as other outputs. A reviewer should be able to reproduce the issue and identify the control that failed.

Prepare to stop the workflow and recover the customer

Before deployment, define who can disable an agent, revoke its credentials, pause a connector or quarantine a knowledge source. Rehearse those actions rather than assuming an emergency switch will work.

An AI incident may require evidence beyond conventional server logs. Capture the relevant request, document or retrieved source, model and configuration version, tool calls, approval records and resulting business actions. Protect that evidence with appropriate access and retention controls; indiscriminate prompt logging can create another repository of customer data.

Containment should lead to business recovery. If an assistant disclosed records, determine the affected scope. If it changed accounts or influenced decisions, reconcile those actions and identify customers who need correction or assistance. Follow the organisation’s applicable notification and reporting process.

Define degraded operation in advance. Essential banking or lending services need a tested path when AI is unavailable. The fallback might involve a smaller queue, deterministic checks or manual processing, with clear capacity limits and ownership.

Measure value and harm together. Track investigation time alongside missed incidents; fraud detection alongside false declines; answer usefulness alongside material factual errors. Also measure unauthorised tool actions, cross-customer disclosure in testing, reviewer corrections and recovery time during exercises.

Accuracy alone can mislead.

In a hypothetical set of 10,000 transactions containing 100 fraudulent transactions, a system that labels every transaction legitimate is 99 percent accurate and detects none of the fraud. Report metrics that reflect the business problem, including precision, recall, false-positive burden and the cost of mistakes.

Treat a model’s verbal confidence as an output to evaluate. It is not a calibrated probability merely because the model expresses it as a percentage.

Start with a practical first 90 days

Use the following as a planning sequence to adapt to the institution’s size and dependencies, not a guarantee of production readiness.

Days 1 to 30

Inventory existing AI use, including purchased features and employee tools. Assign owners, map sensitive data flows and choose one bounded pilot. Capture a baseline for accuracy, effort and customer impact before introducing the new workflow.

Days 31 to 60

Implement access controls, source traceability, evaluation cases and monitoring. Test the pilot in shadow mode, where recommendations can be compared with the established process without independently changing outcomes. Exercise the failure and escalation paths.

Days 61 to 90

Review evidence against predefined acceptance criteria. If the criteria are met, release to a limited group with rollback capability and close monitoring. Broader access and additional actions should depend on the evidence, with accountable approval for any accepted limitations.

A sensible first deployment might help analysts assemble evidence or help staff find approved policy. It should produce enough operational learning to inform the next decision.

Return to the employee at 4:57 on Friday.

In a well-designed workflow, the call creates a request. The attachment supplies information. The assistant contributes a summary. The payment system independently checks permissions and beneficiary details, and the required approver sees exactly what is being authorised.

The attempted deception may still be persuasive. The institution has arranged its controls so that persuasion alone cannot release the payment.

That is a practical ambition for AI security in banking and NBFCs: give AI useful work, make its contribution verifiable, and keep authority accountable at every consequential step.