How to deploy AI in a bank without sending data to the cloud

Over the last two years almost every bank and large financial organisation in Kazakhstan has piloted generative AI. Most pilots stopped at the same point: the security team asked where the data goes, and the honest answer was “to a foreign provider's servers”. That ended the project. Here is how to build AI so the question never comes up.

Why cloud AI is a problem specifically for banks

For a retailer, sending text to a cloud model is a policy decision. For a bank it is a matter of law and licence. Three constraints apply at once:

• Banking secrecy and customers' personal data: processing outside the perimeter needs legal grounds and consents the cloud scenario usually lacks.
• The Unified Requirements for ICT and information security (Government Decree No. 832): information assets of financial organisations must meet requirements on hosting, access, logging and cryptographic protection. An external LLM API does not fit that model.
• The Law on Artificial Intelligence (No. 230-VIII): the owner and the user of an AI system carry duties on risk classification, user disclosure and control — and you cannot control someone else's cloud model.

So for a bank the question “which model” is secondary. The primary question is “where does it run and who controls it”.

The principle: bring the AI to the data, not the data to the AI

On-premise architecture inverts the usual scheme. Instead of sending fragments of documents and transactions outside, the language model is deployed on servers inside the bank's data centre, in a closed network segment. Data never leaves the perimeter at any step: not for training, not for inference, not for logging.

In practice this is four layers:

1. Infrastructure: one or more NVIDIA GPU servers sized for the load. A legal-department assistant for 50 users fits on one server; a call centre handling tens of thousands of dialogues a day needs a cluster.
2. Models: own LLMs fine-tuned for the domain and language. For Kazakhstan the critical part is high-quality Kazakh and Russian with local terminology (1C, reporting forms, regulatory vocabulary).
3. Connectors: read-only access to 1C, the core banking system, DWH, document management and email, with personal data masked before text reaches the model.
4. Control plane: role-based access through the corporate directory, MFA for administrators, an immutable log of every request and answer, export to SIEM.

What actually gets automated in year one

Ten deployments show that value appears not in “chatting with AI” but in three concrete processes:

• Financial analysis: consolidation of reporting from several 1C and core-banking databases, plan-vs-fact with explained deviations, early signals on transaction anomalies. This frees weeks of finance-team work every month.
• Legal work: first-pass contract review against the bank's playbook, search across internal regulations and legislation, drafting of standard documents.
• Customer service: an assistant on WhatsApp and the website trained on tariffs and regulations, with hand-off to an operator and a full dialogue log.

How deployment runs

A typical project takes eight to sixteen weeks. The first two weeks are discovery: use cases, data landscape, security requirements, hardware sizing. Then a pilot on real data with measurable metrics. After the pilot — integration, SSO and audit configuration, conformity documentation and acceptance testing with the security team.

Checklist for the first conversation with a vendor

• Where does the model physically run, and does the vendor retain access after deployment?
• Are external LLM APIs used for any part of the requests?
• How is personal data masked before processing?
• Is there a log of every request and answer, and in what format is it exported?
• Which conformity documents are handed to the security team before acceptance?
• Does the model support Kazakh at the level of official documents?

If a vendor answers any of these evasively, the project will most likely stop at the same point as the previous pilots.

This article is general information, not legal advice. Check the current edition of each act before relying on it.