|
|
 |
| |
|
|
|
TFDNet: A Transformer-Based Multimodal Framework for Fraud Detection in Multilingual Kazakh-Russian-English SMS and Voice Calls |
|
|
|
PP: 1-17 |
|
|
|
Author(s) |
|
|
|
Azamat Serek,
Aisultan Shoiynbek,
Nurshapagat Shapay,
|
|
|
|
Abstract |
|
|
| Fraudulent SMS (smishing) and voice calls (vishing) remain among the most common vectors for social-engineering attacks, exploiting urgency, authority, and trust to extract money or credentials from victims. Existing detection systems typically treat text and speech as separate problems and are almost universally developed and evaluated for monolingual English traffic, leaving code-switched, lower-resource settings under-served. We target Kazakhstan as a concrete case: Kazakh holds constitutional status as the state language, Russian carries official status alongside it in state institutions, real-world financial and telecom communication routinely mixes Kazakh and Russian text with English financial and technical loanwords, and Kazakhstan’s Financial Monitoring Agency has documented active SMS-blaster and impersonation fraud campaigns against Halyk Bank and Beeline customers as recently as 2026, alongside vishing schemes impersonating the National Bank and law enforcement. In this paper we propose and train TFDNet (Transformer-based Fraud Detection Network), a unified multimodal architecture for fraud detection across SMS and voice-call channels, comprising a text encoder, a paralinguistic-feature encoder, a modality-gating mechanism for missing modalities, and a cross-attention fusion module that models the interaction between linguistic content and paralinguistic cues such as prosody, pacing, and hesitation, together with a training objective combining weighted cross-entropy and a supervised contrastive term. We construct two synthetic, incident-inspired Kazakh-Russian corpora – an SMS fraud corpus of 4,903 labeled messages and a voice/vishing corpus of 2,817 labeled call transcripts with paralinguistic descriptors, evaluated separately rather than as paired multimodal samples – with the Kazakh and Russian wording translated and reviewed by professional translators based in Almaty. Because pretrained transformer and speech-model checkpoints were not accessible to us for this study, TFDNet’s text and fusion modules are trained end-to-end from randomly initialized weights rather than fine-tuned from a pretrained checkpoint; we report this trained-from-scratch model’s results directly, alongside classical TF-IDF baselines for comparison, rather than substituting a different model family for TFDNet. TFDNet’s fusion model reaches F1 = 0.974 (AUC = 0.997) on the voice corpus, ahead of every unimodal and naive-fusion baseline, at 2.85 ms mean CPU model-inference latency per call (excluding paralinguistic feature extraction from audio). On the SMS corpus, TFDNet’s text branch reaches F1 = 0.962 (AUC = 0.960) at 1.27 ms mean latency, within a point of the strongest classical TF-IDF baseline (F1 = 0.967); their bootstrap confidence intervals overlap substantially, so this difference should not be read as statistically decisive, and we discuss why a transformer trained from scratch, without pretrained initialization, does not clearly exceed a well-tuned linear bag-of-words baseline on a few thousand training examples. We discuss architectural choices, ablations, an explainability analysis of the classical text baseline, and, integrated into the discussion, the study’s limitations and the ethical considerations of processing voice-biometric signals for fraud detection. |
|
|
|
|
|
|
|
