Architecture

System Overview

Turno LMS integrates Prefect 3 (orchestration), Quart (async API), RQ/Redis (job queue), and dbt (data transformation) to handle:

• S3 lender file ingestion with YAML-based column mapping
• Fineract LMS operations (client/loan lifecycle)
• Payment webhook processing (Razorpay/Cashfree)
• Automated and manual EMI repayment posting
• Transaction and loan-level reconciliation
• Payment link generation
• Loan settlement and write-off

Tech Stack

Component	Technology
Web Server	Quart 0.20 (async Python, port 5002), gunicorn in production
Job Queue	RQ + Redis
Orchestration	Prefect 3 (port 4201)
Database	PostgreSQL 16 (SQLAlchemy 2, asyncpg)
Data Transform	dbt-core 1.11 + dbt-postgres
Cloud Storage	AWS S3 (bucket: turnofile)
Payment Gateways	Razorpay SDK, Cashfree
Auth	JWT (PyJWT + bcrypt), X-Internal-Token for API
Frontend	Vue 3 + Vite (port 3000), deployed at turnoops.incentius.net
Fineract	Apache Fineract (Docker, port 8443) with local PostgreSQL
Deployment	PM2 (3 processes), Nginx + SSL, Docker

Infrastructure

Domains

Service	URL
Ops UI	https://turnoops.incentius.net
Prefect Dashboard	https://turnoprefect.incentius.net
Fineract	https://turnomifos.incentius.net

Data Flow

Payment File Processing

Webhook Processing (Razorpay / Cashfree)

Fineract Operate Pipeline

Database Schemas

RAW Schema

Raw data as-is from sources. Every payment file lender has a RAW table with the standardized 10-column schema:

loan_id, amount, date, status, charge, note, payment_id, sub_reference_id, source_file, lender

Additional RAW tables:

• RAW_*_RAZORPAY_API / RAW_*_CASHFREE_API - webhook payloads
• RAW_*_LOS - Fineract operation payloads
• RAW_FINERACT_API_LOG - audit trail for all Fineract calls
• RAW_S3_FILE_PROCESSING_LOG - file processing tracker
• RAW_UMRN_MASTER - UMRN-to-loan mapping
• RAW_REJECTED_RECORDS - rows rejected during file processing

BASE Schema

Cleaned and typed data. dbt incremental models that:

• Type-cast columns (numeric, timestamp)
• Parse loan IDs (split_part for Link ID / order_receipt)
• Join against UMRN Master (GB, UGRO)

CORE Schema

Business logic layer:

• CORE_TRANSACTION_DATA - unified view of all payment sources (webhooks + files), deduplicated by source_key, with reconciliation status
• CORE_TRANSACTION_DATA_PROCESSED - tracks which payments have been posted to Fineract
• CORE_LOAN_MASTER - one row per loan with lifecycle info
• CORE_FINERACT_PENDING_TRANSACTIONS - pending Fineract API calls
• CORE_FINERACT_TRANSACTION_ARCHIVE - completed Fineract API calls
• CORE_*_LOS_PROCESSED - LOS operation results (7 tables)

YAML Lender Mapping

The core innovation for file processing. Each lender has a YAML config (config/lender_mappings/*.yml) that defines:

• Column name mapping (CSV header -> standardized field)
• Type casting with format strings
• Row-level filters (status, amount)
• Post-compute operations (column arithmetic)

See YAML Mapping System for details.

Deduplication

Source Key

Every transaction gets a deterministic source_key = md5(lender + loan_id + date + amount). This ensures the same payment from different sources (webhook + file) is recognized as one.

Anti-Join Pattern

CORE_TRANSACTION_DATA LEFT JOIN CORE_TRANSACTION_DATA_PROCESSED — only unprocessed rows are picked up for Fineract posting.

Recon Match Key

For reconciliation, a separate recon_match_key = md5(loan_id + date + amount) (without lender prefix) matches NACH file records against webhook records.

Key Files

File	Role
quart_app/__init__.py	App factory; registers 11 blueprints
quart_app/blueprints/webhooks.py	Razorpay & Cashfree webhook endpoints
quart_app/blueprints/fineract.py	Fineract operate + raw endpoints
quart_app/blueprints/s3_upload.py	File upload, processing log, rejected records
quart_app/blueprints/reconciliation.py	Transaction + loan level recon
quart_app/blueprints/umrn_master.py	UMRN master upload + list
quart_app/blueprints/payment_links.py	Razorpay payment link generation
quart_app/blueprints/settlement.py	Loan settlement & write-off
config/lender_mappings/*.yml	YAML column mapping per lender
utils/lender_mapping_loader.py	YAML loading + column transform logic
utils/csv_processor.py	CSV -> RAW with mapping, filters, UMRN lookup
tasks/flows/s3_processing.py	S3 lender config registry + Prefect flow
tasks/flows/webhook_repayment_pipeline.py	30-min scheduled repayment pipeline
tasks/flows/reconciliation_pipeline.py	Recon-triggered Fineract posting
tasks/flows/fineract_operate_pipeline.py	LOS operation processing
lender_loan_products.py	Lender -> Fineract product ID mapping
dbt/models/core/CORE_TRANSACTION_DATA.sql	Unified transaction model with recon