Industry: Financial Services / Data Management
Service Provided By: Virstack – Custom API Development & Cloud Solutions
Project Duration: ~3 Months
- Backend: Node.js
- Database: MongoDB Atlas
- Cloud Hosting: AWS ECS with Auto-Scaling
A financial data provider approached Virstack with a pressing challenge: their capital gains data—critical for their software partners—was stored and maintained manually in massive Excel spreadsheets. As the demand for this data grew, their existing approach became unmanageable, slow, and error-prone.
They needed a scalable, secure, and standardized method for delivering capital gains data to external software platforms in real time. This required transforming static Excel data into a robust, cloud-hosted REST API—capable of handling thousands of requests, supporting various query types, and ensuring seamless updates with zero downtime.
Key client concerns included:
- Delivering programmatic access to complex, frequently updated datasets.
- Supporting multiple query options: single ticker, multiple tickers, and bulk requests.
- Maintaining data integrity, uptime, and partner-level access control.
- Seamless synchronization with the source Excel files, without service interruptions.
- Logging and tracking API usage for compliance and partner accountability.
Our team of DevOps, backend, and cloud engineers designed a custom RESTful API solution, hosted on AWS ECS, to deliver high availability and auto-scaling. The project was executed with a focus on performance, security, and partner-centric access control.
We developed a REST API using Node.js, capable of delivering capital gains data in standardized JSON format. The API was designed to support:
- Single Ticker Symbol Queries
- Multiple Ticker Symbol Queries
- Bulk Queries for all tickers associated with a verified company
- The endpoints were optimized with pagination to efficiently manage large data volumes and response times.
To ensure secure access and transparency:
- A unique API key system was implemented for each partner.
- Access roles (e.g., read-only) were enforced via middleware to ensure granular control.
- All API usage was logged and tagged per partner, allowing the client to monitor and audit consumption levels.
Recognizing the client's dependency on Excel:
- We developed an automated parser that converts Excel data into structured MongoDB documents.
- The system supports full database replacement, allowing updates without disrupting live service availability.
- Backups and integrity checks were added to ensure update consistency.
To meet performance goals:
- The API was deployed on AWS ECS (Elastic Container Service) with auto-scaling policies for peak traffic.
- We achieved 99.9% uptime, with support for concurrent queries from multiple software partners.
We provided partners with detailed API documentation:
- Endpoint descriptions
- Query parameters
- Response structures
- Example success and error responses