This year, at the end of August, we launched express delivery of products from Magnit through the Delivery Club service.
Since then, the retailer has managed to integrate with Yandex. Eda launched its mobile application, the number of connected stores has increased from ten to 800, and the system at its peak processes more than 20,000 requests per second.
Nikolay Karlov, head of the architecture department at VK Digital Technologies, tells how to launch such a system when it’s only a month from setting a task to starting.
Tasks And Difficulties At The Start Of The Project
In July, Magnit initiated the launch of a new sales channel – express delivery of products. The first in line was the Delivery Club service: the partners agreed that DC couriers would start delivering products from the first Magnets by autumn. Next, the company planned to connect to Yandex.Ede and launch its delivery service. The “Magnit” chain includes tens of thousands of stores throughout Russia.
At the same time, part of the data – about the assortment of a particular store, balances in warehouses, current prices – is stored within themselves and, for example, a catalog of goods, which contains more than a million items, is located in a central database.
Problem # 1. Short Time
We had only a month to implement the project: from the formation of requirements to the launch. For comparison: usually, preparing such projects with companies similar in scale to Magnit takes up to six months, that is, six times more time than we had.
Problem # 2. Relevance Of Information
Information about the “Magnet” system balance is updated every few hours. This is more than enough for an offline business; however, it is necessary for the exchange of data to occur once every five minutes for organizing distance selling. Otherwise, there is a high probability that the product will be on the online storefront, and the user will put it in the basket, but it will not be in a particular store. These are reputational risks for the delivery service.
Problem Number 3. Offline Orientation
In the process, it turned out that the Magnit system lacked information about products and categories, which was needed for the system to work correctly. There was no quantization of goods. Quant is the minimum portion of a specific product that can be bought in a given store (for example, for watermelons, quantum is 3 kg, for meat – 400 g). Without this information, it was impossible to organize trade through online services.
How It Built A Data Management System To Connect An Unlimited Number Of Stores
Due to the tight deadlines, we did some stages of the work in parallel. Concurrently, we worked through the data models and project specifications with the initial assessment. Information about individual products needed to be enriched or modified, for which we developed transformation rules within Tarantool.
We had to build a system that would receive data streams from stores, from the central database of “Magnet,” combine them, process them – and give out up-to-date information on balances, prices, etc. It was supposed to become a link between “Magnet” “And express delivery services information systems.
The individual business processes required to launch the project took longer than we had for the entire implementation. Deploying traditional infrastructure is a lengthy process. You can, for example, order servers – and wait for their delivery up to nine months, although everything is ready. We couldn’t afford such a luxury, so we decided to deploy the system in VK Cloud Solutions
To do this, we had to resolve the safe and correct data transfer from Magnit to the cloud via a VPN tunnel.
First, we created an imitation of Magnit’s IT infrastructure and implemented a semblance of a system on the Tarantool platform to test the solution. Then, when the VPN tunnel was set up, we conducted tests on actual data: colleagues from Delivery Club tried to place orders. At the same time, we loaded product catalogs manually.
The first couriers and shops were connected when some things were not yet fully automated, due to which we were able to get the first feedback on the system’s operation and make adjustments in time. With such a time frame, it was essential to conduct development parallel with the analysis of implementation results.
We launched the finished system based on the Tarantool Data Grid. Setting up and deploying the platform are resolved “out of the box,” significantly affecting the launch speed. Tarantool is well suited for collecting and processing data in real-time and provides high performance. It also allows you to scale the system quickly. When new stores are connected, data begins to flow; the system does not need any modifications.
From Delivery Club To Yandex.Food And Other Channels
From the system’s point of view, there is no need for revision or modification; delivery services connect and start reading data from Tarantool. The system processes up to 20,000 requests per second. If the points become not a thousand but tens of thousands, we will only need to find out about this in advance and connect additional capacities.
For example, integration with Yandex, Eda turned almost seamless. We focused on the specification that Delivery Club developed. Yandex. Eda had specific content requirements. For example, product photos require a white background. It turned out that some products, which are only in regional stores, are missing images. However, nothing was needed to be configured and developed to integrate with Tarantool.
Also Read: Common Queries All You Need To Know About IT Workforce
