Myth or Fact? 4,000 transactions per second on the private Catapult blockchain
Original text(Japanese): https://qiita.com/planethouki/private/9733aa83096a988ee57a
First appeared on NEMjapan.jp
4,000 transactions per second
I am frequently asked if 4,000 transactions per second will be capable on Catapult. I usually answer these questions by telling them “only on private, not on public”. That being said, I have not personally tried to achieve this result so I cannot say I am sure…yet. In this article I explain my journey of trying this out on the open sourced Catapult on a private chain.
Some Background
To achieve 4,000 transactions per second, you would need block intervals of 15 seconds and 60,000 transactions per block. If the interval is set to 1 minute, you would need about 240,000 transactions per block. FYI, NEM v.1 blocks can hold 120 transactions per block.
The Process
When attempting to achieve 4,000 transactions per second, I’ve kept the settings as close to the default configurations to keep this process repeatable.
I ran a Azure VM machine, using size F32s_v2 with the below environment.
- ubuntu 18.04 LTS
- git 2.17.1
- docker 18.06.1
- docker-compose 1.22.0
- nodejs v10.11
In my VM I have ports 22, 3000 and 8000 open while running.
- 22
- 3000
- 8000
Launch Catapult
git clone https://github.com/tech-bureau/catapult-service-bootstrap
cd catapult-service-bootstrap
git checkout 77e6cf38a7845194aa2ce72f4ed4d87e5ab791e3
docker-compose -f docker-compose-with-explorer.yml up -d
Done with Alpaca running.
It seems that processing signatures makes the transactions heavier, which will probably inhibit us from reaching optimal performance. So instead, I created a mass amount of pre-signed transactions.
git clone https://github.com/planethouki/yonsen.git
cd yonsen
npm install
I obtained the nemesis address private key since we need accounts with leftover balances. This was done by opening “addresses.yaml” and obtaining all “private” numbers under “nemesis_addresses:”, then creating file “nemesiskeys.json”.
cat ../build/generated-addresses/addresses.yaml
vi nemesiskeys.json
nemesis_addresses:
- private: 1947340C6102E18927B98D49FB4A7947AB0C5AFCDB31F34F7EA85209A2252CF2
public: <something>
address: <something>
- private: 1627E64F23841C748DB9B1650ADFEC8868003158D7803A0652FE76FBD2D617B0
<and more>
Copy and paste the above private keys into the below.
[
"1947340C6102E18927B98D49FB4A7947AB0C5AFCDB31F34F7EA85209A2252CF2",
"1627E64F23841C748DB9B1650ADFEC8868003158D7803A0652FE76FBD2D617B0",
"2D280ADE3C870B2AD6B962DE191743C763823EC07E897846E5398546E9D69EC7",
"354AF480EE43126FDC3DD11FD0F6537647F8413B7676536DFFFC671F850DF8A4",
"<and more>"
]
Now we will start processing the function. We’ve automated transactions to be made from the private key we made earlier, and to randomly choose recipients and signees. This process is going to take about 2 hours so brb reading and gaming.
npm run create32
Now we have 8 text files with 320,000 transactions each. The file created is lined with payloads.
A50000000363C105F26EC44563137F35A3389E17E15541171118CA45A2253BE379226A0E2142F03DF9F54A14140456B35B1154C6D87BC5C9C771AF66D4883DB7263B970FAAD87BCE49947988710B561FF8F197791E82D15FAE02C1FC661913221B3168C5039054410000000000000000068181361300000090D122B0A1031846C76CBC7D59B298B4CE74307D1562F8CEE60100010029CF5FD941AD25D580841E0000000000
A50000000A201FD031F321F37875266BABAB299B18D3D3B07995C40E1C4F45F838DCD8F776DB75EF24FC6807BB9458FD5B578436DFECE17FEEB7393EE8489B8EB84B9708BDC47DA1770D60BE59458955F6FAFEB1BCCAC724068A08B16E709B49B038AC6303905441000000000000000015818136130000009038E6207AD52AAAA352952E40303B9507A9B548AC96D0176E0100010029CF5FD941AD25D5808D5B0000000000
A5000000C19C58720E65C1FE48586B0697EF251E0C1339596F8631535CE721739330F44A1CBCC66B1B01D0D1E52DBC7A963CB86CEA602A220CCBC9F3336E007AAC4F1906A111E52857B11526FC947E5B35BEB120003CF683D8D58C933C66B9F2479E3ED5039054410000000000000000248181361300000090A3ADA63DEB12930DE99439873D54679EEA3D8D50D293FA3C0100010029CF5FD941AD25D580841E0000000000
A5000000D54EFB265418A258BED1E66D5D2AB00DAC428A402500D117D2CD36CA8920A68F06FC88606D0CEC636D73DB4BD698FCB6632CBE5677D7F10288879F52E85FEA0BEF895B241FE52C943732D66D331810F96859AAF5F01B8ABD11B2AB8B5F4725380390544100000000000000004681813613000000901451D6A14DD5998906CA091C7E5F3D941233717EEE3304E70100010029CF5FD941AD25D5C0CF6A0000000000
A500000006347DABEE0F6037957A1E80232BC2DFF32ACA313C1592D16D222634EFED5E651424FF6FE49B1395BF7BA3BF72402CB35108DEBFBEFB23E26D45814B57A35707A594C49AF8E95AE87450D99957BF7052A00CDE4E2345475415C081CCC53F401103905441000000000000000067818136130000009091064DF93DAE8C16706BEDA010AD0E4475B513E3E1CBF2850100010029CF5FD941AD25D580841E0000000000
A500000074662C063BE55D7D01A9A412791E532B70CDCFF93451DBFE5B011F170193FD5A134030BF4148E35B5FC54E310CA4A36CCF314E9838046A9C476450C7C7D7E101BDC47DA1770D60BE59458955F6FAFEB1BCCAC724068A08B16E709B49B038AC63039054410000000000000000908181361300000090D122B0A1031846C76CBC7D59B298B4CE74307D1562F8CEE60100010029CF5FD941AD25D500127A0000000000
A50000000BF8C02E4DFD5A90DF65A7D5B18C701819C2F234FED93CBD6D6D8B01D78ADADED9B0CC316AFCAAE31FBDBDD4CE83646749FB693CF7B23F8CC96A04289AC4500A3548D5BD43DD840726778609A6E5CCAF7359A80E8EF4117F07010CE7E54BFB800390544100000000000000009F818136130000009016CDA788B3EDF7C3498EF5CCDB24885BBB7977FD661B40860100010029CF5FD941AD25D500127A0000000000
A5000000B32236F28B618D562367A2F52362C1311A93D8291AFFD3937840FB1BCC50F7C4C2E2D09CE9216BCBEBAD8890686B653AB2D5F10C7F02343C3C012B405BFFF30B8364AD86104064C268D166E2522A8D88C36FD3BB6FEA9862C679F2706178FBEB039054410000000000000000C281813613000000901451D6A14DD5998906CA091C7E5F3D941233717EEE3304E70100010029CF5FD941AD25D5C0CF6A0000000000
A500000062E538ED968016ABF8500DAE6A727842FB56CF582D5BAF9357D3E23EA677432C81141D11A12C3BC1A8A586966AA46699D4F7B9BBAE3E41D2677400FA374B6D0408F034041AB4C0CB6692985BE06DDB887289F52D2E0A95AFF9C6D2A3BD42055E039054410000000000000000828181361300000090C8453CA8D5FAB0A7363F448311FD0FA266C646D59C9D312B0100010029CF5FD941AD25D580841E0000000000
A50000002A028616EE26B5EC09BC15A8A2742A8CD5E4A1D8494BDB4E4E0D35870CEB14F2DCB1B72456A36C35BA0A20BFB76519462FA7799FAA0100570A1CC4A82DE6830CDCB11E2550BDBF17C7D89E86B0E2C80A5B63DBA187CC1780D905951DF3135915039054410000000000000000E381813613000000909B9DC59709A46FABF6EBE28A80794D443DDAF91FE9DB1D300100010029CF5FD941AD25D5C0CF6A0000000000
...
Sending the transactions
I’ll try sending. 1 first, then sending 500 after. (send 1 and send 500 respectively)
npm run send1
npm run send500
We can keep track of this level of transaction through Block Explorer. (Http-access to VM port 8000 through your browser)
Now let’s try sending 320,000 x 8 transactions. This will probably take about 15 minutes. At this quantity, block explorer will most likely crash so I’ll keep track through the following method.
npm run send32
On Block Explorer, “CONTENT_LENGTH_MISMATCH”, this error stops displaying transactions. So we will use “nem2-sdk” to fetch data and excel to count transactions.
npm run view
Toggle
- [sec]：block production intervals
- [TXs]：amount of transactions per block
- [tx/sec]：transactions per second
| Height | Timestamp | [sec] | [TXs] | [tx/sec] |
|---|---|---|---|---|
| 1 | 2016-04-01T00:00:00.000Z | 25 | 0000.00 | |
| 2 | 2018-12-09T05:14:51.094Z | 51 | 0 | 0000.00 |
| 3 | 2018-12-09T05:15:13.097Z | 22 | 1 | 0000.05 |
| 4 | 2018-12-09T05:15:36.100Z | 23 | 0 | 0000.00 |
| 5 | 2018-12-09T05:16:04.103Z | 28 | 500 | 0017.86 |
| 6 | 2018-12-09T05:16:16.106Z | 12 | 0 | 0000.00 |
| 7 | 2018-12-09T05:16:38.108Z | 22 | 13146 | 0597.55 |
| 8 | 2018-12-09T05:17:06.123Z | 28 | 113285 | 4045.89 |
| 9 | 2018-12-09T05:17:30.251Z | 24 | 98982 | 4124.25 |
| 10 | 2018-12-09T05:17:56.340Z | 26 | 105007 | 4038.73 |
| 11 | 2018-12-09T05:18:19.433Z | 23 | 28303 | 1230.57 |
| 12 | 2018-12-09T05:18:41.470Z | 22 | 153513 | 6977.86 |
| 13 | 2018-12-09T05:18:54.594Z | 13 | 25990 | 1999.23 |
| 14 | 2018-12-09T05:19:19.624Z | 25 | 127507 | 5100.28 |
| 15 | 2018-12-09T05:19:41.733Z | 22 | 43008 | 1954.91 |
| 16 | 2018-12-09T05:20:05.772Z | 24 | 140779 | 5865.79 |
| 17 | 2018-12-09T05:20:35.922Z | 30 | 125231 | 4174.37 |
| 18 | 2018-12-09T05:20:47.040Z | 12 | 28053 | 2337.75 |
| 19 | 2018-12-09T05:21:11.064Z | 24 | 30522 | 1271.75 |
| 20 | 2018-12-09T05:21:35.092Z | 24 | 179782 | 7490.92 |
| 21 | 2018-12-09T05:21:56.232Z | 21 | 18894 | 0899.71 |
| 22 | 2018-12-09T05:22:14.254Z | 18 | 138686 | 7704.78 |
| 23 | 2018-12-09T05:22:37.431Z | 23 | 30691 | 1334.39 |
| 24 | 2018-12-09T05:22:51.469Z | 14 | 118690 | 8477.86 |
| 25 | 2018-12-09T05:23:09.561Z | 18 | 16144 | 0896.89 |
| 26 | 2018-12-09T05:23:17.595Z | 8 | 89018 | 11127.25 |
| 27 | 2018-12-09T05:23:42.667Z | 25 | 100977 | 4039.08 |
| 28 | 2018-12-09T05:24:00.747Z | 18 | 72699 | 4038.83 |
| 29 | 2018-12-09T05:24:18.804Z | 18 | 72705 | 4039.17 |
| 30 | 2018-12-09T05:24:39.881Z | 21 | 32319 | 1539.00 |
| 31 | 2018-12-09T05:24:53.919Z | 14 | 111107 | 7936.21 |
| 32 | 2018-12-09T05:25:16.017Z | 23 | 88856 | 3863.30 |
| 33 | 2018-12-09T05:25:38.100Z | 22 | 19475 | 0885.23 |
| 34 | 2018-12-09T05:25:57.128Z | 19 | 146135 | 7691.32 |
| 35 | 2018-12-09T05:26:15.261Z | 18 | 24239 | 1346.61 |
| 36 | 2018-12-09T05:26:34.284Z | 19 | 125204 | 6589.68 |
| 37 | 2018-12-09T05:26:48.377Z | 14 | 44516 | 3179.71 |
| 38 | 2018-12-09T05:27:04.442Z | 16 | 78799 | 4924.94 |
| 39 | 2018-12-09T05:27:14.504Z | 10 | 14025 | 1402.50 |
| 40 | 2018-12-09T05:27:38.517Z | 24 | 0 | 0000.00 |
| 41 | 2018-12-09T05:27:59.520Z | 21 | 0 | 0000.00 |
| 42 | 2018-12-09T05:28:17.522Z | 18 | 0 | 0000.00 |
RESULTS
As we can see above, we achieved 4,000 transactions per second (!), by averaging 4,040 transactions per second using 8-38 blocks (2,529,116 transactions by 626 seconds).
…And how much did this cost?
Table contents listed respectively
VM SIZE, OFFERING, FAMILY, VCPUS, RAM (GB), DATA DISKS, MAX IOPS, TEMPORARY STORAGE, PREMIUM DISK SUPPORT, COST/MONTH (ESTIMATED).
This time we used the Azure VM F32_v2 that ran about 1,500 US dollars a month. This cost is excluding any costs for data transfers and disc usage.
Diagram pictured above shows a approximate price breakdown. A monthly cost of 175,655 Japanese Yen, usage costs and fees from network, discs and mijin.
Now I can finally tell people, in confidence, that 4,000 transactions per second is possible in a private Catapult environment. In addition, I did no tuning in this experiment so a more price point model could more than likely be possible. Please let me know what you think of anything above!
Thank you!
@planethouki
