This contest started as a fun chat with an employee at HackerRank and how it ended up with a coding contest. One of the engineers from our team, Aditi Tayal, has recently started learning to build a web app using Mysql and Redis. She built a simple task manager and shared this link with me https://aditiflaskcrudapptutorial.herokuapp.com/.
My first instinct is to figure out how well the app scales and does it have any security issue. I built a small script to insert tasks at a rapid rate and she wrote a code to delete the entries I made. Soon, we were trying to figure out which code runs faster. And ended up planning for a contest to see which script runs fast.
The rules of the contest were as follows
- The script/code we write will run for 120 seconds. We will use timeout to make sure the code runs exactly for 120 seconds
- We will run the script from my personal server in linode.com, so that we there is no unfairness because of home internet speed.
- The script which can make the maximum number of insert within the 2 minutes wins.
- We still haven’t figured out what the winner is going to get.
My initial plan was to write a script using go (something which I’ve been planning to learn for a while), but ended up just using ab
My only worry is that since this is a single web server with not many scaling options if I hit the server with a very high concurrency, it might end up chocking. So, I ended up running ab with multiple concurrent options.
| Concurrency Level | Total Time | Request count | Requests per sec | Time per request | Connection Times | |||
|---|---|---|---|---|---|---|---|---|
| Connect | Processing | Waiting | Total | |||||
| 50 | 137.546 | 12000 | 87.24 | 11.462 | 218 | 353 | 352 | 570 |
| 100 | 131.48 | 12000 | 91.27 | 10.957 | 219 | 871 | 870 | 1090 |
| 200 | 126.304 | 12000 | 95.01 | 10.525 | 218 | 1869 | 1868 | 2087 |
| 250 | 119.747 | 12000 | 100.21 | 9.979 | 219 | 2250 | 2249 | 2469 |
| 300 | 126.955 | 12000 | 94.52 | 10.58 | 222 | 2904 | 2903 | 3125 |
| 300 | 78.369 | 12000 | 153.12 | 6.531 | 220 | 1708 | 1707 | 1928 |
| 300 | 80.179 | 12000 | 149.66 | 6.682 | 222 | 1751 | 1750 | 1973 |
| 300 | 75.941 | 12000 | 158.02 | 6.328 | 222 | 1643 | 1643 | 1865 |
| 300 | 76.482 | 12000 | 156.9 | 6.373 | 222 | 1655 | 1654 | 1876 |
| 300 | 79.311 | 12000 | 151.3 | 6.609 | 220 | 1707 | 1707 | 1928 |
| 300 | 91.529 | 12000 | 131.11 | 7.627 | 220 | 2032 | 2031 | 2252 |
| 350 | 108.376 | 12000 | 110.73 | 9.031 | 221 | 2889 | 2888 | 3110 |
| 400 | 114.424 | 12000 | 104.87 | 9.535 | 222 | 3502 | 3501 | 3724 |
| 400 | 89.537 | 12000 | 134.02 | 7.461 | 223 | 2706 | 2706 | 2929 |
| 400 | 114.09 | 12000 | 105.18 | 9.507 | 223 | 3523 | 3522 | 3746 |
| 400 | 121.505 | 12000 | 98.76 | 10.125 | 223 | 3770 | 3769 | 3993 |
| 400 | 86.615 | 12000 | 138.54 | 7.218 | 223 | 2611 | 2611 | 2834 |
| 400 | 96.388 | 12000 | 124.5 | 8.032 | 222 | 2935 | 2934 | 3157 |
| 500 | 102.697 | 12000 | 116.85 | 8.558 | 226 | 2974 | 2973 | 4201 |
| 500 | 95.559 | 12000 | 125.58 | 7.963 | 226 | 3650 | 3649 | 3875 |
| 600 | 104.151 | 12000 | 115.22 | 8.679 | 230 | 4848 | 4847 | 5078 |
| 700 | 115.35 | 12000 | 104.03 | 9.613 | 239 | 6267 | 6266 | 6506 |
| 750 | 118.049 | 12000 | 101.65 | 9.837 | 240 | 6634 | 6634 | 6874 |
| 800 | 115.709 | 12000 | 103.71 | 9.642 | 245 | 7259 | 7259 | 7505 |
| 900 | 110.169 | 12000 | 108.92 | 9.181 | 248 | 7644 | 7643 | 7892 |
| 1000 | 119.041 | 12000 | 100.81 | 9.92 | 256 | 9033 | 9033 | 9289 |
The only number to worry about is time taken. Here is the plot of the time taken to hit 12K requests vs concurrency level.
Looking at the graph above, guessing 300 concurrency is the best.
Results
We ran the script thrice for 2 minutes. Each of the run was a completely different where thing were good, bad and finally evil.
Fair Run
I let her script run for 2 minutes. A very fair game. I didn’t do anything to disrupt the overall timings or anything at all. In this one, I was able to run 12923 rows. Aditi was able to get only 12140 (though the script had a few extra print commands in it). When we ran the same script in her machine, she was able to hit close 11k+, and my ab script reached only 8k. We both mutually decided to consider the first result which ran in my Linode server and agreed that I won the contest. We may do a re-run soon after we fix some issues in our code.
Unfair Run
It is hard to be fair in a game, where you have so many ways to break the rules. This time, I was running another script to fetch the data while she ran her script to insert new records. The reads were hit at a high rate, that the simple SQLite server couldn’t handle both the reads and writes. And all I had to run was
ab -n 10000 -c 500 https://aditiflaskcrudapptutorial.herokuapp.com/
Evil Run
Taking the unfair game to the next level, this time I tried to be pure evil. I was running another script to actually delete the rows she was adding while she ran her script. In fact, she even wrote this 3 line python script, which made the job a lot easier for me. This was fun 🙂 The only catch is to delete only a few rows so that she doesn’t get suspicious.
import requests
for i in range(1, 5000):
content = requests.get('https://aditiflaskcrudapptutorial.herokuapp.com/delete/'+str(i))
Overall, this was a fun experiment. Even though I won, what matters the most is that we learned a bunch of things about ab, timeout, and how a single web-server behaves under pressure.