{"id":1177,"date":"2020-06-07T13:19:42","date_gmt":"2020-06-07T13:19:42","guid":{"rendered":"https:\/\/sp2hari.com\/?p=1177"},"modified":"2020-06-07T19:58:55","modified_gmt":"2020-06-07T19:58:55","slug":"request-war","status":"publish","type":"post","link":"https:\/\/sp2hari.com\/index.php\/2020\/06\/07\/request-war\/","title":{"rendered":"Request War"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">This contest started as a fun chat with an employee at HackerRank and how it ended up with a coding contest. One of the engineers from our team, <a rel=\"noreferrer noopener\" href=\"https:\/\/www.linkedin.com\/in\/adititayal9\/\" target=\"_blank\">Aditi Tayal<\/a>, has recently started learning to build a web app using Mysql and Redis. She built a simple task manager and shared this link with me <a href=\"https:\/\/aditiflaskcrudapptutorial.herokuapp.com\/\">https:\/\/aditiflaskcrudapptutorial.herokuapp.com\/<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My first instinct is to figure out how well the app scales and does it have any security issue. I built a small script to insert tasks at a rapid rate and she wrote a code to delete the entries I made. Soon, we were trying to figure out which code runs faster. And ended up planning for a contest to see which script runs fast. <\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.27.37-PM.png\" alt=\"\" class=\"wp-image-1182\" width=\"631\" height=\"267\" srcset=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.27.37-PM.png 973w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.27.37-PM-300x127.png 300w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.27.37-PM-768x325.png 768w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.27.37-PM-708x300.png 708w\" sizes=\"auto, (max-width: 631px) 100vw, 631px\" \/><figcaption>Task list before the contest.<\/figcaption><\/figure><\/div>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.26.59-PM.png\" alt=\"\" class=\"wp-image-1183\" width=\"646\" height=\"324\" srcset=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.26.59-PM.png 973w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.26.59-PM-300x150.png 300w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.26.59-PM-768x385.png 768w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Screenshot-2020-06-03-at-3.26.59-PM-598x300.png 598w\" sizes=\"auto, (max-width: 646px) 100vw, 646px\" \/><figcaption>Task list after the contest. I am going to assume that &#8220;slap him&#8221; was not meant for me.<\/figcaption><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">The rules of the contest were as follows<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>The script\/code we write will run for 120 seconds. We will use timeout to make sure the code runs exactly for 120 seconds<\/li><li>We will run the script from my personal server in linode.com, so that we there is no unfairness because of home internet speed.<\/li><li>The script which can make the maximum number of insert within the 2 minutes wins. <\/li><li>We still haven&#8217;t figured out what the winner is going to get.<\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">My initial plan was to write a script using go (something which I&#8217;ve been planning to learn for a while), but ended up just using ab<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">My only worry is that since this is a single web server with not many scaling options if I hit the server with a very high concurrency, it might end up chocking. So, I ended up running ab with multiple concurrent options. <\/p>\n\n\n\n<style type=\"text\/css\">\n\ttable.tableizer-table {\n\t\tfont-size: 14px;\nwidth:98%;\n\t\tborder: 1px solid #CCC; \n\t\tfont-family: Arial, Helvetica, sans-serif;\n\t} \n\t.tableizer-table td {\n\t\tpadding: 4px;\n\t\tmargin: 3px;\n\t\tborder: 1px solid #CCC;\n                text-align: center;\n\t}\n\t.tableizer-table th {\n\t\tbackground-color: #104E8B; \n\t\tcolor: #FFF;\n\t\tfont-weight: bold;\n                text-align: center;\n\t}\n<\/style>\n<center>\n<table class=\"tableizer-table\">\n<thead><tr class=\"tableizer-firstrow\"><th>Concurrency Level<\/th><th>Total Time<\/th><th>Request count<\/th><th>Requests per sec<\/th><th>Time per request<\/th><th colspan=\"4\" width=\"35%\">Connection Times<\/th><\/tr><\/thead><tbody>\n <tr><td>&nbsp;<\/td><td>&nbsp;<\/td><td>&nbsp;<\/td><td>&nbsp;<\/td><td>&nbsp;<\/td><td>Connect<\/td><td>Processing<\/td><td>Waiting<\/td><td>Total<\/td><\/tr>\n <tr><td>50<\/td><td>137.546<\/td><td>12000<\/td><td>87.24<\/td><td>11.462<\/td><td>218<\/td><td>353<\/td><td>352<\/td><td>570<\/td><\/tr>\n <tr><td>100<\/td><td>131.48<\/td><td>12000<\/td><td>91.27<\/td><td>10.957<\/td><td>219<\/td><td>871<\/td><td>870<\/td><td>1090<\/td><\/tr>\n <tr><td>200<\/td><td>126.304<\/td><td>12000<\/td><td>95.01<\/td><td>10.525<\/td><td>218<\/td><td>1869<\/td><td>1868<\/td><td>2087<\/td><\/tr>\n <tr><td>250<\/td><td>119.747<\/td><td>12000<\/td><td>100.21<\/td><td>9.979<\/td><td>219<\/td><td>2250<\/td><td>2249<\/td><td>2469<\/td><\/tr>\n <tr><td>300<\/td><td>126.955<\/td><td>12000<\/td><td>94.52<\/td><td>10.58<\/td><td>222<\/td><td>2904<\/td><td>2903<\/td><td>3125<\/td><\/tr>\n <tr><td>300<\/td><td>78.369<\/td><td>12000<\/td><td>153.12<\/td><td>6.531<\/td><td>220<\/td><td>1708<\/td><td>1707<\/td><td>1928<\/td><\/tr>\n <tr><td>300<\/td><td>80.179<\/td><td>12000<\/td><td>149.66<\/td><td>6.682<\/td><td>222<\/td><td>1751<\/td><td>1750<\/td><td>1973<\/td><\/tr>\n <tr><td>300<\/td><td>75.941<\/td><td>12000<\/td><td>158.02<\/td><td>6.328<\/td><td>222<\/td><td>1643<\/td><td>1643<\/td><td>1865<\/td><\/tr>\n <tr><td>300<\/td><td>76.482<\/td><td>12000<\/td><td>156.9<\/td><td>6.373<\/td><td>222<\/td><td>1655<\/td><td>1654<\/td><td>1876<\/td><\/tr>\n <tr><td>300<\/td><td>79.311<\/td><td>12000<\/td><td>151.3<\/td><td>6.609<\/td><td>220<\/td><td>1707<\/td><td>1707<\/td><td>1928<\/td><\/tr>\n <tr><td>300<\/td><td>91.529<\/td><td>12000<\/td><td>131.11<\/td><td>7.627<\/td><td>220<\/td><td>2032<\/td><td>2031<\/td><td>2252<\/td><\/tr>\n <tr><td>350<\/td><td>108.376<\/td><td>12000<\/td><td>110.73<\/td><td>9.031<\/td><td>221<\/td><td>2889<\/td><td>2888<\/td><td>3110<\/td><\/tr>\n <tr><td>400<\/td><td>114.424<\/td><td>12000<\/td><td>104.87<\/td><td>9.535<\/td><td>222<\/td><td>3502<\/td><td>3501<\/td><td>3724<\/td><\/tr>\n <tr><td>400<\/td><td>89.537<\/td><td>12000<\/td><td>134.02<\/td><td>7.461<\/td><td>223<\/td><td>2706<\/td><td>2706<\/td><td>2929<\/td><\/tr>\n <tr><td>400<\/td><td>114.09<\/td><td>12000<\/td><td>105.18<\/td><td>9.507<\/td><td>223<\/td><td>3523<\/td><td>3522<\/td><td>3746<\/td><\/tr>\n <tr><td>400<\/td><td>121.505<\/td><td>12000<\/td><td>98.76<\/td><td>10.125<\/td><td>223<\/td><td>3770<\/td><td>3769<\/td><td>3993<\/td><\/tr>\n <tr><td>400<\/td><td>86.615<\/td><td>12000<\/td><td>138.54<\/td><td>7.218<\/td><td>223<\/td><td>2611<\/td><td>2611<\/td><td>2834<\/td><\/tr>\n <tr><td>400<\/td><td>96.388<\/td><td>12000<\/td><td>124.5<\/td><td>8.032<\/td><td>222<\/td><td>2935<\/td><td>2934<\/td><td>3157<\/td><\/tr>\n <tr><td>500<\/td><td>102.697<\/td><td>12000<\/td><td>116.85<\/td><td>8.558<\/td><td>226<\/td><td>2974<\/td><td>2973<\/td><td>4201<\/td><\/tr>\n <tr><td>500<\/td><td>95.559<\/td><td>12000<\/td><td>125.58<\/td><td>7.963<\/td><td>226<\/td><td>3650<\/td><td>3649<\/td><td>3875<\/td><\/tr>\n <tr><td>600<\/td><td>104.151<\/td><td>12000<\/td><td>115.22<\/td><td>8.679<\/td><td>230<\/td><td>4848<\/td><td>4847<\/td><td>5078<\/td><\/tr>\n <tr><td>700<\/td><td>115.35<\/td><td>12000<\/td><td>104.03<\/td><td>9.613<\/td><td>239<\/td><td>6267<\/td><td>6266<\/td><td>6506<\/td><\/tr>\n <tr><td>750<\/td><td>118.049<\/td><td>12000<\/td><td>101.65<\/td><td>9.837<\/td><td>240<\/td><td>6634<\/td><td>6634<\/td><td>6874<\/td><\/tr>\n <tr><td>800<\/td><td>115.709<\/td><td>12000<\/td><td>103.71<\/td><td>9.642<\/td><td>245<\/td><td>7259<\/td><td>7259<\/td><td>7505<\/td><\/tr>\n <tr><td>900<\/td><td>110.169<\/td><td>12000<\/td><td>108.92<\/td><td>9.181<\/td><td>248<\/td><td>7644<\/td><td>7643<\/td><td>7892<\/td><\/tr>\n <tr><td>1000<\/td><td>119.041<\/td><td>12000<\/td><td>100.81<\/td><td>9.92<\/td><td>256<\/td><td>9033<\/td><td>9033<\/td><td>9289<\/td><\/tr>\n<\/tbody><\/table>\n<\/center>\n\n\n\n<p class=\"wp-block-paragraph\">The only number to worry about is time taken. Here is the plot of the time taken to hit 12K requests vs concurrency level.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"472\" src=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2-1024x472.png\" alt=\"\" class=\"wp-image-1208\" srcset=\"https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2-1024x472.png 1024w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2-300x138.png 300w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2-768x354.png 768w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2-651x300.png 651w, https:\/\/sp2hari.com\/wp-content\/uploads\/2020\/06\/Time-Taken-vs.-Concurrency-Level-2.png 1078w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Looking at the graph above, guessing 300 concurrency is the best. <\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Results<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">We ran the script thrice for 2 minutes. Each of the run was a completely different where thing were good, bad and finally evil.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Fair Run<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">I let her script run for 2 minutes. A very fair game. I didn&#8217;t do anything to disrupt the overall timings or anything at all. In this one, I was able to run 12923 rows. Aditi was able to get only 12140 (though the script had a few extra print commands in it). When we ran the same script in her machine, she was able to hit close 11k+, and my ab script reached only 8k. We both mutually decided to consider the first result which ran in my Linode server and agreed that I won the contest. We may do a re-run soon after we fix some issues in our code.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Unfair Run<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">It is hard to be fair in a game, where you have so many ways to break the rules. This time, I was running another script to fetch the data while she ran her script to insert new records. The reads were hit at a high rate, that the simple SQLite server couldn&#8217;t handle both the reads and writes. And all I had to run was<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">ab -n 10000 -c 500 https:\/\/aditiflaskcrudapptutorial.herokuapp.com\/<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Evil Run<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Taking the unfair game to the next level, this time I tried to be pure evil. I was running another script to actually delete the rows she was adding while she ran her script. In fact, she even wrote this 3 line python script, which made the job a lot easier for me. This was fun \ud83d\ude42 The only catch is to delete only a few rows so that she doesn&#8217;t get suspicious. <\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">import requests\nfor i in range(1, 5000):\n  content = requests.get('https:\/\/aditiflaskcrudapptutorial.herokuapp.com\/delete\/'+str(i))<\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Overall, this was a fun experiment. Even though I won, what matters the most is that we learned a bunch of things about ab, timeout, and how a single web-server behaves under pressure.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This contest started as a fun chat with an employee at HackerRank and how it ended up with a coding contest. One of the engineers from our team, Aditi Tayal, has recently started learning to build a web app using Mysql and Redis. She built a simple task manager and &hellip;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[13],"tags":[],"class_list":["post-1177","post","type-post","status-publish","format-standard","hentry","category-code"],"_links":{"self":[{"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/posts\/1177","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/comments?post=1177"}],"version-history":[{"count":44,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/posts\/1177\/revisions"}],"predecessor-version":[{"id":1229,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/posts\/1177\/revisions\/1229"}],"wp:attachment":[{"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/media?parent=1177"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/categories?post=1177"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sp2hari.com\/index.php\/wp-json\/wp\/v2\/tags?post=1177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}