How a reduction in the number of threads led to a performance improvement @ Fudr !

Intro

In the technology world, we techies are always crunched for performance. From …

Seconds to sub-seconds

Less than 500 milliseconds

Less than 100 milliseconds ( more common these days )

Too often the answer is creating a new Thread. It works too, but only till a certain limit.

For the non-techies among us : A Thread is like a servant who does the work on our behalf. Lots of work can get delegated. And so the work can finish quickly.

.Having lots of servants seems like a good idea ! Doesn’t it ?

Sanskrit has a rule that says : Ati Sarvatra Varjayet. Meaning, that the excess of anything is bad. Goes for Threads too !

We look at how reducing threads in fact helped improve performance.

Threads breeding like Rabbits

A piece of code was doing something like below:

private void doMyWork(final GuestDto guestDto) {
     CompleteableFuture.runAsync(() -&gt; doWorkInBackground(guestDto));
}

While the work gets done in the background, the code creates a new thread for each request. Meaning, for 500 requests 500 threads get created.

On a 2 core CPU can only handle 4 – 8 threads efficiently. Why ?

●When a new thread is born. It consumes certain CPU + Memory.

●Context is copied from the main thread to the worker thread. Again takes some CPU time.

●When an existing thread ends. Garbage collector gets work to do.

Think of the overhead as a servant’s salary.

Too many servants will take too much salary. After a while, the total expense might become too big to be justifiable.

Same goes for threads.

In such cases – 500 threads is surely an overload. Especially during peak volume times. Here is a jMeter performance report for an API with such load.

Simple Solution

Though recognizing the problem might be a bit tricky, the solution is pretty simple.

Create a ThreadPool

Executor service provides an easy way to create and use the pooling functionality.

ExecutorService pool = Executors.newFixedThreadPool(numberOfOptimalThreads);

.
.

Use it !

The pool can then be simply used to trigger all code !

We used a pool with a min size of 10 and max of 20

private void doMyWork(final GuestDto guestDto) {
     CompleteableFuture.runAsync(() -&amp;gt; doWorkInBackground(guestDto), pool);
}

Results

We saw immediate results.

Here is the math :

Old performance : 185 ms

New performance : 165 ms

Improvement : 20 ms

% Improvement : ( 20/185 ) * 100 = ~10%

Meaning, a few minutes change led to a 10% performance improvement. The benefit to effort ratio is huge.

Wrap up

Servants, like threads, are good. But too many of them and the salaries to be paid become an overhead.

Too many servants and the master might go bankrupt !