Multithreading in real world

Necessity of multithreading becomes obvious when we talk about scenario where we need to process some 100 messages each one taking 20-50 seconds on average to process.
Mostly in these processing ordering doesn’t matter. So instead of processing sequentially we can make use of threads to process them (virtually) parallel.

Consider there is system which processes user scheduled actions by web application. It is commonly known as job scheduler.

Requirement here is that user triggers some activity from UI application. When these triggers should be executed and all that logic depends on scheduling policy for that trigger. So there should be one monitoring thread which will be checking when to execute that trigger based on this scheduling policy. Now this execution can take some time for execution which is completely unknown to executer. Also there can be more than one job ready for execution at that time. So it is necessary for executer to execute this job in separate thread and return back it its actual monitoring work.

So main multithreading concepts used by executer. Executer uses threadpool. Threadpool has some configured number of threads configured which keeps monitoring some jobqueue. Whenever thread finds some job in the queue it removes it from queue and owns it for its execution. Then it continues polling.
There can be other approach to this implementation. Like wait-notify approach. Thread will wait for some condition and then will be notified to all by main monitoring thread.

Thread pool size is configured based on load of processing. If there are more messages to be processed simultaneously to reduce response time then we can increase the thread pool size. Thing to remember is that thread creation is heavy process and should be minimized by reusing the thread.

This wait-notify or task queue approaches works when triggering and triggered both thread are running in same JVM. Mostly the case is where these triggering threads are in different JVMs. E.g. Web application with which user is interacting is one thread of process running in container. Actions triggered by this user interaction may cause job monitor to execute some job in some other JVM. This communication can be easily handled by database. User interaction with the screen will create entry in database and background job scheduler will get triggered from this database entry.

Main problem is when we want to notify immediately to job scheduler to execute some critical job. You can’t use wait-notify or job queue approach. One way to do this is to use Oracle signals. Concept is almost same as that of wait-notify only difference is in wait-notify we wait on object locks which are bounded to its JVM but in this case we can wait-notify at database level so spanning more than one JVM’s.