Difference between softirq, tasklet, workqueue and threaded_irq

 While Linux interrupt handling, To perform "bottom half",  There are couple of methods used as below.

• Tasklet

• It can not sleep.
• It run in software interrupt context
• These are atomic. These are guaranteed to never run on more than one CPU of a given processor, for a given tasklet. In other terms they do not run concurrent, (If you have multiple different tasklet in single driver then they can run concurrent)
• They may be scheduled to run multiple times, but tasklet scheduling is not cumulative; the tasklet runs only once, even if it is requested repeatedly before it is launched
• However, another interrupt can certainly be delivered while the tasklet is running, so locking between the tasklet and the interrupt handler may still be required
• Tasklets can be statically allocated using DECLARE_TASKLET(name, func, data) or can also be allocated dynamically and initialized at runtime using tasklet_init(name, func, data)
• Tasklets are actually run from a softirq

• Workqueue

• It can sleep.
• It run in Non- interrupt based process context.(But Its not complete userspace process)
• These are non atomic.  In Multiprocrssor system, same Workqueue can run on different processors at same time.
• For long processing or higher latenyc processing we should use workqueue.

• softirq

• It can not sleep.
• It run in interrupt context. (Its quite similar to Tasklet)
• Softirqs are statically allocated at compile-time. (Unlike tasklets, you cannot dynamically register and destroy softirqs)
• Softirqs can run concurrently on several CPUs, even if they are of the same type because softirqs are reentrant functions and must explicitly protect their data structures with spinlocks

From https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/interrupt.h?id=v4.2-rc4#n400

/* PLEASE, avoid to allocate new softirqs, if you need not _really_ high
   frequency threaded job scheduling. For almost all the purposes
   tasklets are more than enough. F.e. all serial device BHs et
   al. should be converted to tasklets, not to softirqs.
 */

enum
{
        HI_SOFTIRQ=0,    /* High Priority */
        TIMER_SOFTIRQ,
        NET_TX_SOFTIRQ,
        NET_RX_SOFTIRQ,
        BLOCK_SOFTIRQ,
        BLOCK_IOPOLL_SOFTIRQ,
        TASKLET_SOFTIRQ,
        SCHED_SOFTIRQ,
        HRTIMER_SOFTIRQ,
        RCU_SOFTIRQ,    /* Preferable RCU should always be the last softirq */

        NR_SOFTIRQS
}

Here HI_SOFTIRQ and TASKLET_SOFTIRQ are actually Tasklet 

• threaded irq

As SoftIRQ run in the exit-path of a hardware interrupt handling code when raised by hardware interrupt handlers, they run at a high-priority with scheduler preemption being disabled, not relinquishing CPU to processes/threads until they complete. 

Thus, if functions registered in SoftIRQs fail to finish within a jiffy (1 to 10 milliseconds based on CONFIG_HZ of the kernel), this can severely impact the responsiveness of the kernel as processes/threads would be starved for CPU.

when you use the request_threaded_irq() function passing two functions as arguments, this would create a dedicated kthread which would be visible in the 'ps ax' process listing similar to [irq-16/my_device_intr] where the number 16 represents the irq number associated with this thread, and my_device_intr would be the name provided as a string argument to the request_threaded_irq() function.

The request_threaded_irq() function accepts two functions:

• The handler function. This function acts as a hard-IRQ handler which would ideally read the interrupt-cause or interrupt-status register of the device that raised the interrupt and determine if the processing for the interrupt can finish within few microseconds .If it could finish interrupt processing fast for some set of interrupt causes, it would return IRQ_HANDLED after processing and acknowledging the interrupts. But, if the interrupt processing needs 100s of microseconds or more than a jiffy, it could simply acknowledge the interrupt to the device and return IRQ_WAKE_THREAD. Returning this value makes sense only when the thread_fn is also registered.
• The thread_fn function. When the hard-IRQ handler function returns IRQ_WAKE_THREAD, the associated kthread with this interrupt handler is added to the scheduler run-queue. When this kthread is scheduled, it would invoke the thread_fn function which can perform bottom-half interrupt processing fully in process context. Note that the thread_fn runs competing for CPU along with other processes on the runqueue, but it can run for longer duration without hurting the overall system responsiveness as it does not run in high-priority by default unlike SoftIRQs. When the thread_fn completes, the associated kthread would take itself out of the runqueue and remain in blocked state until woken up again by the hard-IRQ function.

Threaded IRQs are preferred if interrupt processing can take consistently long periods of time. 

One of the advantages of using threaded IRQs in multi-core/SMP architectures is that - you can set affinity of hard-IRQ for one CPU, while setting affinity of the associated kthread to another CPU, thus allowing top half and bottom half to run in parallel.