Azure Functions vs. Azure Web Jobs vs. Azure Batch

What And Why Background Processing?

Background processing is a computer process that runs in the background and without user intervention. It’s utilized in a solution design where a task is initiated and processed in separate computing infrastructure. For example, when a user submits a request to develop an image processing job. The server will respond when the job is produced, confirming that the requested job has been created and queued for processing. The project processor runs in a separate machine to update the job status. The user can get results, monitor project status, and obtain results.

There are several reasons to use this approach:

The first reason is to separate the process that creates the jobs from the process that executes them. This separation avoids the condition where the server that serves users’ requests becomes too busy with job processing, making users unable to connect with it. The whole system then becomes unresponsive.

The second reason is scalability. Having the processes run on separate computing infrastructure will make it easier to scale them independently.

Now we will talk about the options that the Azure cloud platform provides to run background job processing.

Azure Functions

Azure Functions is a computing platform that allows you to build functions with various triggers events, including queue triggers. It supports multiple programming languages, e.g., C#, Java, Javascript, PowerShell, Python.

Its queue trigger functions model allows you to quickly create a queue-based job processor with built-in queue polling and poison queue handling.

It provides three primary pricing models: Consumption plan, Premium plan, and an App Service plan.

Advantages:

1. Easy to create queue trigger functions.
2. Built-in poison queue handling.
3. With the Consumption plan, we only pay when our functions are running.

Drawbacks And Constraints:

1. With the Consumption plan, there is 5 minutes time limit of execution. However, if you need to run longer, you must upgrade to the App Service plan to receive unlimited run time.
2. The consumption plan runs in shared computing infrastructure. Your function might not always be available, but this is called cold start, which will take longer for the process to be up and running. If you need something available continuously, then you will have to upgrade to App Service Plan.
3. A consumption plan does not allow you to control the scale-out. For queue trigger functions, it will scale- out/in based on the queue length.

Want to learn more? Go to Azure Functions Overview | Microsoft Docs

Azure Batch

Azure Batch allows us to run, manage and scale a pool of virtual machines, aka nodes in Azure. Using Azure Batch API, we can install applications that we want to run in the nodes. Works are organized into jobs and tasks. You can think of jobs as the container of functions.

There are two types of VM in Azure Batch. By allocating Low Priority VM, you can take advantage of surplus capacity in Azure. Tasks running in low priority VM might be pre-empted, i.e., task execution is aborted and rescheduled. The other VM type is Dedicated VM, which is a VM that is allocated 100% to running your tasks.

Unlike Azure Functions, Batch does not have an out-of-the-box queue trigger task model. Fortunately, it provides Azure Batch API that allows us to create tasks. Combined with table and queue storage, we can create a program that will monitor the jobs queue and make a batch task for each job, as illustrated in figure 2.

Advantages:

1. It can run any program executable in the target virtual machine, allowing you to write using your preferred programming language.
2. It provides low execution cost with its Low Priority VM.
3. Tasks run time limit is seven days. In my opinion, this is a long time compared with the Azure Function Consumption plan’s five-minute limit.
4. Very easy to scale.

Drawbacks and Constraints:

1. Tasks that are running in Low priority VM might be pre-empted.
2. No built-in queue trigger and poison queue handling.

Azure WebJobs

Azure Web Jobs allows you to run background processing in Azure App Service. It is very similar to Azure Functions, which is developed using Azure WebJobs SDK.

Advantages:

1. It provides queue trigger models with built-in poison queue handling.
2. We can run WebJobs in the same app service where we run your web app. It is a good option if you don’t want to pay for separate computing power to run background processing.

Drawback And Constraints:

1. It -only supports C#.
2. It only runs in Azure Windows App Service and is unavailable in the Linux app service.
3. Pay-per-use pricing is not available.

Summary

Azure Functions and Web Jobs programming models provide many options for triggering events to focus more on the business process.

Azure Batch provides a computing platform that allows us to run tasks in a highly scalable pool of VMs. We need to write some of our own ‘plumbing’ code to monitor and trigger job processing for background job processing.

Each platform has advantages and drawbacks. When choosing which platform to use, we need to consider how much time and encompassing the job is to be processed, programming language support, programming models, availability, and operational cost. If you need to run background processing into an existing Azure app service, Azure WebJobs is the best option. If you need to run behind-the-scenes processing in a separate Azure app service and utilize Azure WebJobs SDK, then Azure Function is the best choice. Use Azure Batch to efficiently run large-scale parallel and high-performance computing (HPC) batch jobs in Azure.

For optimum results, we can also mix the usage of the three platforms in our solution.

Author:
Anang Prihatanto – Analyst

References:
[1] Background Process https://en.wikipedia.org/wiki/Background_process
[2] Azure Batch Overview Azure Batch runs large parallel jobs in the cloud – Azure Batch | Microsoft Docs