java: what are the best techniques for communicating with a batch server?

I've a WEB application (with pure Java servlet) that have some heavy computational work, with database access, that can be done in asynchronous mode. I'm planning to use a dedicated server to execute such batch jobs and I'm wondering which tools/techniques/protocols to use for communication between servlets in the WEB server and batch jobs in the new dedicated server. I'm looking at JMS. Is it the right choice? There are industry standard and/or widely adopted techniques? I need also queue and priority handling for multiple simultaneous jobs.

Asked by: Miranda312 | Posted: 28-01-2022

Answer 1

JMS is a pretty standard solution. The high-end platforms (Sun's JCAPS, for example) makes heavy use of JMS to partition and manage the workload of web services.

There are many advantages to buying a high-end JMS implementation from Sun (or IBM or Microsoft). First, you get things like reliable message queues that are backed to the file system. No message can get lost. Second, you get some monitoring and management tools.

One cool thing is to have a JMS queue with (potentially) multiple subscribers to do workload balancing.

Another cool thing is to have JMS topic which has a logging process as well as the real work process subscribed. The logging process picks off the messages and simply records the essential stages of the job being started and stopped.

Answered by: Maddie647 | Posted: 01-03-2022

Answer 2

Messaging is one of the best options.

Make the messaging framework very generic so that it can handle any type of batch jobs.

One approach is to have an event/task manager where you put an event on the queue and the queue consumer processes the event and converts it into a set of tasks. The tasks can then be executed by separate task handlers. A task can also generate some more events that can be again put on the queues to provide a feedback loop. This way you can add work flow like features to the framework and allow your batch jobs to have dependencies on each other.

Answered by: Miller610 | Posted: 01-03-2022

Answer 3

JMS would be the appropriate solution for sending your batch jobs from the servlet. It may not be the best solution for the batch server to communicate with the servlet though, as it cannot be a listener to messages.

As I don't know what the communication from the batch server to the servlet is supposed to entail, I can only say that there are probably several options you can use (yes JMS is one of them). But they all basically rely on polling calls to the servlet which will then check in some way to see if there is anything from the batch server waiting. This could simply be a servlet on the batch server or making receive calls to a JMS response queue. Other solutions are available, but the point is it is not asynchronous, unless you have the ability to push from the batch server all the way to you client end (a browser I am guessing) via something like AJAX.

Anyway, just something to keep in mind.

Answered by: Lucas477 | Posted: 01-03-2022

Answer 4

Another alternative for asynchronous processing is to have the web application store the request in the database, and have the batch process poll the database for new batch jobs to process. Since your application appears to be smaller (pure Java Servlets) this may be a simpler and lower cost solution.

Hope it helps.

Answered by: David669 | Posted: 01-03-2022

Answer 5

We use JMS with web services:

  1. Client requests computation via web service
  2. Server writes JMS message, and creates an ID value which is stored in a database along with a status (initially "Pending"). Server returns the id to the client.
  3. Server (can be separate server) reads JMS message, does computation, and when finished updates the status to "Completed" in the database
  4. While the computation is ongoing, the client is polling the server to determine the status using another web service (along with the id). The server returns the status which is retrieved from the database. Once the server computation is completed, the client will see the "Completed" status and know that the computation is complete.

Answered by: Vanessa808 | Posted: 01-03-2022

Similar questions

java - Should I expect problems when communicating via RMI between Java5 and Java6?

Basically the subject says it all: We have a couple of components running on Java 5, they're talking to each other via RMI. Should we expect any problems, if we move some of them to Java6? By moving I mean compiling them with -source/target 1.6 and running on a java6 vm.

What are Java's native ways of communicating with devices directly?

What are Java's native ways of communicating with devices or ports such as LPT1, COM1, USB directly?

java - How to Communicating between a J2ME Midlet and PC?

How do I make a J2ME Midlet to communicate with a java program or some application on the pc through the cable connected?

flash - Communicating between Adobe Air or browser and Java Web Start

We're designing an application and the client has requested that a portion of their app stay in Java Web Start and another portion be in a browser. I'm thinking about AIR as an alternative to the browser because that may give us more features b/c we don't have to stay in the browser security sandbox. How would I go about having an Air app talk to a Java Web Start app? Do they have to talk through a server? I gue...

java - Communicating between classes

I have a form that is divided into two classes. Each class represents the widgets on part of the form. What is the best way to allow these classes to share data between each other and update each other. Example: Button in class A is clicked. Update text field in class C

command line - Java - communicating with a sub app through the input stream

Is there a way I can start a command-line application from java and then send strings (commands) to its input stream and display its response from its output stream? I'm using an application with a pretty sophisticated command line interface (vlc). The application has an interpreter that responds to a set of commands. For example, after I start the app, I can start or stop a movie by issuing the command 'pause' o...

linux - Communicating with a command line tool in Java

I want to use a linux command line tool from my Java program. I start the program and get the output using the Process class ( /* @param args * @throws IOException */ public static void main(String[] args) throws IOException {...

java - Android communicating with servers

I come from a .Net background and recently x-training to java. I wanted to develop an Android application that required simple functionality to communicate with a server. For example, functionality such as post a username/score to a website (or service), or request information such as top 10 scores. What type of communications should I focus on? What is the common technology to use? Does Java have the equiv...

Communicating with C++ process from Java

First, I saw a few Q's about this issue in the site, but didn't see any answer that solve my problem. I have a program written in Java and it calls a cmd program written in C++. (this is an assumption since I don't have the actual source) I know the expected I/O of the C++ program, in the cmd it is two lines of output and then it waits for string input. I know that the first output line of the program is through er...

Communicating between Android phone and PC using TCP in java

I'm working on a project where I want my Droid 2 to be able to send and receive data over 3G, to a device connected to an ethernet port (not necessarily a PC, but I'm using this for testing the communication). I believe that eventually the end to end communication will be accomplished over an SSL tunnel, but for right now, I just need to establish basic communication to show that the devices can communicate. I've w...

Still can't find your answer? Check out these amazing Java communities for help...

Java Reddit Community | Java Help Reddit Community | Java Community | Java Discord | Java Programmers (Facebook) | Java developers (Facebook)