Spring-mvc – Quartz with spring – clustered environment

quartzquartz-schedulerspring-mvc

I'm using old version of quartz (2.1.2) with spring 3.1.0.
I have simple configuration that should prevent parallel job execution:

<bean id="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
    <property name="jobFactory">
        <bean class="com.azdne.infrastructure.schedule.ScheduledMethodJobFactory" />
    </property>
    <property name="dataSource" ref="schedulerDataSource" />
    <property name="quartzProperties">
        <props>
            <prop key="org.quartz.scheduler.instanceName">scheduler-cluster</prop>
            <prop key="org.quartz.jobStore.driverDelegateClass">org.quartz.impl.jdbcjobstore.StdJDBCDelegate</prop>
            <prop key="org.quartz.jobStore.isClustered">true</prop>
            <prop key="org.quartz.threadPool.threadCount">4</prop>
            <prop key="org.quartz.scheduler.instanceId">AUTO</prop>
            <prop key="org.quartz.scheduler.skipUpdateCheck">true</prop>
        </props>
    </property>
</bean>

'threadCount' is set to 4 because I have more jobs to execute at scheduled time.
I have also annotated my job class with @DisallowConcurrentExecution:

@Component 
@DisallowConcurrentExecution
public class MyJob{
   @Transactional(timeout=900)
   @Scheduled(cron = "0 0 2 * * ?")
   public void execute() {
   ... implementation goes here
   }
}

Job can be started from one of two different server nodes that uses the same database for configuration purposes and data processing. Unfortunatelly my task sometimes starts on both of them. Sample application logs:

NODE 1: task-executor.log.2017-05-16.gz:16/05/2017 02:00:00.068 [org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-3] INFO  com.package.schedule.job.MyJob.execute - MyJob job - start

NODE 2: task-executor.log.2017-05-16.gz:16/05/2017 02:00:00.103 [org.springframework.scheduling.quartz.SchedulerFactoryBean#0_Worker-1] INFO  com.package.schedule.job.MyJob.execute - MyJob job - start

This is unusual situation. Sometimes job starts on both machines, sometimes only on one. I have checked my application logs and didnt find any errors during data processing. Sometimes gap between job start is small – milliseconds, sometimes its seconds. Where should I look for mistake? Am I missing something in configuration?

Best Answer

It could be about clock drift. You need to make sure that the clocks are synchronized between every nodes.

source: http://www.quartz-scheduler.org/documentation/quartz-2.x/configuration/ConfigJDBCJobStoreClustering.html and https://stackoverflow.com/a/12629332/7321097

Related Topic