After reading your post, it sounds a lot like the handful of jobs that are not executing are very likely misfiring. The reason that I believe this:
I get log messages that indicate Quartz is creating and executing jobs for roughly one minute after the round of jobs starts.
In Quartz.NET the default misfire threshold is 1 minute. Chances are, you need to examine your logging configuration to determine why those misfire events are not being logged. I bet if you throw open the the floodgates on your logging (ie. set everything to debug, and make sure that you definitely have a logging directive for the Quartz scheduler class), and then rerun your jobs. I'm almost positive that the problem is the misfire events are not showing up in your logs because the logging configuration is lacking something. This is understandable, because logging configuration can get very confusing, very quickly.
Also, in the future, you might want to consult the quartz.net forum on google, since that is where some of the more thorny issues are discussed.
http://groups.google.com/group/quartznet?pli=1
Now, your other question about setting the policy for what the scheduler should do, I can't specifically help you there, but if you read the API docs closely, and also consult the google discussion group, you should be able to easily set the misfire policy flag that suits your needs. I believe that Trigger's have a MisfireInstruction property which you can configure.
Also, I would argue that misfires introduce a lot of "noise" and should be avoided; perhaps bumping up the thread count on your scheduler would be a way to avoid misfires? The other option would be to stagger your job execution into separate/multiple batches.
Good luck!
What you need is the JobChainingJobListener
class, which is there to help you create a chain of execution for your jobs in a specific order you desire..
using System;
using System.Text;
using Quartz;
using Quartz.Impl;
using Quartz.Impl.Calendar;
using Quartz.Listener;
using Quartz.Impl.Matchers;
using System.Threading;
namespace QuartzNET.Samples
{
class Program
{
static void Main(string[] args)
{
// Create RAMJobStore instance
DirectSchedulerFactory.Instance.CreateVolatileScheduler(5);
ISchedulerFactory factory = DirectSchedulerFactory.Instance;
// Get scheduler and add object
IScheduler scheduler = factory.GetScheduler();
StringBuilder history = new StringBuilder("Runtime History: ");
scheduler.Context.Add("History", history);
JobKey firstJobKey = JobKey.Create("FirstJob", "Pipeline");
JobKey secondJobKey = JobKey.Create("SecondJob", "Pipeline");
JobKey thirdJobKey = JobKey.Create("ThirdJob", "Pipeline");
// Create job and trigger
IJobDetail firstJob = JobBuilder.Create<FirstJob>()
.WithIdentity(firstJobKey)
//.StoreDurably(true)
.Build();
IJobDetail secondJob = JobBuilder.Create<SecondJob>()
.WithIdentity(secondJobKey)
.StoreDurably(true)
.Build();
IJobDetail thirdJob = JobBuilder.Create<ThirdJob>()
.WithIdentity(thirdJobKey)
.StoreDurably(true)
.Build();
ITrigger firstJobTrigger = TriggerBuilder.Create()
.WithIdentity("Trigger", "Pipeline")
.WithSimpleSchedule(x => x
.WithMisfireHandlingInstructionFireNow()
.WithIntervalInSeconds(5)
.RepeatForever())
.Build();
JobChainingJobListener listener = new JobChainingJobListener("Pipeline Chain");
listener.AddJobChainLink(firstJobKey, secondJobKey);
listener.AddJobChainLink(secondJobKey, thirdJobKey);
scheduler.ListenerManager.AddJobListener(listener, GroupMatcher<JobKey>.GroupEquals("Pipeline"));
// Run it all in chain
scheduler.Start();
scheduler.ScheduleJob(firstJob, firstJobTrigger);
scheduler.AddJob(secondJob, false, true);
scheduler.AddJob(thirdJob, false, true);
Console.ReadLine();
scheduler.Shutdown();
Console.WriteLine("Scheduler shutdown.");
Console.WriteLine(history);
Console.ReadLine();
}
}
class FirstJob : IJob
{
public void Execute(IJobExecutionContext context)
{
var history = context.Scheduler.Context["History"] as StringBuilder;
history.AppendLine();
history.AppendFormat("First {0}", DateTime.Now);
Console.WriteLine("First {0}", DateTime.Now);
}
}
class SecondJob : IJob
{
public void Execute(IJobExecutionContext context)
{
var history = context.Scheduler.Context["History"] as StringBuilder;
history.AppendLine();
history.AppendFormat("Second {0}", DateTime.Now);
Console.WriteLine("Second {0}", DateTime.Now);
}
}
class ThirdJob : IJob
{
public void Execute(IJobExecutionContext context)
{
var history = context.Scheduler.Context["History"] as StringBuilder;
history.AppendLine();
history.AppendFormat("Third {0}", DateTime.Now);
Console.WriteLine("Third {0}", DateTime.Now);
Console.WriteLine();
}
}
}
Best Answer
For the record, I consider this a design flaw of Quartz. If a job can't be constructed once, that doesn't mean it can't always be constructed. This is a transient error and should be treated as such. Stopping all future scheduled jobs violates the principle of least astonishment.
Anyway, my hack solution is to catch any errors that are the result of my job construction and instead of throwing an error or returning null to return a custom IJob instead that simply logs an error. This isn't perfect, but at least it doesn't prevent future triggering of the job.