How to ensure I don't queue repetitive jobs using yii2-queue

[dp778899]

I am working on yii2-queue and it works well. To kick it off, I use yii2-scheduling to run it every minute, along with several other jobs. Both yii2-queue and yii2-scheduling are working. My question is more conceptual as this is the first time I’ve actually used queues to dispatch work vs doing the work synchronously.

Let’s say I have a class that does this:

if: dayOfWeek=3 and db.LAST_RUN < TODAY()
	email.user
	update db.LAST_RUN = TODAY()
end-if

Easy enough! Now, let’s say I need to do this for every user in my app:

for every user in app
	if: dayOfWeek=3 and db.LAST_RUN < TODAY()
		email.user
		update db.LAST_RUN = TODAY()
	end-if
end-for

I need to run this in a loop throughout the day because a user may signup at any time. I never know when. And if they signup today, I need the action done (it’s not actually an email, but it is something that should only happen ONE time per hour/day/whatever).

All that being said, I have a race condition with my code that I need help understanding and solving.

So my queue job code is this:

yii2-scheduling.job (every N minutes)
for every user in app
	if: dayOfWeek=3 and db.LAST_RUN < TODAY()
		send schedule.queue.job to yii2-queue
	end-if
end-for

schedule.queue.job:
	email.user
	update db.LAST_RUN = TODAY()

The race condition will result in an error if schedule.queue.job takes > N minutes to complete because I would queue more than 1 job for that user because I don’t update db.LAST_RUN until I’m done with schedule.queue.job.

My thinking here is I can:

• Update db.LAST_RUN before running the bulk of the code - con: doesn’t actually solve the race condition, just makes it very unlikely; also, if there is an error in execution of the work, I would have incorrectly set this value.
• Lock this specific user in some way so that the job won’t repeat on them.

How would I tell yii2-queue this? Or do I have to program this myself using yii\mutex\Mutex?

Basically, I think I need to do this:

yii2-scheduling.job (every N minutes)
for every user in app
	if: dayOfWeek=3 and db.LAST_RUN < TODAY()
		if: already.working.on.dayOfWeek3.for.this.user
			continue
		else
			send schedule.queue.job to yii2-queue
	end-if
end-for

schedule.queue.job:
	email.user
	update db.LAST_RUN = TODAY()

Is this correct logic for this situation?

If so, is this generally done using yii\mutex\Mutex by hand or should I be doing something with how I setup jobs for yii2-queue… or something else entirely?

[dp778899]

Well, looks like I may have my answer here:

Yii Queue, how can I avoid overlaping

I would like two jobs not to overlap, i.e. only one job can be executed at a time. Is there anything to do this? If not, as far as I can see, I think to establish some kind of “flag” set to 1 when Queue::EVENT_BEFORE_EXEC and to 0 when Queue::EVENT_AFTER_EXEC. Also, how can I set a real listener? a cron just works once each minute, but my jobs can be handled in seconds.

Still would love to have a convo to discuss best path on this!

[dp778899]

Okay, well, that was easy. For anybody with the same need, this is all you need to do to use yii-mutex in a yii2-queue job (example below):

<?php

namespace common\models\jobs;

use Yii;
use yii\base\Model;
use yii\mutex\FileMutex;

use common\models\SchedulerEngine;

class SchedulerEngineJob extends Model implements \yii\queue\JobInterface
{
    public $ruleScheduleID;
    private $mutexParamName = 'SchedulerEngineJobMutex';

    public function execute($queue)
    {
        $mutexName = sprintf("%s.id.%d", Yii::$app->params[$this->mutexParamName], $this->ruleScheduleID);
        d("mutexName=$mutexName");

        $mutex = new FileMutex();

        if ($mutex->acquire($mutexName)) {
            $schedulerEngine = SchedulerEngine::findOne($this->ruleScheduleID);
            dassert(isset($schedulerEngine));

            $schedulerEngine->runRule();

            $mutex->release($mutexName);
        }
        else {
            d("job is locked");
            return;
        }
    }
}