Why to use?
We use a queue when there are operations that take a large amount of runtime and it should be done in the background without disturbing the UI part.
Eg: Downloading large files, Entering huge amount of data to DBs, Import/Export
Normally when users click on a button to download or export/import large files or to do some other operations/jobs that takes more time, buffering starts as the file execution is in progress and users have to wait on the same page till the file execution is done. They can’t browse other sections.
But if we use queue, these operations/jobs will execute in background and users can browse other sections and they will get an update that the job/operation is done or not using Cron table.
For example users want to download a large excel file. When they click on the button (Button is for this particular action), background job will start.
Setup for Queue Execution
Here we have used Phpstorm Editor.
First we have to install the queue Extension through the terminal.
composer require --prefer-dist yiisoft/yii2-queue
Now we have to paste below code in ‘components’ of ‘web.php’ & ‘console.php’ file in ‘config’ directory (For Db Driver).
'queue' => [
'class' => \yii\queue\db\Queue::class,
'db' => 'db', // DB connection component or its config
'tableName' => '{{%queue}}', // Table name
'channel' => 'default', // Queue channel key
'mutex' => \yii\mutex\MysqlMutex::class, // Mutex used to sync queries
],
Add 'bootstrap' => ['log','queue'], In ‘web.php’ & ‘console.php’ file.
Now we have to choose a driver for storage. Here we have used Db Driver.
Now we have to create a ‘queue’ table in the database to check how many queues are running. When the queue is in running state then we can see the queue details in this ‘queue’ table. After queue execution is completed, details will be removed from the ‘queue’ table.
Command for creating a ‘queue’ table. (My SQL)
CREATE TABLE `queue` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`channel` varchar(255) NOT NULL,
`job` longblob NOT NULL,
`pushed_at` int(11) NOT NULL,
`ttr` int(11) NOT NULL,
`delay` int(11) NOT NULL DEFAULT 0,
`priority` int(11) unsigned NOT NULL DEFAULT 1024,
`reserved_at` int(11) DEFAULT NULL,
`attempt` int(11) DEFAULT NULL,
`done_at` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `channel` (`channel`),
KEY `reserved_at` (`reserved_at`),
KEY `priority` (`priority`)
) ENGINE=InnoDB
Basic example of Queue
First we have to create a file for the queue. We can put this file in any folder.
Here we have created a new Folder with the name ‘Queue’ and created a file with the name ‘QueuePractice.php’.
We want to execute some task in the background. The task is that we want to save 2000 entries of one person in the table in the background.
Before this we have created a table with the name ‘student’. The ‘db’ is connected in ‘db.php’.
For this we have written below code in ‘QueuePractice.php’ file.
<?php
namespace app\queue;
use Yii;
use yii\base\BaseObject;
class QueuePractice extends BaseObject implements \yii\queue\JobInterface
{
public function execute($queue)
{
for ($i=0; $i<=2000; $i++){
echo $i.' Entry Inserted To Student Table'.PHP_EOL;
$sqlQuery=Yii::$app->db->createCommand()->insert('student',[
'Name'=>'Mridula',
'Email'=>'mriduladas@gmail.com',
'PhoneNo'=>'9833377723',
'FileName'=>'mridula.xlsx'
])->execute();
}
}
}
Now you have to create an action in the controller to run this queue.
public function actionQueuePractice(){
Yii::$app->queue->push(new QueuePractice());
}
Now we have to run the ‘actionQueuePractice()’ action through the url.
After hitting this action, a ‘queue job’ will be created. You can check there is an entry in the ‘queue’ table. That particular entry indicates that this job is to be done.
Now run php yii queue/run or ./yii queue/run in the terminal.
After running this command, we can see in the terminal how many tasks in that particular queue have been completed.
Now that particular job is done and there is no entry in the ‘queue’ table. As queue execution is completed, details is removed from the ‘queue’ table and data that we wanted to save is saved to the ‘student’ table.
We can run php yii queue/listen or ./yii queue/listen in the terminal. This command will run in the background. When we run the ‘actionQueuePractice()’ action, ‘queue job’ will automatically execute. We don’t have to run the php yii queue/run command again and again.
Difference between ‘php yii queue/listen’ & ‘php yii queue/run’.
Queue/Listen:
- It is used to process background jobs by listening to a queue.
- It starts a listener process that continuously waits for new jobs to arrive in the queue.
- When a job is added to the queue, the listener picks it up and executes it.
- It is typically used when you have long-running processes or a separate worker process dedicated to executing jobs from the queue.
- It is suitable for scenarios where you want to keep the queue continuously running and process jobs as they arrive.
Queue/Run:
- It is used to manually trigger the execution of a single background job from the command line.
- It allows you to specify the job to be executed, and once the job is completed, the command terminates.
- It is useful for running a specific job on-demand without the need for a continuous listener process.
- It is suitable for scenarios where you want to execute a specific job immediately without relying on the queue listener.
Track The Status Of Queue using Cron table
Now we want to save the queue status in a different table. We have used the same files with some modifications in code.
For this we have to create a table in the same database. Here the table name is ‘queue_data_track’. Here we use ‘queue_data_track’ table as Cron table.
Thereafter we have to create a model with the table name ‘queue_data_track’. Here we have created a model ‘QueueDataTrack’ using gii.
Controller Action
public function actionQueuePractice(){
$model= new QueueDataTrack();
$model->Status="Pending";
$model->Description="Job is Executing";
if ($model->save()){
Yii::$app->queue->push(new QueuePractice([
'id'=>$model->Id
]));
echo 'Job Done';
}
else{
echo '<pre>';
print_r($model->getErrors());
return "Unable to create a Job";
}
In this action we have created an instance of the ‘QueueDataTrack’ model and defined ‘Status’ as ‘Pending’ and ‘Description’ as ‘Job is Executing’. Then we have written a code for the queue and passed Model Id as ‘id’ parameter.
‘QueuePractice.php’ file for queue.
<?php
namespace app\queue;
use app\models\QueueDataTrack;
use Yii;
use yii\base\BaseObject;
class QueuePractice extends BaseObject implements \yii\queue\JobInterface
{
public $id;
public function execute($queue)
{
for ($i=0; $i<=2; $i++){
echo $i.' Entry Inserted To Student Table'.PHP_EOL;
$sqlQuery=Yii::$app->db->createCommand()->insert('student',[
'Name'=>'Mridula',
'Email'=>'mriduladas@gmail.com',
'PhoneNo'=>'9833377723',
'FileName'=>'mridula.xlsx'
])->execute();
}
$model= QueueDataTrack::findOne($this->id);
if ($model){
$model->Status='Success';
$model->Description='Job is Completed';
$model->save();
}
else{
echo 'Error';
}
}
}
When we run the ‘actionQueuePractice()’ action through the url, ‘queue job’ will be created. There is no data in the ‘student’ table as the ‘queue job’ is created but not executed.
Now run php yii queue/run or ./yii queue/run in the terminal. ‘Queue job’ will be executed.
There is no entry in the ‘queue’ table as ‘queue job’ is executed.