Newsletter Memory Issue

Yii 1.1 General Discussion
1

Hi,

I’m using the phpmailer extension to send my newsletters. Work fine, but around number 300 I get a out of memory error. So I use:


ini_set("memory_limit","256M");

But I think it will only be a matter of time before it will be out of memory again, say 1000 users.

Is it normal that the memory usage build up during sending or is this some kind of memory leak? When you send a newsletters, do you need 1GB memory or more?

Any help or information would be appreciated!

Script I use (simplefied):




	class NewsletterCommand extends CConsoleCommand

	{	

		public function actionIndex()

		{

			ini_set("memory_limit","256M");	

				

			echo '------------------------------------'.PHP_EOL;	

			echo 'Start sending newsletter............'.PHP_EOL;			

			echo '------------------------------------'.PHP_EOL;


			$criteria = new CDbCriteria();

			$criteria->addCondition('receive_newsletter = 1');

			$criteria->order = 'id';						

		

			$companies = Company::model()->findAll($criteria);

			

			foreach($companies as $company)

			{

				$user = User::model()->findByPk($company->user_id);									

				echo $user->email.PHP_EOL;;

				

				// Email content.

				$content = array(

					'text'=>'....',	

					'more personal data here'=>'....',				

				);				

				

				// Build email html.				

				$html = $this->render('newsletter',array('content'=>$content),true);

				Yii::app()->csstoinline->setHTML($html);

				Yii::app()->csstoinline->setUseInlineStylesBlock();				

				$html = Yii::app()->csstoinline->convert();

				$text = Yii::app()->functions->createMailAltBody($html);

				

				// Send email.

				try 

				{				

					Yii::app()->mailer->AddAddress($user->email);

					Yii::app()->mailer->Subject = 'Newsletter 21';

					Yii::app()->mailer->Body = $html;

					Yii::app()->mailer->AltBody = $text;

					Yii::app()->mailer->send();

					Yii::app()->mailer->ClearAddresses();

				}

				catch (phpmailerException $e) { echo $e->errorMessage(); } catch (Exception $e) { echo $e->getMessage(); }											

				

				usleep(1000000);

			}

			return 1;

		}


		private function render($template, array $data = array())

		{

		    $path = Yii::getPathOfAlias('application.views.mail').'/'.$template.'.php';

		    if(!file_exists($path)) throw new Exception('Template '.$path.' does not exist.');

		    return $this->renderFile($path, $data, true);

		}

	}
2

Do you have any idea where in the code the memory error is occurring?

Also, does it happen straight away or does it process a bunch of messages first?

3

I find it hard to debug a ‘memory problem’ not sure how to do this. I did not save the ‘Allowed memory size of … bytes exhausted (tried to allocate … bytes)’ so I have to reproduce it.

Sending starts perfect and work as I want, until about the 300th user, that it crashed with the error above. When the script was running, I can see in my controlpanel that it was eating memory every second ::). But I guess the mailserver is using memory too?

4

I can only assume that the object Yii::app()->mailer is continually eating more memory. Instead of that, can you try creating a new mail object each time (within the try block)? Once the reference to the object is changed, it’ll be available for garbage collection rather than lasting for the lifetime of the request.

Edit:

Also, what’s the usleep() for?

5

Does the actual newsletter change for every recipient? Meaning does this section of code change what is sent for each user?


// Build email html.                            

       $html = $this->render('newsletter',array('content'=>$content),true);

       Yii::app()->csstoinline->setHTML($html);

       Yii::app()->csstoinline->setUseInlineStylesBlock();                             

       $html = Yii::app()->csstoinline->convert();

       $text = Yii::app()->functions->createMailAltBody($html);

If it is generating the exact same $html and $text each time, maybe the leak is here. You could move this to above the foreach() loop.

Edit Oops, just went back and noticed the ‘More Personal data’=>… line. nevermind :(

6

Nevermind, thanks for thinking along with me. Indeed the emails vary in usernames, content etc.

Ok, so not use the Yii::app()->mailer bust just the PHPMailer (or whatever class) instead you mean?

I presume this is better to give the server some extra time between each send request instead of dumping it all at once to the smtp server. Is this approach useless?

7

I don’t know what Yii::app()->mailer is, but I was thinking something along the lines of:




   try

   {

       $message = Yii::app()->mailer->createMessage();

       $message->AddAddress($user->email);

       ...

   }

I don’t know how your mailer component is implemented, but it seems like you’re using a single object to represent lots of conceptually distinct objects (messages). If this single object is doing something to update its state on each send, it could be slowly eating more and more memory. This object can’t be garbage collected during the request because there is a constant reference to it in the application class.

It may not be the problem, but it’s where I’d start looking.

8

Indeed, but I think I have a solution. BTW, it’s this extension implementing phpmailer: http://www.yiiframework.com/extension/mailer and it works not bad at all.

But someone else gave me some advice I believe it’s a nice and more stable solution. First get all the emailaddresses and put them in a queue table. Then I run the command an let it send 100 or 200 emails then it ends. The emails in the queue which are send are marked and the next time my cronbjob runs the command, it send the next 200 emails and so on an so fort.

I add some checks and if something happens I write this to my log table. So I keep track of it all and I don’t have to worry that it crash on email nr 1000 or nr 10.000. If it works for 200 emails it will work again and again, right?

Could be a good solution I think.

9

Yes, implementing a queue is a better solution.