Wrong cache dependency when using joins

pavlepredic · May 9, 2012, 10:07am

This one is a little tricky to explain, so I created a sample project and will explain using that example. In a nutshell, the issue here is that queries are saved to cache with the wrong cache dependency when (and only when) using a ‘with’ clause in criteria.

The project contains a simple db with a couple of book names (table ‘book’) and their respective authors (table ‘author’). There are only two models: Book and Author. There is also a simple controller that creates a CActiveDataProvider instance for Book and specifies a ‘with’ relation (‘author’) in criteria.

Here’s how to create the sample project:

Start with a new yii webapp.
Set up CFileCache in config (or use the attached main.php config file)
Import the attached db schema (MySQL) and set up yii to use this db.
Set up query caching and use extremely high values, eg:




'queryCachingDuration' => 365 * 86400,

'queryCachingCount' => PHP_INT_MAX,

Create models for the two tables using gii (or use the attached files Book.php and Author.php).
Override getDbConnection() in both models, like so:




public function getDbConnection()

{

	$db = parent::getDbConnection();

	$db->queryCachingDependency = new CGlobalStateCacheDependency(__CLASS__ . 'CacheDependency');

	return $db;

}

(The purpose of this method is to create the appropriate cache dependency for each model. It will create an ‘AuthorCacheDependency’ for Author and ‘BookCacheDependency’ for Book.)

Create an action that instantiates the data provider and renders the view (see attached SiteController):




public function actionIndex()

{

	$dataProvider = new CActiveDataProvider('Book', array(

		'criteria' => array(

			'with' => 'author'

		)

	));

	$this->render('index', array('dataProvider' => $dataProvider));

}

All you need in the view file is this:




$this->widget('zii.widgets.grid.CGridView', array(

	'dataProvider' => $dataProvider,

));

That’s it, the project is ready. Now, to reproduce the bug, simply open the page in the browser and then check the cached data in runtime/cache. One of the cache files will contain the following (I removed some non-printable characters):




a:2:{i:0;a:5:{i:0;a:5:{s:5:"t0_c0";s:1:"1";s:5:"t0_c1";s:1:"1";s:5:"t0_c2";s:23:"The Catcher in the Rye ";s:5:"t1_c0";s:1:"1";s:5:"t1_c1";s:14:"J. D. Salinger";}i:1;a:5:{s:5:"t0_c0";s:1:"2";s:5:"t0_c1";s:1:"1";s:5:"t0_c2";s:16:"Franny and Zooey";s:5:"t1_c0";s:1:"1";s:5:"t1_c1";s:14:"J. D. Salinger";}i:2;a:5:{s:5:"t0_c0";s:1:"3";s:5:"t0_c1";s:1:"2";s:5:"t0_c2";s:12:"The Stranger";s:5:"t1_c0";s:1:"2";s:5:"t1_c1";s:12:"Albert Camus";}i:3;a:5:{s:5:"t0_c0";s:1:"4";s:5:"t0_c1";s:1:"3";s:5:"t0_c2";s:20:"Crime and Punishment";s:5:"t1_c0";s:1:"3";s:5:"t1_c1";s:18:"Fyodor Dostoyevsky";}i:4;a:5:{s:5:"t0_c0";s:1:"5";s:5:"t0_c1";s:1:"3";s:5:"t0_c2";s:22:"Notes from Underground";s:5:"t1_c0";s:1:"3";s:5:"t1_c1";s:18:"Fyodor Dostoyevsky";}}i:1;O:27:"CGlobalStateCacheDependency":4:{s:9:"stateName";s:21:"AuthorCacheDependency";s:23:"CCacheDependency_data";N;s:14:"CComponent_e";N;s:14:"CComponent_m";N;}}

As you can see (scroll to the right), the dependency for this cache entry is ‘AuthorCacheDependency’, and it should be (I believe) ‘BookCacheDependency’, since we are querying Book model and only joining Author model. If you clear the cache and retry without specifying the ‘with’ relation, the correct dependency is used. The upshot is that you can’t rely on this dependency to invalidate cached data.

Hope I managed to explain it properly.

Cheers!

Haensel · May 9, 2012, 1:12pm

Ok, just out of curiosity I tried to find out what could be the issue here and I think I was able to condense it down to a line in CActiveFinder (used to do the active record joins) that caught my interest: http://code.google.com/p/yii/source/browse/tags/1.1.10/framework/db/ar/CActiveFinder.php#219

It basically uses


CActiveRecord::model($withClassName);

The point is that the model() method should create a new CActiveRecordMetaData object for the modelclass that was defined in the "with" part of your criteria. So this metaData object uses the dbConnection of the author model object now and not the initial book object to get the table schema => redefining the cache dependency. I think you could be able to solve this issue by not putting the cacheDependency into your getDbConnection method but rather into init() like


public function init()

{

    $db=this->getDbConnection();

    $db->queryCachingDependency = new CGlobalStateCacheDependency(__CLASS__ . 'CacheDependency');

}

Maybe that’s the issue?

However: Brainfuck

Greetings,

Hannes

pavlepredic · May 9, 2012, 2:05pm

Haensel,

Thanks a lot for your ideas. However, putting the logic in init() doesn’t seem to work. When I do that, no cache dependency is used. I also tried putting it into afterConstruct() with the same effect.

The more I think about this issue, the more I start to realize that when you use caching on join queries, you need to also merge dependencies in some way. In the example above I said that yii should use ‘BookCacheDependency’ and not ‘AuthorCacheDependency’, but the truth is that it should use both. That way, if either dependency is changed, cache entry will be invalidated. I hope I’m making sense

Haensel · May 9, 2012, 2:49pm

I think you should only use caching in combination with controller code, not in your models. It is a lot easier to mess things up otherwise. Have you tried using the cache() method in combination with your dataprovider?




$dependency=new CGlobalStateCacheDependency('BookCacheDependency');

$dataProvider=new CActiveDataProvider(Book::model()->cache(365 * 86400, $dependency, 2),array(

                'criteria' => array(

                        'with' => 'author'

                ))

The 2 is used to cache 2 queries: One for counting the results and one for the actual query

pavlepredic · May 9, 2012, 3:22pm

What I’m trying to achieve is to have certain models that don’t change often and that are not large to always be cached. Here’s an active record implementation that I’m using in my project (all the models that need to be cached are descended from it):




<?php

/**

 * ActiveRecord that uses query caching and a simple 

 * cache dependency, that invalidates cache data every time

 * an insert or update is performed

 */

abstract class ECachingActiveRecord extends CActiveRecord

{

	public static $cachedDb;

	public $useCache = true;

	

	public function getDbConnection()

	{

		if (!$this->useCache)

			return parent::getDbConnection();

		

		if (!self::$cachedDb)

		{

			$config = $this->getConfigOptions(Yii::app()->db);

			//set cache duration and count to extremely high values - we will rely on CacheDependency

			//to invalidate cache each time an insert or update is performed

			$config['queryCachingDuration'] = 365 * 86400;

			$config['queryCachingCount'] = PHP_INT_MAX;

			self::$cachedDb = Yii::createComponent($config);

		}

		

		self::$cachedDb->queryCachingDependency = new CGlobalStateCacheDependency($this->getGlobalStateKey());

		

		return self::$cachedDb;	

	}

	

	public function afterSave()

	{

		parent::afterSave();

		if ($this->useCache)

			$this->updateCacheDependency();

	}


	public function afterDelete()

	{

		parent::afterDelete();

		if ($this->useCache)

			$this->updateCacheDependency();

	}

	

	public function updateCacheDependency()

	{

		Yii::app()->setGlobalState($this->getGlobalStateKey(), microtime(true));

		Yii::app()->saveGlobalState(); //it's safer to save immediatelly

	}

	

	protected function getConfigOptions($component)

	{

		$config = array('class' => get_class($component));

		$rc = new ReflectionClass($component);

		foreach ($rc->getProperties(ReflectionProperty::IS_PUBLIC) as $prop)

			$config[$prop->getName()] = $prop->getValue($component);

		return $config;

	}

	

	protected function getGlobalStateKey()

	{

		return get_class($this) . 'CacheDependency';

	}

}

Any model that is descended from this class will automatically be cached and whenever an update or delete is performed, cache will be invalidated.