Performance considerations when using many modules

I’m tinkering with the idea of a modular CMS, and what would be more natural than using CModule to package modules of add-on functionality for a CMS? :slight_smile:

But I’m concerned about performance.

This CMS would potentially include many small modules, perhaps as many as 50 or 100 in a single application.

I’ve looked through the framework code, and the codebases of individual modules are tied into the autoloader by using Yii::import() - which simply adds paths to the PHP search paths. A nice, simple, clean implementation, which relies on a pre-existing feature in PHP, rather than trying to reinvent the wheel - I like that.

My concern is that, as far as I can tell, at least the base folder on the module will be added to search paths - the module’s base folder, it’s components subfolder and models subfolders.

So at least 3 paths (potentially more) added for every module.

If I add 50 modules, that gives PHP at least 150 subfolders to search to find a given class-file.

That can’t be good for performance - or what?

I suppose I could override the autoloader and implement a cache? Simply reflect on the class-name once loaded, then cache the class-name and path. Should be pretty straightforward.

But is it necessary? Or does PHP already do some kind of caching internally?

If not, would it be interesting to have this feature built into the framework itself?

Appreciate your thoughts :slight_smile:

See here. If you define the class, there’s obviously no auto-loading needed.

I wouldn’t care so much, as there’s always an option left ;). E.g. on my live servers i’ve made good experiences with setting apc.stat to 0. That makes APC cache pretty much all PHP files and never touch the filesystem again. It comes with a minor drawback though: i have to flush the cache, each time i update any PHP file. But that happens not so often on my live system.

I think you misunderstood my question?

Whether you define the class-name or not, there’s the same overhead of autoloading the class. See Yii::autoload() which gets called for any class to autoload, whether the class-name is defined or not…

My concern is not for overhead in Yii as such, but since Yii::import() adds to PHP’s search paths, my concern is about the overhead of having PHP search all of these folders when Yii::autoload() invokes the require() statement.

If you define the class with path alias (eg "application.components.WebUser"), there is no autoloading for this file I think. Because in Yii::import(), getPathOfAlias() is called when full alias is given. and getPathOfAlias() returns full file path.

See here. So instead of importing a directory, the full path of the class file gets cached for later access:




self::$_classes[$className]=$path.'.php';



Then in autoload():




else if(isset(self::$_classes[$className]))

                        include(self::$_classes[$className]);



So there should be no overhead because autoload is NOT searching any directories.

Or did I miss something here?

No, that’s true - but using that strategy requires you to issue a Yii::import() statement pretty much before using any class from any module. Which is tantamount to just simply doing a regular old-fashioned include() or require() before using any class - perhaps slightly better in terms of mapping the class-names to directories, but no more convenient.

Having to manually specify that classes should load, sort of defeats the purpose of having an autoloader, don’t you think? ;)

And there is still the concern that the PHP search path will grow out of control and cause autoloaded classes in third-party code to slow down…

I see, I thought you only mean the xModule.php files :mellow:

But what should be the dependency of the cache? Only problem I see…

It’s a difference: include() and require() (also their _once equivalents) include immediately. Yii::import() doesn’t, which is a good thing.

IMO the drawback you describe is the price you have to pay if you want to do clean OOP in PHP, with a flexible file structure and without having all classes in 1 file. Like i said above you can overcome the situation with a good opcode cache. If that’s not sufficient, then let me ask: what would be your suggestion? I don’t see an alternative here and think the current implementation already does a pretty smart job.

Well, it turns out, my concern was unwarranted - I wrote a benchmark that demonstrates why.





<?php


header('Content-type: text/plain');


define('NUM_PATHS', 200);

define('NUM_CLASSES', 100);


$paths = explode(PATH_SEPARATOR, get_include_path());


// create folders:


for ($i=0; $i<NUM_PATHS; $i++)

{

  $dir = dirname(__FILE__).DIRECTORY_SEPARATOR.$i;

  $paths[] = $dir;

  @mkdir($dir);

}


// create files:


for ($i=0; $i<NUM_CLASSES; $i++)

{

  file_put_contents($dir.DIRECTORY_SEPARATOR."test{$i}.php", "<?php return 'test';\n");

}


// set include path:


set_include_path(implode(PATH_SEPARATOR, $paths));


// benchmark:


$start = microtime(true);


for ($i=0; $i<NUM_CLASSES; $i++)

{

  $test = include $dir.DIRECTORY_SEPARATOR."test{$i}.php";

  if ($test!='test') die('error include file');

}


$end = microtime(true);


echo "Time taken: ".number_format(1000*($end-$start),3)." msec";



You don’t need to run this - it’ll create two hundred folders, and put 100 php scripts in the last folder - then add all 200 paths to PHP’s search paths, the same way Yii does it, then benchmark the time taken for PHP to find and include all 100 scripts.

There is no overhead, except for the first run. For subsequent runs, the time required to search 200 folders for 100 scripts, is the same as to search 1 folder for 100 scripts.

My guess is, once the file is found, PHP records the location where the file may be found - then looks there the first time, before searching the other 199 paths again.

I ran my tests on Windows with Apache 2.2 and PHP 5.3, with no bytecode cache.

The answer to my initial question is, 1 module, 200 modules, it won’t make any difference! :slight_smile: