Application Scalability Issues

Hi all,

I’m having an issue with testing scalability of my application. I set up methods to add 10,000 records to my database, and that works just fine. However, an issue arises when I try to look at those records in a Grid View.

If there are no conditions on the Data Provider, it returns all the records just fine, but when I apply conditions (i.e. visibility=‘1’), all I get is a blank page. From what I can tell, the Grid View is not loading properly in that scenario.

Is there a way to fix this issue?

Any help is appreciated.

I can’t believe you display 10,000 records in a gridview so you most probably have pagination enabled. Thus, only pageSize records should be fetched from db.

In either case, this sounds more like a problem with the criteria you supply to the data provider. You may want to enable logging and check the generated SQL.

/Tommy

Alright, some more info for you and hopefully it helps:

So I checked with some smaller numbers of data, and from what I can see, it works just fine at ~2500 records, but breaks at ~3000.

I turned on logging to check the SQL, but I can only see the SQL for the query at 2500, as all I get is a blank page when I have 3000. And yes, I have pagination enabled, it is paginating at 20.

Here is the SQL I was able to find from the query at 2500:


"SELECT * FROM `x2_contacts` `t` WHERE visibility='1' ||

assignedTo='Anyone' || assignedTo='Yii::app()->user->getName()' ORDER BY

lastName ASC LIMIT 20"

It looks entirely valid and I’m not sure why adding 500 records would break it. Furthermore, it’s different for different tables. Contacts breaks at 3000, but I have another table called Actions and it doesn’t break until about 4000.

I also started messing around with the view a bit. I turned off the GridView widget, and simply made a call of


$dataProvider=$model->search()

. This worked just fine, but when I tried to run getData() on the dataProvider, it fell apart.

Jake

You can avoid this by using file logging.

Seems like you’ve made Yii::app()->user->getName() a string literal. Remove the quotes and you should see a user name.

When you call getData(), a findAll() call will be executed.

For debugging purposes you can replace getData() with




// define a CDbCriteria here

$data = Your_model::model()->findAll($your_criteria);



/Tommy

Fixed the issue.

I’m not entirely sure what the problem was, but I was extending the Models and calling search from the child class. When the method returns a dataProvider, it was returning a dataProvider for the child model class, and by changing it to the parent class name it ended up working just fine.

Thanks for the help!

Jake

Update, the issue is not actually fixed. By changing the model name it increased the amount of records that will display, but now it caps out at about 9500 for Contacts instead of 3000. That’s a pretty good increase but I’d still like to try to figure out why the issue is happening.

I’ve been tinkering around with it a bit, and I’ve figured out basically what is causing the issue. I get a 500 Internal Server Error when I try to run the findAll command, or when I use the Yii query builder to select everything from the larger table. I don’t know if Yii has an issue with criteria on large numbers of records, and it appears to be what is happening.

If anyone has any input on how to resolve this issue, that would be amazing.

Jake

Maybe some max execution time issues?


Try new extension by samdark, maybe it would make thing clearer

http://www.yiiframework.com/extension/dbprofiler/

You should also turn on PHP’s error logging to find out if you’re missing a PHP error (my suspect would be: max memory exceeded)