Saas Multi-Tenant Separate Db

Okay, I think I will try to have best of both worlds:

I first wanted to have a main db with a table containing all the tenants. Then I have a separate db for each tenant with their tenant-id forming part of their db’s name.

Now, I think I will change it to this:

I still have a main db with a table containing all the tenants.

Then I have a single secondary db which stores all the tenants’ data. This secondary db thus contains multiple tenants. IF a tenant needs to be moved to its own db, then I create another secondary db for that tenant.

The main db will store the name of the secondary db to use for each tenant.

So each secondary db can have just one or multiple tenants.

This way, you can keep all tenants in one db, until you need the ability to separate some of them. You also have full control over the number of secondary databases created.

Gerhard,

Did you manage to build a production multi tenant site on this hybrid architecture. I really like your idea. I’m starting to build a SaaS myself and I’m thinking on mixing this method and code, with the DB structure that you proposed. Would you mind sharing your experience and lessons learned? I would really appreciate it.

Read this wiki, it might help you: A Multi-Tenant Strategy Using Yii and MySQL. Also read the linked articles. Whoever recommends a single database per tenant has not had to handle hundreds or thousands of them.

As for unique invoice numbers, that could be a separate DB field from the primary key so it can be handled easily.

Individual tenant DB backups require a some thought but they can be managed through DBMS choice, partitioning, etc. Certainly there is no free lunch so YMMV.

Good luck.

@Yomer

I halted the ‘hybrid architecture’ idea for a while when I found out that in Yii 1, you can not retrieve data from separate databases via SQL, because sql statements use only one connection string. You can thus only do it with AR, because each model can use a different connection string. (Or you have to merge separate sql results.)

This is a problem for me, because I have reports that tunnel through 4 or 5 tables producing thousands of pages. In these reports, AR could hammer the memory.

But I see that Yii 2 has the ability to retrieve multiple records via AR, without creating a huge new model for each record - which should sort out the memory problem. I don’t know if Yii 2 can use single sql statements to multiple DBs.

So I will soon try that ‘hybrid architecture’ again.

@JFReyes

Your mentioning of ‘Individual tenant DB backups’ should be an interesting idea. Being able to “restore a tenant’s backup” without influencing other tenants is definitely one of the major reasons I want to have multiple DBs. But if you are able to restore only one tenant’s data, then having a single DB might be an option.

Thanx for the link to that new wiki that both you guys included.

Gerhard,

I was actually using JFReyes' article to do my multi-tenant architecture. When I read your idea about a hybrid arch., I started to wonder if I could modify JFReyes' solution to allow his last bit of code in the article(The Behavior), to change the DB connection string upon reading a specila "flag" field in the tenant table. 

@JFReyes, @Gerhard

Wouldn't it be possible to change the AppStartup Behavior's beginRequest method from this:

public function beginRequest()

    {

        // switch db credentials for logged in users

        if (!Yii::app()->user->isGuest) {

            $u = TUser::model()->findByPk(Yii::app()->user->id);

            $tu = TTenant::model()->findByPk($u->tenant_id)->dbu;

            $tp = TTenant::model()->findByPk($u->tenant_id)->e_dbpwd;

            Yii::app()->db->setActive(false);

            Yii::app()->db->username = $tu;

            Yii::app()->db->password = $tp;

            Yii::app()->db->setActive(true);

        }

    }

To this:


public function beginRequest()

    {

        // switch db credentials for logged in users

        if (!Yii::app()->user->isGuest) {

            $u = TUser::model()->findByPk(Yii::app()->user->id);

            $tu = TTenant::model()->findByPk($u->tenant_id)->dbu;

            $tp = TTenant::model()->findByPk($u->tenant_id)->e_dbpwd;

            $odb = TTenant::model()->findByPk($u->tenant_id)->own_db; //Boolean or Tinyint if Tenant uses own DB. 

            Yii::app()->db->setActive(false);

            Yii::app()->db->username = $tu;

            Yii::app()->db->password = $tp;

            if ($odb === 1) {

               Yii::app()->sb->connectionString = 'mysql:host=localhost;dbname=db_' . $tu;

            }

            Yii::app()->db->setActive(true);

        }

    }

This way, if the tenant has paid or asks for a separate database, and using db_[tenantuser] as the name of the tenant’s own database. The connection will be made to store the data to another database. Since in my case, each of the tenants’s write actions would only need to access one database at a time. Be it its own or the shared one if he doesn’t have the “own_db” flag active.

Anyway, I’m still configuring the Controllers, this is an idea I’m planning to implement to maybe get hte Hybrid DB theory working. What do you guys think?

Hi Yomer

There should not be any differences in controllers used for single DBs and controllers used for multiple DBs. The only differences should be in the models. To do this, check out this wiki and my comment here.

Regards

It seems complicated without having to change app code as well. In your scenario you would need to duplicate both the tenant and user tables in every DB, albeit having only one tenant in the tenant table and few users in the user table for each particular tenant. How will you keep their primary keys in sync? For example, when a new user is added to the user table how would you ensure that its primary key will match its counterpart in the master (commingled) DB so it can be read upon app startup? I’m not sure how to tackle that.

You’re spot on about the keys, I guess I didn’t think that through.

How about this. In my case, each tenant has a different subdomain/path, so upon connection, the app relates this starting point to the tenant username. Then it asks through the default DB, if it needs to connect to the same default DB or an independent one. When the tenant’s client authenticates, it does to its corresponding DB. And when a new user is added, it’s added only in the tenant’s DB, since user information will only exist in that DB, there’s no need to have a counterpart in the master DB. For Tenants and their clients, this architecture should work out well; the problem would present itself for the app-staff/automated-registration-process when a new tenant needs to be created; the tenant info goes into the master DB and the user account into the master or independent DB, depending on the registered working/connection mode. After tenant registration, depending on the working mode, all data connections should fall through to the correct destination.

The only primary/foreign key relation that would be broken is the tenant_id in the tenant and user tables. All other table’s key relations would remain intact, since tables don’t share data between DBs(except the user table). The solution would be to emulate the tenant_id key relation through the app, or use only the tenant dbu to link the tenant table to the corresponding user table, be it in the master or independent DB.

Does all this sound coherent?

I should maybe leave this hybrid DB architecture out for now and concentrate on building the app with a single shared database, and later try another solution like the one I mentioned above(if it works).

That is what I am doing at the moment. I think I will try this when I split the DB later:

All users are stored in the master DB. So any internet user can create a profile in the master DB.

However, users need to be “invited” by the tenant’s “user-manager” to access the tenant’s DB (which is in a joined DB or separate DB). When a user wants to access a tenant’s DB, he will have to supply the tenant’s code as well when he logs in, which will then give him access to the right tenant DB.

This allows you to build in more security features and a tenant’s “user manager” has full control over who is allowed in the DB and what they are allowed to do.

I’m sorry but I’ve read your post a few times and am still confused. The problem is that the Yii application lifecycle always resets the db connection and urlManager rules to the configuration default when it initializes the app at every request. The tenant client corresponding DB isn’t accesible at this time. You would need to set all of this up during onBeginRequest.

@yomer and @Gerhard:

If I were to handle separate DB’s for each tenant (and will soon have to because of an upcoming project) I would use 2 db connections simultaneously. db1 would contain the tenant and user data while db2 (identified in db1’s tenant) would contain the rest of the application data for each tenant. In onBeginRequest I would only change the db2 connection. I believe in MySQL you can have foreign keys to a different database but if not it gets messy; I haven’t gone that far yet. Models and controllers have to be very careful about which database they’re talking to.

@yomer:

To answer your questions about the wiki article sent via PM…

  1. In the controllers, the actionUpdate method is actually simpler than actionCreate because the tenant info is already there:

UserController.php




public function actionUpdate($id)

{

	$model = $this->loadModel($id); // returns TUser or VUser model

	// Uncomment the following line if AJAX validation is needed

	$this->performAjaxValidation($model);


	if(Yii::app()->user->isUserTenantAdmin() && isset($_POST['VUser']))

		$postVars = $_POST['VUser'];

	elseif (Yii::app()->user->isUserAppStaff() && isset($_POST['TUser']))

		$postVars = $_POST['TUser'];


	if(isset($postVars)) {

		$model->attributes = $postVars;

		if($model->save())

			$this->redirect(array('view','id'=>$model->id));

	}


	$this->render('update',array(

		'model'=>$model,

	));

}



actionView and actionDelete are as per Gii default which BTW is how I always start developing the CRUD functionality.

  1. Other non-tenant/user controllers are similar in this respect:

a) for create, they first check if the logged in user is a tenant or app staff and instantiate the appropriate new VModel (MySQL view based) or TModel (table based) model. If table based they populate the tenant_dbu column according to the _form view’s selected tenant; otherwise they populate the tenant_id column according to the tenant to which the session’s user belongs.

b ) for updates it’s similar to #1 above.

c) as mentioned before, view and delete actions are as per Gii default.

Hope this helps.

I dug up some old code that I plaid with before. It should give you a start.

My config file forces all requests through onBeginRequest via applicationBehavior and it also

connects both db1 and db2 to the same database. applicationBehavior will then change the connection string of db2.




<?php


return array(

	'basePath'=>dirname(__FILE__).DIRECTORY_SEPARATOR.'..',

	'name'=>'InterChest Commercial Applications',


	'import'=>array(

		'application.models.*',

		'application.components.*',

		'ext.yii-mail.YiiMailMessage',

	),


	// Force all requests through applicationBehavior

	'behaviors' => array(

		'class' => 'application.components.applicationBehavior', 

	),


	// application components

	'components'=>array(

		...	

	

		/* Set db1 connection */

		'db1'=>array(

			'connectionString' => 'mysql:host=localhost;dbname=db1',

			'emulatePrepare' => true,

			'username' => 'root',

			'password' => '',

			'charset' => 'utf8',

			'tablePrefix' => 'cap_',

		),


		/* Set db2 connection. Use db1 as default so that exceptions

			occur if db2 is not set correctly in applicationBehavior (if you use db2

			as default here, then everything looks okay and errors are not picked up.) */

        'db2' => array(

            'connectionString' => 'mysql:host=localhost;dbname=db1',

			'emulatePrepare' => true,

			'username' => 'root',

			'password' => '',

			'charset' => 'utf8',

			'tablePrefix' => 'cap_',

			'class' => 'CDbConnection',

        ),

		...

);



Here is the behavior. It simply generates a new user-specific connection string for db2.




class applicationBehavior extends CBehavior 

{       

    public function events(){ 

 		return array(

			'onBeginRequest'=>'assignBranchDb',

		);

    }

 

    public function assignBranchDb() 

    {

		$owner=$this->getOwner(); 

		

		if(!strstr($_SERVER['REQUEST_URI'],"captcha")) /*ignore requests involving the captcha*/

		{

			if(!$owner->user->isGuest && $owner->user->hasState('branchID'))

			{ 

				$dsn = 'mysql:host=' . Yii::app()->user->getState('dbHost') . ';dbname='.Yii::app()->user->getState('dbName');

				$username = Yii::app()->db->username;

				$password = Yii::app()->db->password;


				$owner->db2->setActive(FALSE);

				$component=Yii::createComponent(array(

	                	'class'=>'CDbConnection',

						'tablePrefix' => 'cap_',

	                	'connectionString' => $dsn, 

	                	'emulatePrepare' => true, 

	                	'username' => $username,

	                	'password' => $password,

	                	'charset' => 'utf8', 

	                	'enableParamLogging'=>true, 

	                //	'schemaCachingDuration'=>3600,

	            )); 

	            $owner->setComponent('db2',$component);

				$owner->db2->setActive(TRUE);

			}

		}

	}

}



All models must have this code to add their relevant database names to queries. See the link in my post #13 above.




/* New tableName function */

public function tableName()

{

	preg_match("/dbname=([^;]+)/i", $this->dbConnection->connectionString, $matches);

	return $matches[1].'.{{tablename}}';

}

	

/* Override the parent getDbConnection() */

public function getDbConnection()

{

    return self::getDbtenantConnection();

}



@JFReyes

It does help a lot.

Just to clear your point 2.a. The triggers in every table force the tenant_dbu

when inserting a new record, but when the AppStaff inserts a record, it skips the trigger, because he would have control over which tenant_dbu and tenant_id to assign to that record. Is this right? So every time a record is inserted a tenant_dbu and tenant_id will always be populated either manually by the AppStaff or automatically by the TenandAdmin and Clients’ actions.

About my overly confusing post…

You mention that by design the tenant client DB wouldn’t be accesible. What I was thinking was that if you already have the tenant_dbu and tenant_id in session. On the beginRequest, if I understand correctly, you can change the connectionString on the fly.


[s]$u = TUser::model()->findByPk(Yii::app()->user->id);[/s]

$tu = TTenant::model()->findByPk(Yii::app()->session['tenant_id'])->dbu;

$tp = TTenant::model()->findByPk(Yii::app()->session['tenant_id'])->e_dbpwd;

$odb = TTenant::model()->findByPk(Yii::app()->session['tenant_id'])->own_db; //Boolean or Tinyint if Tenant uses own DB. 

In this first lines, could I remove the requirement of getting the User model if I have a tenant_id already stored in session? Then it would go get the tenant’s dbu and dbpwd from the default master db which has the tenant table. Then…


Yii::app()->db->setActive(false);

Yii::app()->db->username = $tu;

Yii::app()->db->password = $tp;

if ($odb === 1) {

   Yii::app()->sb->connectionString = 'mysql:host=localhost;dbname=db_' . $tu;

}

Yii::app()->db->setActive(true);

If the tenant has his own DB, then the connectionString changes to go to look for DB: db_[dbusername]. The credentials are the same, what changes is where are the credentials for the tenant going to be valid, wither in the master db or the independent db.

When the connection is made to either of those, the session will already have the tenant_id to write to the tables and populate. Can the primary/foreign key relationship between all tables and the tenant table be omitted in favor of using the current user’s stored tenant_id and tenant_dbu to view and modify the tables?

I’m probably oversimplifying things. I’m not implying all this as an authority in Yii, since I’m far from it. If it turns out that this method isn’t possible or a safe, I’m expecting the community to correct me and learn a bit more about Yii.

Thanks for the help.

Yes.

Yes but I would be concerned about security as sessions can be hacked into, but there are ways to protect against it.

I wouldn’t trust the app code to maintain referential integrity in the database, there are too many ways that can go wrong.

After preparing some models, I’m trying to create a Tenant and a User. For that, I allowed for anyone to create in the rules section. But I’m getting a Property “CStringValidator.0” is not defined. error when I try to create either the Tenant or the User. I thought the error was in the model, but I did everything accordingly only to get this error.

What am I doing wrong?

Need to see your code and error message. Please post it.

Sorry, here’s the code. Let me know if you need any other code snippet. I really appreciate it.

_form.php:


<?php

/* @var $this TenantController */

/* @var $model TTenant */

/* @var $form CActiveForm */

?>


<div class="form">


<?php $form=$this->beginWidget('CActiveForm', array(

	'id'=>'ttenant-form',

	// Please note: When you enable ajax validation, make sure the corresponding

	// controller action is handling ajax validation correctly.

	// There is a call to performAjaxValidation() commented in generated controller code.

	// See class documentation of CActiveForm for details on this.

	'enableAjaxValidation'=>false,

)); ?>


	<p class="note">Fields with <span class="required">*</span> are required.</p>


	<?php echo $form->errorSummary($model); ?>




	<div class="row">

		<?php echo $form->labelEx($model,'dbu'); ?>

		<?php echo $form->textField($model,'dbu',array('size'=>16,'maxlength'=>16)); ?>

		<?php echo $form->error($model,'dbu'); ?>

	</div>


	<div class="row">

		<?php echo $form->labelEx($model,'e_dbpwd'); ?>

		<?php echo $form->textField($model,'e_dbpwd',array('size'=>60,'maxlength'=>1024)); ?>

		<?php echo $form->error($model,'e_dbpwd'); ?>

	</div>


	<div class="row">

		<?php echo $form->labelEx($model,'business_name'); ?>

		<?php echo $form->textField($model,'business_name',array('size'=>60,'maxlength'=>128)); ?>

		<?php echo $form->error($model,'business_name'); ?>

	</div>


	<div class="row">

		<?php echo $form->labelEx($model,'is_dedicated'); ?>

		<?php echo $form->textField($model,'is_dedicated'); ?>

		<?php echo $form->error($model,'is_dedicated'); ?>

	</div>


	<div class="row">

		<?php echo $form->labelEx($model,'member_type'); ?>

		<?php echo $form->textField($model,'member_type'); ?>

		<?php echo $form->error($model,'member_type'); ?>

	</div>


	<div class="row buttons">

		<?php echo CHtml::submitButton($model->isNewRecord ? 'Create' : 'Save'); ?>

	</div>


<?php $this->endWidget(); ?>


</div><!-- form -->

TTenant.php




<?php


/**

 * This is the model class for table "tenant".

 *

 * The followings are the available columns in table 'tenant':

 * @property string $id

 * @property string $dbu

 * @property string $e_dbpwd

 * @property string $business_name

 * @property integer $is_dedicated

 * @property integer $member_type

 *

 * The followings are the available model relations:

 * @property User[] $user

 */


class TTenant extends CActiveRecord

{

	/**

	 * @return string the associated database table name

	 */

	public function tableName()

	{

		return 'tenant';

	}


	/**

	 * @return array validation rules for model attributes.

	 */

	public function rules()

	{

		// NOTE: you should only define rules for those attributes that

		// will receive user inputs.

		return array(

			array('dbu, e_dbpwd, business_name', 'required'),

			array('is_dedicated, member_type', 'numerical', 'integerOnly'=>true),

			//array('id', 'length', 'max'=>10),

			array('dbu', 'length', 'max'=>16, 'safe'),

			array('e_dbpwd', 'length', 'max'=>1024, 'safe'),

			array('business_name', 'length', 'max'=>128),

			// The following rule is used by search().

			// @todo Please remove those attributes that should not be searched.

			array('id, dbu, e_dbpwd, business_name, is_dedicated, member_type', 'safe', 'on'=>'search'),

		);

	}


	/**

	 * @return array relational rules.

	 */

	public function relations()

	{

		// NOTE: you may need to adjust the relation name and the related

		// class name for the relations automatically generated below.

		return array(			

			'user' => array(self::HAS_MANY, 'TUser', 'tenant_id')

		);

	}

        

        public function beforeSave()

        {

            if ($this->isNewRecord) {

                Common::createMySQLUser($this->dbu,$this->e_dbpwd);

            }

            return parent::beforeSave();

        }

        public function getListOfAllTenants() // used only by app staff to assign users to tenants

        {

            $criteria = new CDbCriteria(array(

                'select'=>'id, business_name',

                'order'=>'business_name ASC',

            ));

            $listOfAllTenants=CHtml::listData($this->findAll($criteria), 'id', 'business_name');

            return $listOfAllTenants;

        }


	/**

	 * @return array customized attribute labels (name=>label)

	 */

	public function attributeLabels()

	{

		return array(

			'id' => 'Primary key',

			'dbu' => 'Usuario BD',

			'e_dbpwd' => 'Contraseña BD',

			'business_name' => 'Nombre negocio',

			'is_dedicated' => 'BD dedicada',

			'member_type' => 'Tipo miembro',

		);

	}


	/**

	 * Retrieves a list of models based on the current search/filter conditions.

	 *

	 * Typical usecase:

	 * - Initialize the model fields with values from filter form.

	 * - Execute this method to get CActiveDataProvider instance which will filter

	 * models according to data in model fields.

	 * - Pass data provider to CGridView, CListView or any similar widget.

	 *

	 * @return CActiveDataProvider the data provider that can return the models

	 * based on the search/filter conditions.

	 */

	public function search()

	{

		// @todo Please modify the following code to remove attributes that should not be searched.


		$criteria=new CDbCriteria;


		$criteria->compare('id',$this->id,true);

		$criteria->compare('dbu',$this->dbu,true);

		$criteria->compare('e_dbpwd',$this->e_dbpwd,true);

		$criteria->compare('business_name',$this->business_name,true);

		$criteria->compare('is_dedicated',$this->is_dedicated);

		$criteria->compare('member_type',$this->member_type);


		return new CActiveDataProvider($this, array(

			'criteria'=>$criteria,

		));

	}


	/**

	 * Returns the static model of the specified AR class.

	 * Please note that you should have this exact method in all your CActiveRecord descendants!

	 * @param string $className active record class name.

	 * @return TTenant the static model class

	 */

	public static function model($className=__CLASS__)

	{

		return parent::model($className);

	}

}

create.php


<?php

/* @var $this TenantController */

/* @var $model TTenant */


$this->breadcrumbs=array(

	'Ttenants'=>array('index'),

	'Create',

);


$this->menu=array(

	array('label'=>'List TTenant', 'url'=>array('index')),

	array('label'=>'Manage TTenant', 'url'=>array('admin')),

);

?>


<h1>Create TTenant</h1>


<?php $this->renderPartial('_form', array('model'=>$model)); ?>

By default Yii assumes "id" as an autoincremented integer primary key unless you override the primaryKey() function in the model. It appears that you defined "id" as a string. This will not work. In fact for every table where you use a surrogate (instead of a natural) primary key it should be integer, not null and autoincremented. Follow the example in the wiki article.

I’m actually using a natural primary key on my tenant table(and every other table), and it’s set as int, NN and unsigned. I followed every step of the wiki, with the only difference that I added two more columns to the tenant model. Should I just delete any reference to the id on my models?

I’m attaching the actual error I’m getting… and here’s my table declaration

CREATE TABLE tenant (

id int(10) unsigned NOT NULL COMMENT ‘Primary key’,

dbu varchar(16) COLLATE utf8_unicode_ci NOT NULL COMMENT ‘Usuario BD’,

e_dbpwd varbinary(1024) NOT NULL COMMENT ‘Contraseña BD’,

business_name varchar(128) COLLATE utf8_unicode_ci DEFAULT NULL COMMENT ‘Business name’,

is_dedicated tinyint(1) unsigned DEFAULT ‘0’ COMMENT ‘BD dedicada’,

member_type smallint(5) unsigned DEFAULT ‘0’ COMMENT ‘Tipo miembro’,

PRIMARY KEY (id)

) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

One more thing I did. I generated both TTenant and VTenant models per the article, and then I created the view, the controller and CRUD using the TTenant as the model class. I didn’t catch that the id in the model was generated as a string until you mentioned it. The weird thing is that the error when trying to create a new tenant. Can’t I just change the property to int, or do I have to try and generate the model code again with Gii’s CRUD Generator?

If you’re using MySQL you’re missing the AUTO_INCREMENT directive for the “id” column. As for the error message, it’s related to “dbu”:




 array('dbu', 'length', 'max'=>16, 'safe'),



You cannot mix validators in the same statement so ‘length’ and ‘safe’ shouldn’t be combined. I didn’t catch it initially but the error message makes it very clear:




C:\wamp\yii-1-1-14\framework\base\CModel.php(287): CValidator::createValidator("length", TTenant, "dbu", array("max" => 16, 0 => "safe"))



Fix those and you should be alright.