Yii2 - OCI schema performance and Lowercase issue

Hi,

In Yii2 vs Yii1 I was confrunting with this issues:

  • oci db reading schema is slower

  • tables/columns in oracle database is mostly set to insensitive case. eg: select "id" from "yi_user" results to table not found if table is not created with create table "yi_user"; and same logic go to columns definition.

so, my solution was to modify sql from findColumns method in Oci/schema and alter pdo attributes. (I’m using basic template)

My steps:

  1. create components folder in root app;

  2. create in commponents folter new class ESchemaOci from extending yii\db\oci\Schema. Full file below, but i’m pointing here the modifications:

removed join on all_col_comments for getting more speed:


protected function findColumns($table)

    {

        //$schemaName = $table->schemaName;

        //$tableName = $table->name;

		$schemaName = strtoupper($table->schemaName);

		$tableName = strtoupper($table->name);

$sql = <<<EOD

SELECT a.column_name, a.data_type ||

    case

        when data_precision is not null

            then '(' || a.data_precision ||

                    case when a.data_scale > 0 then ',' || a.data_scale else '' end

                || ')'

        when data_type = 'DATE' then ''

        when data_type = 'NUMBER' then ''

        else '(' || to_char(a.data_length) || ')'

    end as data_type,

    a.nullable, a.data_default,

    (   SELECT D.constraint_type

        FROM ALL_CONS_COLUMNS C

        inner join ALL_constraints D on D.OWNER = C.OWNER and D.constraint_name = C.constraint_name

        WHERE C.OWNER = B.OWNER

           and C.table_name = B.object_name

           and C.column_name = A.column_name

           and D.constraint_type = 'P') as Key,

    --com.comments as column_comment

           '' as column_comment 

FROM ALL_TAB_COLUMNS A

inner join ALL_OBJECTS B ON b.owner = a.owner and ltrim(B.OBJECT_NAME) = ltrim(A.TABLE_NAME)

--LEFT JOIN all_col_comments com ON (A.owner = com.owner AND A.table_name = com.table_name AND A.column_name = com.column_name)

WHERE

    a.owner = '{$schemaName}'

    and (b.object_type = 'TABLE' or b.object_type = 'VIEW')

    and b.object_name = '{$tableName}'

ORDER by a.column_id

EOD;

Override createColumn , findTableNames and findConstraints methods and lowercase the arrays values;

Override quoteSimpleColumnName:


   public function quoteSimpleColumnName($name)

    {

        return $name;

    }  

Now, we need to say to Yii to use our new class and add PDO attribute. In config, db.php point schemaMap to ESchemaOci and add atrributes:


<?php


return [

    'class' => 'yii\db\Connection',

    'dsn' => 'oci:dbname=//yourIP:yourPORT/yourSID;charset=UTF8',

    'enableSchemaCache'=>true,

    'username' => 'your_username',

    'password' => 'your_pass',

    'charset' => 'utf8',

    'schemaMap'=>['oci'=>'app\components\ESchemaOci',],

    'attributes'=>[PDO::ATTR_CASE=> PDO::CASE_LOWER],

];

That’s it.

ESchemaOci file:


<?php


/*

 * To change this license header, choose License Headers in Project Properties.

 * To change this template file, choose Tools | Templates

 * and open the template in the editor.

 */


/**

 * Description of ESchemaOci

 *

 * @author Cristi.Axe

 */

namespace app\components;


use  yii\db\oci\Schema;


class ESchemaOci extends Schema {

   

    protected function findColumns($table)

    {

        //$schemaName = $table->schemaName;

        //$tableName = $table->name;

		$schemaName = strtoupper($table->schemaName);

		$tableName = strtoupper($table->name);

        $sql = <<<EOD

SELECT lower(a.column_name) as column_name , a.data_type ||

    case

        when data_precision is not null

            then '(' || a.data_precision ||

                    case when a.data_scale > 0 then ',' || a.data_scale else '' end

                || ')'

        when data_type = 'DATE' then ''

        when data_type = 'NUMBER' then ''

        else '(' || to_char(a.data_length) || ')'

    end as data_type,

    a.nullable, a.data_default,

    (   SELECT D.constraint_type

        FROM ALL_CONS_COLUMNS C

        inner join ALL_constraints D on D.OWNER = C.OWNER and D.constraint_name = C.constraint_name

        WHERE C.OWNER = B.OWNER

           and C.table_name = B.object_name

           and C.column_name = A.column_name

           and D.constraint_type = 'P') as Key,

    '' as column_comment

FROM ALL_TAB_COLUMNS A

inner join ALL_OBJECTS B ON b.owner = a.owner and ltrim(B.OBJECT_NAME) = ltrim(A.TABLE_NAME)

WHERE

    a.owner = '{$schemaName}'

    and (b.object_type = 'TABLE' or b.object_type = 'VIEW')

    and b.object_name = '{$tableName}'

ORDER by a.column_id

EOD;


        try {

            $columns = $this->db->createCommand($sql)->queryAll();

        } catch (\Exception $e) {

            return false;

        }


        foreach ($columns as $column) {

            $c = $this->createColumn($column);

            $table->columns[$c->name] = $c;

            if ($c->isPrimaryKey) {

                $table->primaryKey[] = $c->name;

                $table->sequenceName = $this->getTableSequenceName($table->name);

                $c->autoIncrement = true;

            }

        }

        return true;

    }

    

protected function createColumn($column)

    {

        $c = $this->createColumnSchema();

        $c->name = $column['column_name'];

        $c->allowNull = $column['nullable'] === 'Y';

        $c->isPrimaryKey = strpos($column['key'], 'P') !== false;

        $c->comment = $column['column_comment'] === null ? '' : $column['column_comment'];


        $this->extractColumnType($c, $column['data_type']);

        $this->extractColumnSize($c, $column['data_type']);


        if (!$c->isPrimaryKey) {

            if (stripos($column['data_default'], 'timestamp') !== false) {

                $c->defaultValue = null;

            } else {

                $c->defaultValue = $c->phpTypecast($column['data_default']);

            }

        }


        return $c;

    }    

   

protected function findConstraints($table)

    {

        $sql = <<<EOD

        SELECT D.constraint_type as CONSTRAINT_TYPE, C.COLUMN_NAME, C.position, D.r_constraint_name,

                E.table_name as table_ref, f.column_name as column_ref,

                C.table_name

        FROM ALL_CONS_COLUMNS C

        inner join ALL_constraints D on D.OWNER = C.OWNER and D.constraint_name = C.constraint_name

        left join ALL_constraints E on E.OWNER = D.r_OWNER and E.constraint_name = D.r_constraint_name

        left join ALL_cons_columns F on F.OWNER = E.OWNER and F.constraint_name = E.constraint_name and F.position = c.position

        WHERE C.OWNER = '{$table->schemaName}'

           and C.table_name = '{$table->name}'

           and D.constraint_type <> 'P'

        order by d.constraint_name, c.position

EOD;

        $command = $this->db->createCommand($sql);

        foreach ($command->queryAll() as $row) {

            if ($row['constraint_type'] === 'R') {

                $name = $row["column_name"];

                $table->foreignKeys[$name] = [$row["table_ref"], $row["column_ref"]];

            }

        }

    }   

    

   protected function findTableNames($schema = '')

    {

        if ($schema === '') {

            $sql = <<<EOD

SELECT table_name, '{$schema}' as table_schema FROM user_tables

EOD;

            $command = $this->db->createCommand($sql);

        } else {

            $sql = <<<EOD

SELECT object_name as table_name, owner as table_schema FROM all_objects

WHERE object_type = 'TABLE' AND owner=:schema

EOD;

            $command = $this->db->createCommand($sql);

            $command->bindParam(':schema', $schema);

        }


        $rows = $command->queryAll();

        $names = [];

        foreach ($rows as $row) {

            $names[] = $row['table_name'];

        }


        return $names;

    }  

    

    public function quoteSimpleColumnName($name)

    {

        return $name;

    }       

}



Would you please create an issue about it so we won’t forget to merge it into framework?

Thank you for your feedback!

Meanwhile, i have decided to use materialized views for speed up reading db structure, because the issue is present only on huge databases. So i used custom views for application user schema.

On the other hand, customization for lowercase problem seems to achieve expectations, all extensions installed by me works out of the box for Oracle database. Due to simplicity of extending oci/schema (only two method to override) and the posibility to parse PDO config through db.php config, in my opinion, i don’t think it’s an yii issue (my mistake on picking up the title) but a developer choise about how to create objects in oracle db.

However, i hope this customization will help someone… someday.

Kind regards,

Thanks my friend. This post did help me too much.

I do some little changes to work for me(i run yii2 with advanced).

1 - Edit common/config/main-local.php




        'db' => [

            'class' => 'yii\db\Connection',

            'dsn' => 'oci:dbname=//localhost:1521/xe',

            'username' => 'faculdade',

            'password' => 'password',

            'charset' => 'utf8',

            'schemaMap'=>['oci'=>'common\components\ESchemaOci',],

            'attributes'=>[PDO::ATTR_CASE=> PDO::CASE_LOWER],

        ],



2 - Create folder


common/components

3 - Add ESchemaOci.php(query returning a.data_type, a.data_precision, a.data_scale, a.data_length to pass in extractColumnType and extractColumnSize), in common/components




<?php


/*

 * To change this license header, choose License Headers in Project Properties.

 * To change this template file, choose Tools | Templates

 * and open the template in the editor.

 */


/**

 * Description of ESchemaOci

 *

 * @author Cristi.Axe

 */

namespace common\components;


use  yii\db\oci\Schema;


class ESchemaOci extends Schema {

   

    protected function findColumns($table)

    {

        //$schemaName = $table->schemaName;

        //$tableName = $table->name;

                $schemaName = strtoupper($table->schemaName);

                $tableName = strtoupper($table->name);

        $sql = <<<EOD

SELECT lower(a.column_name) as column_name, a.data_type, a.data_precision, a.data_scale, a.data_length,

    a.nullable, a.data_default,

    (   SELECT D.constraint_type

        FROM ALL_CONS_COLUMNS C

        inner join ALL_constraints D on D.OWNER = C.OWNER and D.constraint_name = C.constraint_name

        WHERE C.OWNER = B.OWNER

           and C.table_name = B.object_name

           and C.column_name = A.column_name

           and D.constraint_type = 'P') as Key,

    '' as column_comment

FROM ALL_TAB_COLUMNS A

inner join ALL_OBJECTS B ON b.owner = a.owner and ltrim(B.OBJECT_NAME) = ltrim(A.TABLE_NAME)

WHERE

    a.owner = '{$schemaName}'

    and (b.object_type = 'TABLE' or b.object_type = 'VIEW')

    and b.object_name = '{$tableName}'

ORDER by a.column_id

EOD;


        try {

            $columns = $this->db->createCommand($sql)->queryAll();

        } catch (\Exception $e) {

            return false;

        }


        foreach ($columns as $column) {

            $c = $this->createColumn($column);

            $table->columns[$c->name] = $c;

            if ($c->isPrimaryKey) {

                $table->primaryKey[] = $c->name;

                $table->sequenceName = $this->getTableSequenceName($table->name);

                $c->autoIncrement = true;

            }

        }

        return true;

    }

    

protected function createColumn($column)

    {

        $c = $this->createColumnSchema();

        $c->name = $column['column_name'];

        $c->allowNull = $column['nullable'] === 'Y';

        $c->isPrimaryKey = strpos($column['key'], 'P') !== false;

        $c->comment = $column['column_comment'] === null ? '' : $column['column_comment'];


        $this->extractColumnType($c, $column['data_type'], $column['data_precision'], $column['data_scale'], $column['data_length']);

        $this->extractColumnSize($c, $column['data_type'], $column['data_precision'], $column['data_scale'], $column['data_length']);


        if (!$c->isPrimaryKey) {

            if (stripos($column['data_default'], 'timestamp') !== false) {

                $c->defaultValue = null;

            } else {

                $c->defaultValue = $c->phpTypecast($column['data_default']);

            }

        }


        return $c;

    }    

   

protected function findConstraints($table)

    {

        $sql = <<<EOD

        SELECT D.constraint_type as CONSTRAINT_TYPE, C.COLUMN_NAME, C.position, D.r_constraint_name,

                E.table_name as table_ref, f.column_name as column_ref,

                C.table_name

        FROM ALL_CONS_COLUMNS C

        inner join ALL_constraints D on D.OWNER = C.OWNER and D.constraint_name = C.constraint_name

        left join ALL_constraints E on E.OWNER = D.r_OWNER and E.constraint_name = D.r_constraint_name

        left join ALL_cons_columns F on F.OWNER = E.OWNER and F.constraint_name = E.constraint_name and F.position = c.position

        WHERE C.OWNER = '{$table->schemaName}'

           and C.table_name = '{$table->name}'

           and D.constraint_type <> 'P'

        order by d.constraint_name, c.position

EOD;

        $command = $this->db->createCommand($sql);

        foreach ($command->queryAll() as $row) {

            if ($row['constraint_type'] === 'R') {

                $name = $row["column_name"];

                $table->foreignKeys[$name] = [$row["table_ref"], $row["column_ref"]];

            }

        }

    }   

    

   protected function findTableNames($schema = '')

    {

        if ($schema === '') {

            $sql = <<<EOD

SELECT table_name, '{$schema}' as table_schema FROM user_tables

EOD;

            $command = $this->db->createCommand($sql);

        } else {

            $sql = <<<EOD

SELECT object_name as table_name, owner as table_schema FROM all_objects

WHERE object_type = 'TABLE' AND owner=:schema

EOD;

            $command = $this->db->createCommand($sql);

            $command->bindParam(':schema', $schema);

        }


        $rows = $command->queryAll();

        $names = [];

        foreach ($rows as $row) {

            $names[] = $row['table_name'];

        }


        return $names;

    }  

    

    public function quoteSimpleColumnName($name)

    {

        return $name;

    }       

}



Again, thanks for advanced.