DATETIME handling

Sure, you can store everything as strings and integers - the way I was schooled, that is not considered good database design, and on a modern DBMS, that’s not going to give you any advantage, neither in terms of performance or data volume. Datatypes like DATETIME and ENUM have many advantages over the basic data types, transparency being just one of them.

Integers used to store dates, for example, are not very efficient when it comes to more advanced operations like, say, writing a query that finds every entry posted on the first of the month. I don’t even want to think what a query like that would look like on integer timestamps.

But we’re really getting into an entirely different discussion here - on a topic that was outdated 10 years ago, and thankfully made irrelevant by modern DBMS long ago…

By the way, looking at implementation strategies, I agree that behaviors is probably the way to go - thank you for that idea! This way, if a data mediator was added to the framework, using it would be entirely up to the developer, and it’s existence would not affect any existing code - you could choose to use it or not, which is always a great thing in a framework.

Having many useful features is good, but forcing them on developers (or making them pay in the form of overhead) is not :slight_smile:

mindplay: thank you for bringing up the issue.

I would like to add a few thoughts.

I noticed that in many projects datetime fields do not seem to be that important, they are only used to have a created and changed field for your database rows.

You only learn about the beauty of datetime fields when you really have to use them which I did when I started to program a time recording system for my company.

Pestaa: Have a look at the date/time functions of PostgreSQL, you can do real magic with them, forget MySQL, forget integer columns. I also often use a datetime field instead of date because you can then use infinite and -infinite as a value in PostgreSQL and all the algebra still works beautifully.

In my time recording system I converted my datetime fields back and forward and was never happy with it, so I hoped I would find a nice solution in Yii. Unfortunately there is none, no dateValidator, no convertion for DB. If you want to have multiple locale support its not easy to come up with a good solution, but I think it would be worth it and it should be in the Yii core. Maybe another framework could deliver some inspiration (Django)?

The idea of a data mediator class comes from Java, or more accurately, from JSP.

If you just need to handle dates, you might want to take a look at this:

http://www.yiiframework.com/extension/i18n-datetime-behavior/

With a generic abstraction layer for type conversions, you could handle much more than dates - Rails, for example, has a plugin that abstracts filesystem references in the database, which means you can upload/download files referenced by filename in a column, for example, without manually converting to and from file reference objects.

Basically, with a type mediator, you can abstract any kind of datatype. Think about images managed in this way - you probably don’t want images stored in blobs, you would most likely rather have a filename reference in a column, but if your image was presented to you as an object straight from the model, you could do things like versioning (resizing, cropping, etc.) by simply calling methods directly on the objects in the model, as well as replacing (uploading) or deleting the referenced image file, and so on.

Personally, I love abstraction - the less grunt work you have to do, the more real work you can get done. And doing things like image management, file uploads, date handling or other type management, once and for all, means you don’t have to re-implement these features for every project - you will have stronger and more reliable features every time you improve on your solutions, while spending a fraction of the development time.

For some people this approach works, for some it doesn’t - a feature like this needs to be optional, so that anyone can use the framework in a way that fits their way of thinking…

I hammered out some example mediator code.

It comes in the form of a CAttribute class - an abstract base class for abstract data types, e.g. anything that cannot be represented by simple PHP variable types.

Note that this code has not been tested, and I have not yet attempted to integrate this with CActiveRecord - it’s just an idea at this point.

The integration would consist of a simple modification to CActiveRecord::setAttribute(), which would check if the attribute being set is a CAttribute, and if it is, it would call setValue() on it, rather than overwriting the attribute itself.

Modifications would be required (I’m not entirely sure yet where), when attributes are written into an UPDATE or INSERT statement, there needs to be a check to see if the attribute is CAttribute, and if it is, get the mediator for the current DB connection and call CAttributeMediator::getSqlValue() which converts the attribute to the SQL representation required for the connection’s driver.

Likewise, after a SELECT, when a CActiveRecord is constructed, CAttributeMediator::setSqlValue() needs to be used to convert the SQL string into a CAttribute representation.

I think I could implement most of this without actually hacking any part of the Yii codebase, just creating overrides, so I might continue with this proof of concept to see if I can pull that off - unless someone can convince me that this idea is somehow retarded in the first place :wink:

But most ORMs have a mediation layer of some sort - in the case of PHP it’s even more necessary than in some other languages, because PHP only has a few primitive types of it’s own; a timestamp isn’t even discernible from an integer as such, a file path isn’t discernible from any other string, and so on.

Having more abstract type implementations, however, is only really interesting if we can somehow mediate between SQL and form posts, without relying on manual conversion everywhere - it’s tedious, and error prone :wink:





<?php


/**

 * Abstract base class for an attribute class.

 *

 * An attribute object encapsulates an abstract value, which cannot be represented

 * by a single, native PHP variable-type.

 */

abstract class CAttribute

{

  /**

   * Returns the mediator for the attribute

   * @see CAttributeMediator

   */

  public function getMediator($dbConnection=null)

  {

    if ($dbConnection===null)

      $dbConnection = Yii::app()->db;

    

    $driver = $dbConnection->getDriverName();

    

    static $mediators=array();

    

    $className = get_class($this).ucfirst($driverName).'Mediator';

    if (isset($mediators[$className]))

      return $mediators[$className];

    

    if (class_exists($className))

      return $mediators[$className] = new $className;

    else

      throw new CException("CAttribute::getMediator() : no mediator {$className} was found");

  }

  

  /**

   * Returns a native PHP value that represents the internal value of the attribute

   */

  abstract public function getValue();

  

  /**

   * Sets the internal value of this attribute to a native PHP value, or a string posted by an HTML <form>

   * @param mixed a native PHP value

   * @param boolean if true, $value will be handled as a string posted by an HTML <form>

   */

  abstract public function setValue($value,$post=false);

}


/**

 * Abstract base class for an attribute mediator.

 *

 * An attribute medidator is responsible for conversion of an abstract value, 

 */

abstract class CAttributeMediator

{

  /**

   * Returns an SQL string representation of the internal value of the given attribute.

   * @param CAttribute the attribute to convert to an SQL string

   * @return mixed the SQL string representation of the value, or null if the attribute is unset

   */

  abstract public function getSqlValue($attribute);

  

  /**

   * Sets the internal value of the attribute according to an SQL string value.

   * @param CAttribute the attribute to be set

   * @param string the SQL string value to be interpreted and applied to the given attribute

   */

  abstract public function setSqlValue($attribute, $string);

}


/**

 * This example attribute-type encapsulates a PHP timestamp.

 */

class CDatetimeAttribute extends CAttribute

{

  protected $_timestamp;

  

  public function getValue()

  {

    return $this->_timestamp;

  }

  

  public function setValue($value,$post=false)

  {

    if ($post)

      return $this->_timestamp = !empty($value) ? strtotime($value) : null;

    if (is_numeric($value) || $timestamp===null)

      return $this->_timestamp = value;

    throw new CException("CDatetimeAttribute::setValue() : incompatible value '{$value}'");

  }

  

  public function __isset()

  {

    return $this->_timestamp > 0;

  }

}


/**

 * The example mediator converts to and from a MySQL DATETIME formatted string representation

 */

class CDatetimeAttributeMysqlMediator extends CAttributeMediator

{

  public function getSqlValue($attribute)

  {

    $value = $attribute->getValue();

    return !empty($value) ? date('Y-m-d H:i:s', $value) : null;

  }

  

  public function setSqlValue($attribute, $string)

  {

    $attribute->setValue(

      $string!='' ? strtotime($string) : null

    );

  }

}



I thought about how to integrate this - I think it needs to happen at the lowest level possible. So for conversion to SQL strings, that would be in CDbCommand::prepare() - and for conversion from SQL strings to attributes, in CDbDataReader.

Or not. The other possibility is that CActiveRecord could handle this all internally. If there is some way that CActiveRecord knows that it’s being populated by a finder, it could be responsible for the SQL conversions internally. Using __toString() magic, the CAttribute classes could themselves be responsible for the conversion back to SQL strings.

I think maybe this would approach would be preferable?

Rather than having complex conversion processes integrated in various classes, and at various levels of abstraction, if all of the required extensions could reside in a single class, that would be much cleaner.

Thoughts?

I was thinking about similar topic today.

I wanted to put a layer on __get and __set methods of AR class and implement some conversions in that layer. I would also configure the layer by setting the value of some class variable in AR so it knows which output/input instance of the layer to use.

Internally the values of attributes in AR would be held always in php native format.

The idea was simple.

When i output data from AR to presentation layer in a view i set $ar->setOutputLayer(‘presentation’) and do conversions to human readable formats internally by using just echo $ar->anAttribute.

When i need to save the data to db i call $ar->setOutputLayer(‘db’) and the layer communicates with db schema and db driver and does proper conversions when reading attribute values to save them. AR uses $ar->anAttribute to get the values so the layer would work here.

Similar thing with setting the value to the input layer.

The question is whether to use abstraction or just upgrade AR to do some simple tasks even if i’m sacrificing some concepts of mvc for a simple solution that can be implemented in one place.

As i understand MVC, it tells us that between presentation layer and db layer must be a controller layer. So AR can’t do what a controller should do.

But what if i inject some functionality into AR in a controller or

configure the input/output layer of AR class in a controller

and let him do the hard work?

Abstraction may come for a price. The price may be simplicity. Abstraction may also cause overhead if we wrap simple data into a more complex classes. In my opinion the choice is not simple.

The idea of abstraction can touch different aspects of the application - structure, processes, flows, states, actions, data itself etc. It’s not always a simple task to define what should we abstract or what shouldn’t we.

Currently i’ll stick with simple solutions and my thoughts are:

  • override __get to do conversions

  • override __set to do conversions

  • add functionality of configuration the conversion process

I’m still learning Yii’s functionality and concepts and I’m quite new here so i don’t want to hack what’s deep inside the framework until i fully understand the concept of each of the "C"Classes ::)

@mindplay

I brainstormed CAttribute a while back: http://www.yiiframework.com/forum/index.php?/topic/885-attributes-as-classes/

My concern though is performance. I know django uses this technique

Are you guys talking about casting the database fields to their equivalent CAttributeModel? e.g. text would be CTextAttribute, int CIntAttribute etc

Am I correct to assume this would then allow us to write $product->description->truncate(50); (after adding truncate to the CTextAttribute to string class…by using something like behaviours).

Constructing an object for every single attribute would probably add quite a bit of overhead.

I was advocating abstraction/mediation of types that don’t translate easily between MySQL and PHP - strings and integers already translate well, so I don’t think those require special care. Abstracting a string as an object is not really PHP “style”, is it?

I was not thinking of abstracting strings and integers as objects. I think that would be overkill. I was thinking more of things like timestamps. Also it would help modularize your code. For instance, you could abstract a password attribute as an object, and contain the hash, reset, ect, methods and beforeSave logic in it.

I wonder if it could be somehow done ‘virtually’ though - so that a new object doesn’t get constructed every time. It really doesn’t need to. You have the values in their original form in the model object - all you need is a nice syntax for transparently converting the value, on-demand, so that there’s as little overhead as possible. If a single instance of the converter could handle the conversion for all instances, that would mean much less overhead…

As for the password component logic, you can already have that. For example:




<?php


class User {


  ...

  

  private $_passwordManager;


  public function getPassword()

  {

    if (!isset($this->_passwordManager))

      $this->_passwordManager = new MyPasswordManager($this);

    return $this->_passwordManager;

  }

}


...


$user->password->set('hello');


...


if ($user->password->verify($_POST['password']))

{

   ....

}



The password manager will be loaded on-demand, constructed as-needed.

Lately I find that this approach works great for a number of things :slight_smile:

@mindplay:

A little OT: I also find this pattern very useful. It’s like a “cheap cache” for complex object properties that are not always used. Actually it’s so common in Yii and my code, that i feel like PHP could use another function type for easy declaration ;).

Something like this would be cool:


public cached function getPassword() {

    return new MyPasswordManager($this);

}



PHP should translate that into:


private $__getPassword;

public function getPassword() {

  if($this->__getPassword===null)

    $this->__getPassword=newMyPasswordManager;

}

There could be another keyword to also specify the empty value (would default to null) to make it complete and also allow "null" as value:


public cached function getPassword() empty false{

    return new MyPasswordManager($this);

}




private $__getPassword=false;

public function getPassword() {

  if($this->__getPassword===false)

    $this->__getPassword=newMyPasswordManager;

}

But i can hardly imagine that something this will ever go into PHP ;)

End OT.

Accepting that challenge, I came up with a little hack and syntactic sugar to that effect:




<?php


function cached()

{

  $params = func_get_args();

  $class = array_shift($params);

  

  $caller = debug_backtrace();

  $object = $caller[1]['object'];

  $key = '__'.$caller[1]['function'].'_'.$caller[0]['line'];

  

  if (isset($object->$key))

    return $object->$key;

  

  return $object->$key = call_user_func_array(

    array(new ReflectionClass($class), 'newInstance'), $params

  );

}


class Password {

  

  public $md5;

  

  public function __construct($password)

  {

    $this->md5 = md5($password);

  }

}


class User

{

  public function getPassword()

  {

    return cached('Password', 'abc123');

  }

}


header('Content-type: text/plain');


$test = new User;


var_dump($test->getPassword(), $test);



The first argument to cached() is the class-name, and subsequent arguments will be applied to the constructor.

Note that I added the calling method’s line-number to the property name, since you might have more than one “return cached” statement in the same function. If you don’t care about that, you can simplify slightly:




$key = '__'.$caller[1]['function'];



PHP provides a lot of tricky little mechanisms these days - if you put them to work, you can can it to do pretty much anything you want :wink:

By the way, I don’t know if I like the term “cached” for the function-name… it’s not really a cache - more like a shortcut for lazy instantiation of a helper object.

Perhaps, "return lazy", hehe.

Or the ultra-short and more semantic version: return a(‘Password’);

haha :slight_smile:

By the way, I benchmarked this, and the overhead when compared with the same hard-coded pattern is somewhere around 400% and 500% :wink:

This may sound worse than it is - function call overhead is insignificant in most cases in the first place, so you may add 0.02 milliseconds of overhead per call, if you use this function instead of hard-coding the same pattern.

During a request that uses this pattern 100 times, you would incur an overhead of 0.2 milliseconds, so it’s still pretty insignificant and probably negligible unless you’re building some sort of performance-critical mega-operation :wink:

@mindplay Very clever technique!

@mindplay:

Nice solution. And “lazy” is definitely the right term. While i was sleeping i was thinking about something similar ;). In fact, if your benchmarks are right and since this is such a common requirement, should we think about adding this to CComponent?

The implementation is quite easy (6 new lines, as marked below plus a setter method):


class CComponent

{

    private $_e;

    private $_m;

    private $_l;    // NEW


    public function __get($name)

    {

        $getter='get'.$name;

        $lazyInit='lazy'.$name;     // NEW

        if(method_exists($this,$getter))

            return $this->$getter();

        else if(strncasecmp($name,'on',2)===0 && method_exists($this,$name))

        {

            // duplicating getEventHandlers() here for performance

            $name=strtolower($name);

            if(!isset($this->_e[$name]))

                $this->_e[$name]=new CList;

            return $this->_e[$name];

        }

        else if(isset($this->_m[$name]))

            return $this->_m[$name];

        else if(isset($this->_l[$name]))    // NEW

            return $this->_l[$name];        // NEW

        else if (method_exists($this,$lazyInit))    // NEW

            return $this->_l[$name]=$this->$lazyInit(); // NEW

        else if(is_array($this->_m))

        {

            foreach($this->_m as $object)

            {

                if($object->getEnabled() && (property_exists($object,$name) || $object->canGetProperty($name)))

                    return $object->$name;

            }

        }

        throw new CException(Yii::t('yii','Property "{class}.{property}" is not defined.',

            array('{class}'=>get_class($this), '{property}'=>$name)));

    }


    public function lazySet($name,$value)

    {

        $this->_l[$name]=$value;

    }

To define such a lazy initiated property, we could use the lazy keyword, similar to get or set:


class Something extends CComponent {


    protected lazyPassword() {

      $pw=new Password();

      $pw->something='else';

      return $pw

    }

}

now we could do like:


$x=new Something;

$password =$x->password;  // this would trigger lazy init



On the other hand: Since CComponent is the basis for pretty much everything much care has to be taken that this component doesn’t get bloated. Maybe some more benchmarks could help here.

Adding another member variable to CComponent will add another member variable to (pretty much) every object ever instantiated in Yii.

Also, Behaviors in Yii are already managed and created using lazy instantiation - it’s probably overkill, adding another feature just for that?

If lazy instantiation is going to be available as a separate feature, I think Behaviors need to be refactored to use the same feature, so we don’t end up with two implementations, two standards,…

Yeah, maybe overkill. It was only an idea, but i’d prefer a solution in a clean OOP manner. So maybe i just have to get used to type this pattern again and again … ;).

Just came up with a much simpler way to implement the lazy construction pattern - check this out:





<?php


class Test

{

  public function __get($name)

  {

    echo "invoking the magic accessor:\n\n";

    return call_user_func(array($this, 'get'.ucfirst($name)));

  }

  

  public function getBigValue()

  {

    return $this->bigValue = array(

      'big'=>'value',

      'or'=>'some expensive object',

      '...'

    );

  }

}


header('Content-type: text/plain');


$test = new Test;


var_dump($test->bigValue);


echo "\nmagic accessor not invoked the second time:\n\n";

var_dump($test->bigValue);



The first time the magic accessor is invoked, it simply assigns it’s return-value to a property with the same name - subsequent calls don’t even invoke the magic accessor then, so this is much more effiecient - and requires very little code, just the extra assignment in the return-statement.

The password example would look like this:




class User

{

  public function getPassword()

  {

    return $this->password = new PasswordManager($this, 'supersecretpassword');

  }

}



Pretty simple? :slight_smile: