I have a model which retrieves/process data from a MySQL table.
Due to some limitations we have found (storing very big data in JSON format in the table, and trying to recover them, we hit the max_allowed_packet problem limitation (1GB maximum output)), so we are thinking to switch this approach to a file storage based model.
Basically I want to retrieve the information from the DB, if record already exists, otherwise, retrieve the info from the corresponding saved file. Also, on create a new record, always create the record in file. (don’t use the db table anymore, only for old records)
What kind of approach do you suggest me to use? Create a new data provider like the one here ?? Overwrite/extend the ActiveRecords methods (for example, save() , etc) in the current model to achieve what I want ??
Thank you so much!
[Update]: Other approach will be, insert new records in the DB with the minimum data but with an special flag (field in the table) to let me know that we also have to create a JSON file in the storage with the full content. While recovering a record from this table, if the record has that special flag set to true (read_from_file => true), populate the record with the info from an specific file (JSON file) ?? Does anyone how to achieve this ?