Implementing a ContentManager / FileSystemManager

Hi guys, I’m in the process of writing a fairly sizable application that will have to manage quite a bit of file data. As is good practice I don’t want to store the files as BLOBs in the database as that makes an already large database almost impossible to maintain (e.g. backup regularly, etc) and want to store the files (e.g. images, documents, etc) in the file system. Because of the large number of files (i.e. tens of thousands) it’s also necessary to split files within each folder (e.g. images) into multiple folders using some sort of a partitioning strategy which may be completely independent of the record with which the file is associated or the file name or may be derived based on either the record or the file name (e.g. 001, 002, 003… OR A, B, C…, etc). The strategy would need to depend on the number of files expected and the sparsity of the data.

I have searched both Yii web resources and the internet for an implementation of something like this or something similar in another language but havn’t found anything useful, which is rather surprising given that I would have thought many people would need something like this. Perhaps people just took the simple hack route rather than writing a class (or multiple classes) to do this. So, I’m currently looking to implement it myself and most likely share is as an extension or at least an example.

I’m thinking of implementing it more or less similar to CAssetManager but it will probably be a bit more complex. It would be called something like EContentManager, or EFileManager or EFileSystemManager to stay consistent with the naming of CAssetManager or perhaps even just EFileSystem. I will also probably create something something like IDirectoryPartitioningStrategy and concrete implementations of one or more partitioning strategies (GoF strategy pattern), or perhaps rather than creating too many files maybe even do this in the FileSystem implementation itself (although that’s technically not the best way to do it). I’m thinking I might implement an abstract class which deals with the basics and then perhaps a concrete class which is specific to my particular application and has knowledge of the specifics types of files I will be storing in the file system (e.g. Images, Documents, Logos). I may also implement file name creation using a strategy pattern.

To give you an idea of what I was thinking in terms of implementation, the class will have properties such as baseUrl and basePath like CAssetManager but then on top of that will have other properties like imageDirBasePath, logoDirBasePath, documentDirBasePath and largeImageDirBasePath, mediumImageDirBasePath, smallImageDirPath, which will be application specific. There will also be methods like ensureStructure, which will ensure the file system structure is always right and will be called from the init method. Then there would also be methods like createLargeImagePath($fileName), createDocumentPath($fileName), etc that will use the partitioning strategy to create a path to which the file should be saved.

I’m interested to know if anyone has done something similar in the past or would have any suggestions. I’m particularly interested to hear from anyone on the Yii core team (e.g. Qiang, Sam Dark, Y!!) or any other experienced developers with plenty of Yii or general OO experience. Hope what I’m trying to achieve makes sense.

We’ve solved this problem by using Alfresco’s CMIS web service interface. It’s not that easy, but I can help you with it if you choose this solution. :)

By the way the keyword is CMIS. If you want to use an external ECM to store your files it makes it easier to communicate with it, if you want to create your own implementation it still helps by defining a base functionality that should be implemented.

More info here:

Specification here:

I don’t think the solution you’re proposing is quite what I’m looking for or trying to do here… That’s like wheeling in Formula 1 car when all you want to do is go to your local supermarket to get some groceries. The solution you’re proposing is way too complex and an overkill for what’s needed here, I’m just looking to write a class not create an interoperable store for Enterprise Content Management systems.

Anyone else have any more specific ideas within the boundaries of what I’ve outlined in my post?

The point is not creating but using an existing solution in a standard way(web service or atom pub interface), what you can do relatively easy with a system that implements the CMIS standard.

On the other hand if you still want to create your own solution CMIS standard’s navigation services and object services are good starting points to plan the basic functionality of your class.

If you’re looking for ways storing large number of files, you might want to read this if you haven’t already:

DOes anyone else have any ideas?

I guess that this depends on your particular needs…

I had a similar problem for a site I developed a while ago… it was a travel site with many locations… and for every location there where some documents and images… every location when added to the database gets a code that is calculated depending where the location is… so that from that code the program knows the parent location and the child locations…

something like:

  • Europe is 1,

  • Croatia is 101 (1 Europe, 01 Croatia)…

  • Rovinj (a city in Croatia) is 1010001 (1 Europe, 01 Croatia, 0001 Rovinj)

All pictures are in the folder pictures…

Pictures for location Europe are in the folder pictures/1

Pictures for location Croatia are in the folder pictures/1/01

Pictures for location Rovinj are in the folder pictures/1/01/0001

Additionaly… on the same site… there are objects like private appartments, hotels and so on… that need to have many pictures themselves…

Every object has an ID that is used as a PK…

So pictures are located inside the folder "pictures/objects/id of the object"

Hope that gives you more ideas :)

Not sure if my solution help. This is what I am basically doing for the User Image Gallery, and Video Gallery.

  1. Under images I created a folder called igallery

  2. Under ‘igallery’, I have following folder ‘original’, ‘larger’, ‘medium’, ‘thumbs’, and ‘modified’

  3. In Global Ap Config, I defined as follows:

    define(‘ABS_PATH’, ‘C:/www/home/gfied/’);

    define(‘URL_BASE’, ‘http://localhost/gfied/’);

    // FSO

    // —

    define(‘ABS_FORSALE’, ABS_PATH.‘images/forsale/’);

    define(‘URL_FORSALE’, URL_BASE.‘images/forsale/’);

    define(‘FS_ABS_ORG’, ABS_FORSALE.‘originals/’);

    define(‘FS_ABS_LRG’, ABS_FORSALE.‘larger/’);

    define(‘FS_ABS_THB’, ABS_FORSALE.‘thumbs/’);

    define(‘FS_ABS_MOD’, ABS_FORSALE.‘modifieds/’);

    define(‘FS_URL_ORG’, URL_FORSALE.‘originals/’);

    define(‘FS_URL_LRG’, URL_FORSALE.‘larger/’);

    define(‘FS_URL_THB’, URL_FORSALE.‘thumbs/’);

    define(‘FS_URL_MOD’, URL_FORSALE.‘modifieds/’);

  4. when I save the medias, in my case images, video and bar-codes, I dynamically create folder for that user under the defined ABSOLUTE paths.

    while reading, I append the UID to ABS path, and the media file name. In some case, I save the URL of the physical media in the table column.

Similarly the video resides in vgallery folder, and so forth for the user.

Hope it helps!

Thanks for the reply guys, yeah, that’s kinda what I’m looking to do from a functional perspective and I guess I was wondering what experience people had from a practical perspective instead of structuring this… In terms of defines I don’t think that’s the way I’ll go about it as I just tend to use classes with constants and then getters as that’s more of the OO way rather than using defines. I guess my question was more about what some experienced people thought in terms of structuring something like what I’m looking to implement and whether anyone has written it as an actually FileSystemManager class as such and what your experiences were. I guess I was also wondering if something like this would be a worthy contribution for others to use as a base.

If you can do a neat extension do share with us.