How to generate incremental page updates

Hello all, this question regards Yii-1.1.4 on the Linux/MySQL platform.

Short version: how do I generate a page whose data is obtained slowly, over the course of perhaps a minute, without making the user wait the whole time before page generation even starts? I don’t quite know where to start.

Long version: the page in question will be doing SNMP polling of a large number of devices, and there will be some data associated with each device. The “obvious” way to do this is to collect all the data (making the user wait), then start generating stuff, but I’d like to have a page incrementally update so the user can see what’s going on.

I don’t think this is asking for ajax, as the browser is not driving the process, the controller is. I’ve been looking at it and don’t really even know where to start: though renderPartial looks promising, that just doesn’t feel right either.

I’d sure love a bit of direction.

Thanks!

My first idea when reading your problem: Split this up in 2 applications. A command line app that runs periodically and gathers the SNMP data to write it to a DB and a second app as web interface to display this data. There are even tools that could help you with the first app like Tobias Oetiker’s RRDtool.

Hi Mike: thanks for your reply.

Breaking this up isn’t really solving the problem - any action that might take some time will simply hang until everything finishes, and this is really a disconcerting application experience.

Looking at the source for CController::render(), it seems that the very first thing it does is renderPartial() to take the model and view and do all the real work, producing a $content string - this is the operation that might take the time. After that, the layout is replied to the $content string, and postprocessing is done.

The pipelined nature of this means that it’s going to be very difficult to do this with straight PHP/HTML, as we’d have to allow part of the output (the header, at least) to run through the pipeline before running the time-consuming operations. That looks structurally really intrusive.

So I think the only way to really handle this is with JavaScript: generate the page with fillers, then after the fact use JS to update the individual tags on the fly. This will be some amount of effort too, but it may not be the end of the world.

I appreciate the thoughtful response.

Ah, you want continuous output sent to the browser?

This only works if the markup matches specific criterias (see flush()). Otherwhise the browser will not update the display until the page is transferred completely.

It’s rather a hack, but i once did this to watch the live progress of a DB import script (pseudo code, just showing the concept):




public function importDB()

{

    $this->renderHTMLHead();   // echoes the HTML header

    foreach($this->getImportSql() as $sql) {

        $this->sendOutput("Executing: $sql");

        $this->executeSql($sql);

    }

    $this->renderHtmlFoot();  // echoes the HTML foot

}


public function sendOutput($msg)

{

    printf('<div class="message">%s</div>',$msg);

    flush();   // Does the trick...

}



You can’t embed this in a layout of course for the reasons you already described.

Maybe it gives you an idea.

I suspect there’s a way to hack up the layout file so it’s given a parameter to produce the first or last part of the file, but this all fits in the category of way too intrusive (as opposed to some supported part of the framework I had just missed), so I’m going to punt on this whole area until time permits me to do the kind of refactoring required.

It would be nice if there were some integrated mechanism to make this work, but I don’t really even know what the exact requirements would be.

Thanks for your thoughts…

Steve

If you push your output into Yii::log or Yii::trace, you may be able to accomplish what you desire, by setting parameters in CLogger, like so:


        Yii::getLogger()->autoFlush = 10;

        Yii::getLogger()->autoDump = true;



Autoflush is at 10,000 by default, I’d maybe put it to 1,000 or so personally. Autodump will actually force the logs to be printed to screen (or wherever your logs are configured to route to, such as db, flat file, etc…)