Deleting flash messages from session in an AJAX call

I have a cached page, that requests a controller action do display/render the flash messages after the cached page is loaded.

I get the messages like this, which should delete them:

$notices = Yii::$app->session->getFlash('notice', null, true);

But somehow, if I then reload the cached page, the AJAX request to the messages action still shows the messages.