That is the correct behavior (see ajaxUpdate), it only applies to paging and sorting. To change it you need to override the delete button and provide your own url (see CButtonColumn template, bottom of page shows examples)
Thank you for your answer! Actually, I was aware that I can configure my own delete button. I was just wondering why CGridView is designed so that ajaxUpdate=>false applies to sorting and pagination only and not to deleteButton. But sure, this is not a big problem.