Insert Large Amount Of Data

[mimo (Mabcapelo)]

Hi guys!

I’m trying to export a large amount of data from a csv file and insert it into a database table and I wanted to do it in one query only.

I tried to see if there was a good way to do it, and I thought I found it with createMultipleInsertCommand.

But it only works with 8000 entries in the csv file. With more lines, it starts to ask me for more and more memory (I get "Fatal error: Allowed memory size of 1258291200 bytes exhausted").

I’m not sure if I should try to do something else… There isn’t a way to apply multi-query in Yii, right? So, should I try to do something based on PDO like this post, or is there a better way?

Thanks in advance!

[alirz23]

I also do bulk import in one of my apps from excel, I just increased the memory limit to 128mb no issues

[Ankit_Modi]

Hi guys!

I’m trying to export a large amount of data from a csv file and insert it into a database table and I wanted to do it in one query only.

I tried to see if there was a good way to do it, and I thought I found it with createMultipleInsertCommand.

But it only works with 8000 entries in the csv file. With more lines, it starts to ask me for more and more memory (I get "Fatal error: Allowed memory size of 1258291200 bytes exhausted").

I’m not sure if I should try to do something else… There isn’t a way to apply multi-query in Yii, right? So, should I try to do something based on PDO like this post, or is there a better way?

Thanks in advance!

Hi Mimo,

I think you may use the apply limit @ insert time(create the batch).

e.g If i want to insert the 1000 record at the time and I create the limit 100, so when query will be executed, first 100 record insert after second 100 to 200 etc…

means your loop will be executed 10 times