Hi,
I found several info about switch memory use for application, which can switch in boot.ini from 2GB/2GB to for example 1GB/3GB.
Where can I found file boot.ini, or switch /3GB can put in own file in my program directory ?
I want try it, because my program works with large file stored in variable, max file size is 1,4GB !. With 2GB limit processing crash.
However, I know, must modify source to controlling variable content and break it to several smaller parts....
RAM limit
-
Cliff Wiernik
- Posts: 605
- Joined: Thu Jan 28, 2010 9:11 pm
- Location: Steven Point, Wisconsin USA
- Contact:
Re: RAM limit
We had a similar problem, extracting data to export to a csv file. The user was reading all into array and then writing out, because it was the most efficient in terms of processing time. We changed it to read in amount within memory limits, write to csv, then read the next chunk and append to that file. Same efficiency but works within memory constraints.
Your data may be different, but this is how we did it. We had to adapt the Express function for writing to the csv but that was easy to create our own version of it.
Your data may be different, but this is how we did it. We had to adapt the Express function for writing to the csv but that was easy to create our own version of it.
Re: RAM limit
Hi Cliff,
yes, I used the same method, if read text file to variable/array. Function read portion of file - block about 100MB, process it, save to file, then read other portion etc. This is for processing text file to text file.
In other function I open DBF file, this have size from several kB to 1,5GB and more than 1.000.000 rows, function processing this file and store data to variable/array. After this save to file on disk.
But this works also only for smaller files, then 200-400MB.
Now I must change algoritm also for this functions ,it is not difficult, only put rows counter when read database, after about 500000 records save it to file, null counter and then process next portion.
This is for DBF to text file.
I want check out change memory management, but it is not good way, because user may have only 2GB, or 1GB ram on PC.
And also problem is, when user run some process, and also open other applications, which used some memory.
Viktor alias Victorio
yes, I used the same method, if read text file to variable/array. Function read portion of file - block about 100MB, process it, save to file, then read other portion etc. This is for processing text file to text file.
In other function I open DBF file, this have size from several kB to 1,5GB and more than 1.000.000 rows, function processing this file and store data to variable/array. After this save to file on disk.
But this works also only for smaller files, then 200-400MB.
Now I must change algoritm also for this functions ,it is not difficult, only put rows counter when read database, after about 500000 records save it to file, null counter and then process next portion.
This is for DBF to text file.
I want check out change memory management, but it is not good way, because user may have only 2GB, or 1GB ram on PC.
And also problem is, when user run some process, and also open other applications, which used some memory.
Viktor alias Victorio