Sampling data from CSV to DBF

This forum is for eXpress++ general support.
Message
Author
User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#21 Post by Eugene Lutsenko »

Auge_Ohr wrote:sorry to say but it is simple bad Database Design with so many Fields. :naughty:
even a SQL Database will not accept so many Fields. :roll:
Jimmy, Roger!

I've never needed so many fields. First time. About the ratio of 1 to 1 I know. This idea I also once came when I was faced with this task. But the problem is that there is a large system that works with a single file on which this problem has occurred. And the decision to use multiple databases connected by a relationship of 1 to 1 would require a fairly significant rework of the system.

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#22 Post by Eugene Lutsenko »

Thank you so much, Roger!

I'll use your solution to my problems.
I hope it will be useful not only to me.

Thank you so much, Roger! I just checked your decision. Everything works fine as usual. I will use!!!

User avatar
rdonnay
Site Admin
Posts: 4722
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

Re: Sampling data from CSV to DBF

#23 Post by rdonnay »

I don't know how well my solution will work for you.
You will run into some limitations in coding style, of course, for the following reasons:

Having 5 different work areas (dbfs) requires that you know which file a particular field exists. This can be found by using dbStruct() to acquire the structure of each dbf and then using AScan() to find the field. Due to the fact that some of your field names start with a numeric character, this prevents access to the data via macro calls, therefore you cannot use DC_DbRecord(). You must ALWAYS use FieldGet() and FieldPut().

Example:

Code: Select all

aStru := { TRAIN1->(dbStruct()), TRAIN2->(dbStruct(), TRAIN3->(dbStruct()), TRAIN4->(dbStruct(), TRAIN5->(dbStruct)) }

// Get the value of the field 'AF2389BC' 

nValue := GetValue( aStru, 'AF2389BC', @nFieldPos )
IF !Empty(cAlias)
   nValue := (cAlias)->(FieldGet(nFieldPos))
ENDIF
* ---------
FUNCTION GetValue( aStru, cFieldName, nFieldPos )
LOCAL cAlias, i, xValue
FOR i := 1 TO Len(aStru)
  nFieldPos := AScan(aStru[i],{|a|a[1]==Upper(cFieldName)})
  IF nFieldPos > 0
     cAlias := 'TRAIN' + Alltrim(Str(i))
     xValue := (cAlias)->(FieldGet(nFieldPos))
     EXIT
  ENDIF
NEXT

RETURN xValue
The eXpress train is coming - and it has more cars.

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#24 Post by Eugene Lutsenko »

Of course several tables even with a ratio of one to many is more difficult to use than one. Moreover, in fact, they may be a different number in different tasks. But I will try to modify the system over time so as to remove this restriction on the number of scales that occur with one table

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#25 Post by Eugene Lutsenko »

Thanks, Roger for everything!

I'll tailor Your solution to my conditions. It's working. Of course, this will require further development in other modes of the system. But worth it. Sometimes I have some questions.

For example, in the DC_TxtLine () command in Your example, there is a 3rd parameter = 100000. I looked in the description of this function is only 2 parameters. Intuitively, it seems to me that this is something like the maximum number of characters in the line being read. Could you clarify the meaning of this parameter.

Code: Select all

cLine   := DC_TxtLine(nHandle,Chr(10),100000)
Also, Roger, I'd like to say that your decision is based on an extraordinary thought (supremely outstanding idea). This idea deserves to be implemented at the level of a programming language or libraries. Imagine that in Express there is a command like:

Code: Select all

USE NameDBF EXLUSIVE NEW MULTITABLE
You can create such a multi-table using the following commands:

Code: Select all

aStru := { TRAIN1->(dbStruct()), TRAIN2->(dbStruct(), TRAIN3->(dbStruct()), TRAIN4->(dbStruct(), TRAIN5->(dbStruct)) }
dbCreate(NameDBF ,aStru,'MULTITABLE')
And you can access such a multi-table using your function and just commands to access normal tables, including indexes, filters, and everything else. I think it would be very cool. In such a multi-table there would be no limit to the number of fields and the file size of the multi-table itself could not be limited to 2 GB at all. A multi-table is a special case of a regular table. So if the teams work with the usual table, add the option "MULTITABL" in her work, nothing will change. Determine the name of the table and the number of the field in it by the number of the field in the multi-table and refer to this field.
Attachments
The numbers of table and number field number field in multitable.zip
(7.01 KiB) Downloaded 697 times

User avatar
Auge_Ohr
Posts: 1405
Joined: Wed Feb 24, 2010 3:44 pm

Re: Sampling data from CSV to DBF

#26 Post by Auge_Ohr »

Eugene Lutsenko wrote:For example, in the DC_TxtLine () command in Your example, there is a 3rd parameter = 100000. I looked in the description of this function is only 2 parameters.
have a look in \Source\Dclipx\_dctxt.prg
Eugene Lutsenko wrote:

Code: Select all

USE NameDBF EXLUSIVE NEW MULTITABLE
You can create such a multi-table using the following commands:
i do not think anybody else need so many fields.
greetings by OHR
Jimmy

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#27 Post by Eugene Lutsenko »

Auge_Ohr wrote:
Eugene Lutsenko wrote:For example, in the DC_TxtLine () command in Your example, there is a 3rd parameter = 100000. I looked in the description of this function is only 2 parameters.
have a look in \Source\Dclipx\_dctxt.prg
Eugene Lutsenko wrote:

Code: Select all

USE NameDBF EXLUSIVE NEW MULTITABLE
You can create such a multi-table using the following commands:
i do not think anybody else need so many fields.
Thanks, Jimmy! That's pretty much what I understood. And tables larger than 2 GB can be useful to someone?

User avatar
Auge_Ohr
Posts: 1405
Joined: Wed Feb 24, 2010 3:44 pm

Re: Sampling data from CSV to DBF

#28 Post by Auge_Ohr »

Eugene Lutsenko wrote:And tables larger than 2 GB can be useful to someone?
each DBF is still limited to 2GB :naughty:
... you just have more than 256 Fields.

if you have so many Data you should think about

a.) other Database than DBF
b.) compress Data

ad a.) even in SQL, i guess (never try), you can't have so many Fields
ad b.) Bitmap
greetings by OHR
Jimmy

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Sampling data from CSV to DBF

#29 Post by Eugene Lutsenko »

Jimmy!

The multi-table I'm talking about is one table only logically. And physically it can be one or several DBF-tables, each of which is less than 2 GB. This can be tens or hundreds of DBF tables, and each can have up to 1,500 fields. It turns out that the size of the multi-table is not limited to the size of 2 GB. Sorry for the automatic translation.

Post Reply