Increase of calculation speed

This forum is for eXpress++ general support.
Message
Author
User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Increase of calculation speed

#1 Post by Eugene Lutsenko »

I'm developing an Eidos system that does a lot of mathematical calculations. Quite often, the dimensions of the models and the amount of data processed are very large and calculations can last for weeks and months. So far, I have found a solution to this problem by implementing the most time-consuming computations on the GPU. But there are many others that are not migrated to the GPU. Does anyone know the solution to this problem in Alaska? How to significantly increase the speed of calculations in Alaska? Can I use the GPU for computing in Alaska? Currently, calculations on the GPU are implemented in the Eidos system in the form of separate executable modules, which are run as external programs. And I would like them to be implemented directly in the source text in Alaska. Maybe there are some other ways to significantly increase the speed of mathematical calculations in Alaska? The use of GPU accelerates calculations by orders of magnitude. This acceleration depends on the specific task and is usually tens to hundreds or even several thousand times faster.

User avatar
rdonnay
Site Admin
Posts: 4722
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

Re: Increase of calculation speed

#2 Post by rdonnay »

Eugene -

The Xbase++ language is not capable of what you are asking.
You can run a separate Xbase++ compiled .EXE using the GPU, but you cannot invoke an Xbase++ process or thread to utilize a separate CPU or GPU.

You can use SetCPU() at the beginning of an Xbase++ program to tell it which CPU to use for the entire application, but I know of no such function as SetGPU().
The eXpress train is coming - and it has more cars.

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Increase of calculation speed

#3 Post by Eugene Lutsenko »

Thank You, Roger!

If there was an Express such function as SetGPU(), it would increase the value of Alaska by orders of magnitude! I have some apps using the GPU showed a speedup of the calculations is 3000 times! But usually from about 100 times, and sometimes "only" 10. This is very, very important!

User avatar
rdonnay
Site Admin
Posts: 4722
Joined: Wed Jan 27, 2010 6:58 pm
Location: Boise, Idaho USA
Contact:

Re: Increase of calculation speed

#4 Post by rdonnay »

If there was an Express such function as SetGPU()
For many reasons there would never be such a function in eXpress++.

1. eXpress++ is written entirely in Xbase++.

2. Such a function would require that the computer have multiple GPUs in the same way that it has multiple CPUs.

3. Alaska Software would need to incorporate such a function in the Xbase++ language.
The eXpress train is coming - and it has more cars.

User avatar
Tom
Posts: 1165
Joined: Thu Jan 28, 2010 12:59 am
Location: Berlin, Germany

Re: Increase of calculation speed

#5 Post by Tom »

But there might be a way to address something like OpenCL from Xbase++-applications. Or CUDA. Those libraries can be accessed from C+/++-applications.
Best regards,
Tom

"Did I offend you?"
"No."
"Okay, give me a second chance."

Victorio
Posts: 620
Joined: Sun Jan 18, 2015 11:43 am
Location: Slovakia

Re: Increase of calculation speed

#6 Post by Victorio »

I am also interesting about it. I am using system with running paralell processes on all cores CPU calling as external modules. So I have in my older PC only 2 cores, this increase speed about 2 times, on new PC with Intel i7 i have 8 cores, this is better about 8 x :)
And on central server where processing all databases from country is 12 CPU but this is virtual server.
I can "call" administrator to increase CPU to more cores, but better way is optimalize program. So there are some limits, and "only" 10x 20x ... better speed, and not possible 100x ... 1000x .

I do not know if you can also break processes to several parts and processing paralell ?

But volume of database every year increasing, that I think about create module to speed operations in C++ because my old DOS program in CA Clipper, where I used C++ exe modules was very speed and in Xbase still can not get this speed.
If GPU can use and can run exe module with using it, this will be very interesting. So I do not know how.
So if it is true that can use 100-1000 x better speed with GPU,

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Increase of calculation speed

#7 Post by Eugene Lutsenko »

Victorio wrote:If GPU can use and can run exe module with using it, this will be very interesting. So I do not know how.
So if it is true that can use 100-1000 x better speed with GPU,
On my site visit: http://lc.kubagro.ru/aidos/_Aidos-X.htm posted a full installation of the system Eidos. This system refers to the GPU recognition module as an external program. Developer: Dmitry Bandyk from Belarus. This gives an acceleration of classification processes up to 3000 times. This is a really fixed acceleration. But usually the acceleration is less than about 100 times. I have a simple video card NVIDIA-GT240, which has only 96 cores. But this is ten times more than the computing cores of the Central processor (CPU). And on the most powerful NVIDIA graphics cards about 3000 Shader processors. This GPU-recognition module is part of the installation of the Eidos system. The source code of the system is here: http://lc.kubagro.ru/__AIDOS-X.txt! Running the GPU recognition engine looks like this: LC_RunShell ("Model_rec.exe", 90392051). A start-up function LC_RunShell() it is easy to find by searching: "n LC_RunShell(". The function checks the presence of the launched module and checks its checksum (for security reasons). If the module is not in the current directory or its checksum does not match, a message is displayed. if everything is normal, the GPU module is started for execution. As the source databases using a database system Eidos. As a result of his work, too, obtained database system Eidos. By input and output, it is no different from the recognition function, which implements the same operations on the CPU.

Here is an article describing the research that uses this technology: http://ej.kubagro.ru/2018/10/pdf/33.pdf. Without the use of this technology, such research is almost impossible.

FORMATION OF A SEMANTIC KERNEL IN VETERINARY MEDICINE WITH THE AUTOMATED SYSTEM-COGNITIVE ANALYSIS OF PASSPORTS OF SCIENTIFIC SPECIALTIES OF THE HIGHER ATTESTATION COMMISSION OF THE RUSSIAN FEDERATION AND THE AUTOMATIC CLASSIFICATION OF TEXTS ACCORDING TO THE AREAS OF SCIENCE

This work is a continuation of the author's series of works on cognitive veterinary medicine. The present period is characterized by the appearance of huge vol-umes of texts in different languages in the open access, generated by people. Currently, these texts are accu-mulated in various electronic libraries and biblio-graphic databases (WoS, Scopus, RSCI, etc.), as well as on the Internet on various sites. All these texts have specific authors, dates and can belong simultaneously to many non-alternative categories and genres, in par-ticular: educational; scientific; artistic; political; news; chats; forums and many others. The solution of the generalized problem of attribution of texts is of great scientific and practical interest, i.e. studying these texts, which would reveal their probable authors, date of creation, the ownership of these texts to the above generalized categories or genres, and might evaluate the similarities - differences of authors and texts ac-cording to their content, highlight key words etc. To solve all these problems it seems necessary to form the generalized linguistic images of texts into groups (classes), i.e. to form semantic kernels of classes. A special case of this problem is the creation of the se-mantic kernel in various scientific specialties of the HAC of the Russian Federation and the automatic classification of scientific texts in the areas of science. Traditionally, this task is solved by dissertation coun-cils, i.e. experts, on the basis of expert assessments, i.e. in an informal way, on the basis of experience, intui-tion and professional competence. However, the tradi-tional approach has a number of serious drawbacks that impose significant limitations on the quality and volume of analysis. Currently, there are all grounds to consider these restrictions as unacceptable, because they can be overcome. Thus, there is a problem, the solutions of which are the subject of consideration in this article. Therefore, the efforts of researchers and developers to overcome them are relevant. Therefore, the aim of the work is to develop an automated tech-nology (method and tools), as well as methods of their application for the formation of the semantic core of veterinary medicine by automated system-cognitive analysis of passports of scientific specialties of the HAC of the Russian Federation and automatic classifi-cation of texts in the areas of science. A detailed nu-merical example of solving the problem on real data has been given as well.
Last edited by Eugene Lutsenko on Wed Jan 16, 2019 9:52 am, edited 2 times in total.

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Increase of calculation speed

#8 Post by Eugene Lutsenko »

Tom wrote:But there might be a way to address something like OpenCL from Xbase++-applications. Or CUDA. Those libraries can be accessed from C+/++-applications.
This is how it is done, but it is not implemented in a program written in Alaska, but in an external program. And a program written in Alaska runs this external program for execution. In principle, this is no big deal. In this way, it is possible to use the GPU for non-graphical calculations from a program written in Alaska. This is already used in the Eidos system. And this qualitatively changed its capabilities.

Victorio
Posts: 620
Joined: Sun Jan 18, 2015 11:43 am
Location: Slovakia

Re: Increase of calculation speed

#9 Post by Victorio »

This GPU-recognition module is part of the installation of the Eidos system. The source code of the system is here: http://lc.kubagro.ru/__AIDOS-X.txt! Running the GPU recognition engine looks like this: LC_RunShell ("Model_rec.exe", 90392051). A start-up function LC_RunShell()
Eugene, this is very important, I must look for this. thanks for info.

User avatar
Eugene Lutsenko
Posts: 1649
Joined: Sat Feb 04, 2012 2:23 am
Location: Russia, Southern federal district, city of Krasnodar
Contact:

Re: Increase of calculation speed

#10 Post by Eugene Lutsenko »

Victorio wrote:
This GPU-recognition module is part of the installation of the Eidos system. The source code of the system is here: http://lc.kubagro.ru/__AIDOS-X.txt! Running the GPU recognition engine looks like this: LC_RunShell ("Model_rec.exe", 90392051). A start-up function LC_RunShell()
Eugene, this is very important, I must look for this. thanks for info.
Victorio! If you want, I can introduce you to Dmitri. He has already learned to work well with DBF on GPU.

Post Reply