My Parallel Sort Library and benchmarks ...

[JuhaManninen]

>An unhandled exception occurred at $0809F010 :
>EAccessViolation : Access violation
>  $0809F010  GETTHREADID,  of LockFreePrimitives.pas
>  $0809E9C2  TGJRINGBUFFER__MEASUREEXECUTIONTIMES,  line >240 of RingBuffer.pas

[...]
So, if you want to run it on  Ubuntu 9.10 just
delete the 'uses parallelqueuerb,ringbuffer' inside
threadpool.pas, after that compile it and tell me
if it's working on  Ubuntu 9.10 correctly.

I think the same applies to any Linux, not just Ubuntu. I have OpenSuse.
The code doesn't even compile as is.
Amine, if you plan to support other platforms than Windows, please do the following changes:
1. Replace "IFDEF FreePascal" with "IFDEF FPC".
2. Units are not found in a casesensitive file system. Either rename all units to lowercase or match their (casesensitive) names exactly in USES section.

After those changes I could compile and run and then I also got Access violation.
Then I commented out {parallelqueuerb,ringbuffer} as you suggested and got an exception + SIGSEGV in TThreadPool.Terminate on line :
  for I := 0 to FThreadCount - 1 do FThreads.Terminate;

This code is not inherently Windows dependent so I think it is worth porting and testing it under other platforms, too.

Juha

[aminer]

 JuhaManninen wrote
>I think the same applies to any Linux, not just Ubuntu. I have >OpenSuse.
>The code doesn't even compile as is.
>Amine, if you plan to support other platforms than Windows, >please do the following changes:
>1. Replace "IFDEF FreePascal" with "IFDEF FPC".
>2. Units are not found in a casesensitive file system. Either rename >all units to lowercase or match their (casesensitive) names exactly >in USES section.

 I understand , and i will make those changes..


>After those changes I could compile and run and then I also got >Access violation.
>Then I commented out {parallelqueuerb,ringbuffer} as you >suggested and got an exception + SIGSEGV in >TThreadPool.Terminate on line :
>  for I := 0 to FThreadCount - 1 do FThreads.Terminate;
>This code is not inherently Windows dependent so I think it is >worth porting and testing it under other platforms, too.

That's strange, why are you getting SIGSEGV on
FThreads.Terminate ?

Why it's working on Windows and not working on Linux ?

I don't have Linux or MacOSX , to test it here.

I will think more about this...


Amine.




 

[JuhaManninen]

That's strange, why are you getting SIGSEGV on
FThreads.Terminate ?

Why it's working on Windows and not working on Linux ?

I don't have Linux or MacOSX , to test it here.

Actually the exception happens before FThreads.Terminate, inside a thread I guess.
The cursor was just placed on FThreads.Terminate.
Debugging threaded code can be little tricky.

Well, Linux is free and easy to install (nowadays). :-)

Have you considered publishing the code on a revision control system, to make contributions easier?

Juha

[jarto]

>As i have noticed, you are using this:
>TestParallelSort(10000,1,4); //Tests 10000 names with 1 -> 4 cores.
>As i said before, the number of cores must be 2^N: 1,2,4 etc. 
>So, you have to avoid *3*, just: 1,2,4,6,8 etc.

I have reproduced your problem with a number of cores
equal to 3.

My first test case (which I made blind) didn't notice that. However, my second test case takes care of avoiding cores 3, 5 etc..

I was able to reproduce the sorting problem by running my test program many times in a bat file:

test.bat:

Code: [Select]

parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 1000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 20000
diff results-1.txt results-2.txt
parallelsorttest 20000
diff results-1.txt results-2.txt
parallelsorttest 30000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 60000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt
parallelsorttest 100000
diff results-1.txt results-2.txt

When I run the bat file, in some test runs I get a lot of output from the diff program.

Just out of curiosity, could you post here some results from your 4 core CPU using my test program?

[aminer]

JuhaManninen wrote:
>Actually the exception happens before FThreads.Terminate, inside >a thread I guess.
>The cursor was just placed on FThreads.Terminate.
>Debugging threaded code can be little tricky.

>Well, Linux is free and easy to install (nowadays). :-)


Can you advice me where i can download a *good* version
of Linux from the *Internet* - for example a .iso image fo dvd
or cd - , so that i can test my libraries on it ?


>Have you considered publishing the code on a revision control >system, to make contributions easier?


Not yet.


>Juha


Aminw.

[aminer]

jarto wrote
>My first test case (which I made blind) didn't notice
>that. However, my second test case takes care of
>avoiding cores 3, 5 etc..

>I was able to reproduce the sorting problem by running
>my test program many times in a bat file:


Are you using Parallel QuickSort ?

Tell me if you can reproduce the same thing with Parallel ctHeapSort?


>Just out of curiosity, could you post here some results
>from your 4 core CPU using my test program?

My hardware is too fast that it gives  00:00:000 on 2 cores.




Amine.

[aminer]

jarto write:
>I was able to reproduce the sorting problem by running my test >program many times in a bat file:

I see. It is the 'merge' procedure that is not working
properly , i will take a look at it , and see what's the
problem...


Take care.

Amine.

[JuhaManninen]

Can you advice me where i can download a *good* version
of Linux from the *Internet* - for example a .iso image fo dvd
or cd - , so that i can test my libraries on it ?

The popular desktop distros: Ubuntu, OpenSuse, Fedora, Mandriva
are all good choices. You can find them by simple google search or such.
They provide live CD / DVDs so you can test before installing.
They support dual boot and your Windows still remains functional.
If you have lots of memory you can even run different operating systems under virtual machines at the same time.

Juha

[aminer]

jarto wrote:
>My first test case (which I made blind) didn't notice that. >However, my second test case takes care of avoiding cores 3, 5 >etc..

>I was able to reproduce the sorting problem by running my test >program many times in a bat file:


I have tested it on Delphi 7 and i think this problem doesn't
show on Delphi 7.

Can you please test it on delphi and confirm to me  ?



Amine.


 

[jarto]

Tell me if you can reproduce the same thing with Parallel ctHeapSort?

Yes, I can reproduce the same problem with ctHeapSort. It can also be reproduced with Delphi 7.

My hardware is too fast that it gives  00:00:000 on 2 cores.

Just increase the number of names to sort. You can specify the namecount and cores to test on this line:

TestParallelSort(NameCount, 1, 4)

And make sure to code from my 2nd version:

http://www.starsoft.fi/jarto/ParallelSortTest2.zip

/jarto

[aminer]

jarto wrote:
>Yes, I can reproduce the same problem with ctHeapSort. It can >also be reproduced with Delphi 7.


Here it is jarto , i have solved the problem, you can
download version 2.11 from my website:

http://pages.videotron.com/aminer/


Please test it  and tell me if all is ok...

Note: the problem was not on my algorithm , it was
         very difficult to spot the problem ...


Sincerely,
Amine.

[aminer]

Please JuhaMannin, can you test version 2.11 and
tell me if you have the same exception + SIGSEGV
in Linux ?



Amine.

[JuhaManninen]

Please JuhaMannin, can you test version 2.11 and tell me if you have the same exception + SIGSEGV in Linux ?

I downloaded:
 http://pages.videotron.com/aminer/zip/parallelsort.zip
I hope the link is correct.
Then I had to make the same changes as before to make it compile and run.
And Yes, I got the same exception + SIGSEGV still.

I have no idea what you have changed because I had to download the whole package and replace the old files.

Amine, you should really consider putting your code into a revision control system. It would help you to track your changes. It would help any tester (like me now) to see the changes. It would allow someone to send you patches. You would still have full control over your code, nobody could change it without your permission.
If you for example open a GitHub repository, I promise to fork it and send you some changes.

Juha

[aminer]

Hello JuhaMannine,

I have installed OpenSUSE , but when i have tried to
install FreePascal it complains that libinfo.so is not present,
in wich package i can find this dynamic library ?


Amine.

[JuhaManninen]

I have installed OpenSUSE , but when i have tried to install FreePascal it complains that libinfo.so is not present, in wich package i can find this dynamic library ?

There is no such library in Suse. I guess the RPM file is for another distro and you could ignore the dependency and force install.

The easiest way to install FPC 2.2.4 and Lazarus 0.9.28.2 is from Suse repositories (add Packman repo there).
I use the latest SVN versions of FPC and Lazarus myself. That requires installing many development packages. It is not difficult either because they are all found in Suse repositories.

Juha