Recent

Author Topic: Slower string comparison with newer version  (Read 6837 times)

dhyani

  • Newbie
  • Posts: 5
Slower string comparison with newer version
« on: April 02, 2012, 10:11:04 pm »
Hi,

are there any special compiler switches/options that might influence the speed of string comparisons (strX1 = strX2)??

I scripted a small test app that does about 10 million string comparisons. Compiled with (all out of the box (CodeTyphon))
   Lazarus 0.9.31 (SVN 19-08-2011 Rev 32047)
   FPC 2.7.1 (SVN 19-08-2011 Rev 18872)
   --> Runtime: 5 to 5.5 seconds

   Lazarus 0.9.31 (SVN 22-2-2012 Rev 35557)
   FPC 2.7.1 (SVN 22-2-2012 Rev 20397)
   --> Runtime: 185 - 190 seconds  :o :o :o

Any piece of advice would be very appreciated :)



   

KpjComp

  • Hero Member
  • *****
  • Posts: 680
Re: Slower string comparison with newer version
« Reply #1 on: April 04, 2012, 12:04:11 am »
I wonder if there has been any changes in the way that Lazarus detects multi-threading.

You could try the following on your application startup, and see if speed comes back.

Code: [Select]
IsMultiThread := false;

Of course be careful if you use this if your are creating a multi-threading app and passing strings about.

BigChimp

  • Hero Member
  • *****
  • Posts: 5740
  • Add to the wiki - it's free ;)
    • FPCUp, PaperTiger scanning and other open source projects
Re: Slower string comparison with newer version
« Reply #2 on: April 04, 2012, 07:39:04 am »
Don't know at what time the Unicode string routines were merged into 2.7.1. That could definitely have an impact on performance.
Want quicker answers to your questions? Read http://wiki.lazarus.freepascal.org/Lazarus_Faq#What_is_the_correct_way_to_ask_questions_in_the_forum.3F

Open source including papertiger OCR/PDF scanning:
https://bitbucket.org/reiniero

Lazarus trunk+FPC trunk x86, Windows x64 unless otherwise specified

icey

  • New Member
  • *
  • Posts: 19
Re: Slower string comparison with newer version
« Reply #3 on: September 16, 2012, 11:22:29 pm »
I'm kinda having the same problem here. I updated my lazarus/fpc installation and one of my programs - that's mostly for evaluating large (>10 GByte) files - became like 20-times slower.

It is basically using this code to split the lines inside these files:

Code: [Select]
program StringCompare1;

{$mode objfpc}{$H+}

uses
  Classes, lclintf, StrUtils, SysUtils;

var
  pch1         : Array[0..999] of string;
  P, P1        : PChar;
  key, value   : string;
  i, j, cnt    : integer;
  Start, Stop  : DWord;

begin
  For i := 0 to High(pch1) Do
    pch1[i] := 'key1=value1|key2=value2|key3=value3|key4=value4|key5=value5|key6=value6';

  cnt := 0;
  Start := GetTickCount;
  For i := 0 to High(pch1) Do
    For j := 0 to High(pch1) Do
      begin
      P := PChar(pch1[i]);
      While P^ <> #0 Do
        begin
        P1 := P;
        While (P^ > #31) AND (P^ <> '|') AND (P^ <> '=') Do
          Inc(P);

        If P^ = '=' Then
          SetString(key, P1, P - P1)
        else If P^ = '|' Then
          begin
          SetString(value, P1, P - P1);
          If (key = 'key2') Or (key = 'key4') Then Inc(cnt);
          end;
        Inc(P);
        end;
      end;

  Stop := GetTickCount;

  WriteLn (IntToStr(Stop-Start) + 'ms');
  WriteLn (IntToStr(cnt));
  readln;
end.

That little programm takes about 700ms compiled with the old fpc and 15seconds compiled with the new version.

I even tested a simple
Code: [Select]
for i := 0 to 100000 do "str = str2"; and that is still about 3 times faster on the older fpc  :o

So is there any way to disable the unicode-support, some kind of workaround or any improvement to my code?

bigeno

  • Sr. Member
  • ****
  • Posts: 266
Re: Slower string comparison with newer version
« Reply #4 on: September 17, 2012, 09:18:11 am »
that is why I still use 2.6 + fixes. cwstring  :(

ludob

  • Hero Member
  • *****
  • Posts: 1173
Re: Slower string comparison with newer version
« Reply #5 on: September 17, 2012, 09:25:56 am »
The problem is not the unicode support but the new codepage aware string format. Whenever the compiler thinks that the codepages in an assignment or comparison are different a conversion is done. In this case the compiler detects that key is cp 1252 and 'key2' is cp 0 (CP_ACP). On windows the default codepage (CP_ACP) is 1252 but fpc_AnsiStr_Compare_equal still decides to convert both to utf8 before comparison. Hence the drop in performance.
A workaround is to force the default string cp to CP_ACP:

Code: [Select]
begin
  DefaultSystemCodePage:=CP_ACP;  // add this line
  For i := 0 to High(pch1) Do
....
Code is taking now 3280ms on my slow system. Was 20000 before. 2.6.1 takes 1800 ms.

I'll submit a bug report for this later on.

EDIT: reported in http://bugs.freepascal.org/view.php?id=22906
« Last Edit: September 17, 2012, 11:17:28 am by ludob »

icey

  • New Member
  • *
  • Posts: 19
Re: Slower string comparison with newer version
« Reply #6 on: September 17, 2012, 12:02:33 pm »
Thanks for the DefaultSystemCodePage := CP_ACP hint - it really speeds things up a lot.

So - after a long night with lots of testing i replaced the "str1 = str2" with "CompareStr(str1, str2) = 0" and the SetString procedure with

Code: [Select]
procedure mySetString(out outStr: ansistring; srcStr: PChar; len: Integer); inline;
var
  i : Integer;
begin
  SetLength(outStr, len);
  For i := 1 to len Do
    begin
    outStr[i] := srcStr^;
    inc(srcStr);
    end;
end; 

This way performance loss is only about 10-15% - and i think i can live with that  ::)

 

TinyPortal © 2005-2018