Monday, January 29, 2007

String Concatenation

Recently i have a problem with string concatenation. In Delphi, standard string concat (s:=s+'newstring') just doing fine on short string. Once the string grows up to 1-3 Mb, that would be a problem. The application might stuck in random occasion. The task manager shows a 99% CPU usage which means the application is freezing for 10-30 seconds... given example (Hex Converter) :

const HexConvert: array['0'..'f'] of SmallInt =
( 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,-1,-1,-1,-1,-1,-1,
-1,10,11,12,13,14,15,-1,-1,-1,-1,-1,-1,-1,-1,-1,
-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,
-1,10,11,12,13,14,15);
function HexToStr(str: string): string;
var
i,iLen:integer;
s:string;
begin
i := 1;
iLen := length(str);
try
while i <= iLen do begin
j := i + 1;
//---problem here---//
s:= s + chr((ord(HexConvert[str[i]]) shl 4) or ord(HexConvert[str[j]])); //------------------//
inc(i,2);
end;
finally
Result := s;
end;
end;

everytime s concated with new string, your compiled app will reallocate memory for s to handle the updated length of s. this is done repeatedly and fast, while the reallocate process itself isn't just a simple routine. sometimes under some circumtances, it will locked up the app, So the strategy to overcome this problem is to allocate some fixed size of memory needed to do the operation once, then fill s[index] := desired_character one by one until it reach the last index of the s. The Fixed HexToStr would be like this :

function HexToStr(str: string): string;
var
i,j,k,l:integer;
s:string;
begin
i := 1;
k := 1;
l := length(str);
SetString(s,nil,l div 2 + 1);
try
while i <= l do begin
j := i + 1;
s[k]:= chr((ord(HexConvert[str[i]]) shl 4) or ord(HexConvert[str[j]]));
inc(k);
inc(i,2);
end;
finally
Result := s;
SetString(s,nil,0);
end;
end;

This will perform 5-10 times faster than before. thanks to Mr. Hans Gulo from Delphindo mailing list

No comments: