redirecting standard output to TMemo in real-time

[j0x]

im modifying my GUI app that calls a CLI app to redirect its standard output
to the TMemo of the GUI app in real time, but its just doing nothing, their is no
errors too, the documentation of the console app im using stated this

Code: [Select]

If you want to monitor HandBrake's process, you should monitor the standard pipes.

    * Standard Output: Contains encode progress information.
    * Standard Error: Contains all the logging data.
source: https://trac.handbrake.fr/wiki/CLIGuide

i want the encode progress info be shown on TMemo in real time so that it will
serve as some sort of progress bar or progress indicator, that is why
im piping the standard output but so far i failed, here are parts of the code in question

Code: [Select]

var
    S: TStringList;
    M: TMemoryStream;
    n: longInt;
    BytesRead: longInt;
const
    READ_BYTES = 2048; 
//////////////////////////////////
//Hbk is the TProcess
Hbk.CommandLine := lstOut.Items[i];
Hbk.Options := [poUsePipes]; 
Hbk.Execute;   
    S := TStringList.Create;
          M := TMemoryStream.Create;
          while Hbk.Running do
          begin
               M.SetSize(BytesRead + READ_BYTES);
               n := Hbk.Output.Read((M.Memory + BytesRead)^, READ_BYTES);
               if n > 0 then
               begin
                    Inc(BytesRead,n);
               end;
          end;

          repeat
                M.SetSize(BytesRead + READ_BYTES);
                n := Hbk.Output.Read((M.Memory + BytesRead)^, READ_BYTES);
                if n > 0 then
                begin
                     Inc(BytesRead,n);
                end;
          until n <= 0;

          M.SetSize(BytesRead);
          S.LoadFromStream(M);
  //mmoOut is the TMemo
          mmoOut.Lines := S;
          //mmoOut.Lines.LoadFromStream(M);
          M.Free;
          S.Free;       

on the other question thread i made i got a reply that says this

The article about capturing standard output and sending data to it's standard input worked fine for me on XP (http://wiki.lazarus.freepascal.org/Executing_External_Programs#How_to_redirect_output_with_TProcess).
I used threading though to make it run in parallel with the main thread.

and im new to this standard streams, but did he mean the data i will read on standard output of HandbrakeCLI.exe will have to be send to the standard input of HandbrakeCLI.exe? then if so how can i get the data on standard input and show it on TMemo of my GUI app in real-time?

[eny]

on the other question thread i made i got a reply that says this...

Sry for the confusion, I should have given the link a few paragraphs up: http://wiki.lazarus.freepascal.org/Executing_External_Programs#An_Improved_Example.

Didn't you manage to get this working alread (see http://www.lazarus.freepascal.org/index.php/topic,10898.0.html)?

[j0x]

Didn't you manage to get this working alread (see http://www.lazarus.freepascal.org/index.php/topic,10898.0.html)?

yes i have finished that one, but the problem on that program is that it will only output the contents of the TProcess once TProcess exited

now i want to make some sort of progress indicator that will show the encode progress of handbrakeCLI through TMemo in real-time

Sry for the confusion, I should have given the link a few paragraphs up: http://wiki.lazarus.freepascal.org/Executing_External_Programs#An_Improved_Example.

ye ive tried that too but no luck so far

[eny]

yes i have finished that one, but the problem on that program is that it will only output the contents of the TProcess once TProcess exited

You don't do anything while the external process is running.
Change your while loop and do some of the processing in there:

Code: Pascal  [Select][+][-]

1.   while Hbk.Running do
2.     begin
3.       M.SetSize(BytesRead + READ_BYTES);
4.       n := Hbk.Output.Read((M.Memory + BytesRead)^, READ_BYTES);
5.       if n > 0 then
6.         begin
7.           Inc(BytesRead,n);
8.           //
9.           // --> DO YOUR PROCESSING HERE TO SHOW THE PROGRESS <--
10.           //
11.          end
12.       else
13.         sleep(100);  // <-- Better leave this in place  
14.    end;

[j0x]

thanks eny ill try those suggestions, ill be back tomorrow if im successful or got some more questions

[j0x]

the TMemo is now being populated in real-time but the text are all question marks and sort of encrypted characters like this ->  êæ?????êæ??êæ???

i think the problem is on reading the TMemoryStream or MM, this is the current code i have now

Code: [Select]

//Hbk is TProcess
while Hbk.Running do
          begin
               MM.SetSize(BBytesRead + RREAD_BYTES);
               nn := Hbk.Output.Read((MM.Memory + BBytesRead)^, RREAD_BYTES);
               if nn > 0 then
               begin
                    Inc(BBytesRead,nn);
                    //SS is TStringlist
                    SS.LoadFromStream(MM);
                    //mmoOut is TMemo
                    mmoOut.Lines.Add(SS.Strings[SS.Count-1]);
               end else begin
                    sleep(100);
               end;
          end;               

so need help again

[eny]

To be honest the Wiki needs an update.
The below example does what you want. Not the most efficient (copying ansistrings to a TMemo repeatedly) but try to onderstand what happens !!

The key thing is that you process raw data that comes in at irregular intervals and you need to append it to the raw data you already received (the text in your memo control).
You cannot just assume that the last line in your stringlist is exactly the text that was received from the process.

Code: Pascal  [Select][+][-]

1. const
2.   C_BUFSIZE = 2048;
3. var
4.   Hbk: TProcess;
5.   Buffer: pointer;
6.   SStream: TStringStream;
7.   nread: longint;
8. begin
9.   Hbk := TProcess.Create(nil);
10.  
11.   // Replace the line below with your own command string
12.   Hbk.CommandLine := 'c:\windows\system32\cmd.exe /c dir c:\windows\*.*';
13.   //
14.  
15.   Hbk.Options := [poUsePipes];
16.   Hbk.Execute;
17.  
18.   // Prepare for capturing output
19.   Getmem(Buffer, C_BUFSIZE);
20.   SStream := TStringStream.Create('');
21.  
22.   // Start capturing output
23.   while Hbk.Running do
24.   begin
25.     nread := Hbk.Output.Read(Buffer^, C_BUFSIZE);
26.     if nread = 0 then
27.       sleep(100)
28.     else
29.       begin
30.         // Translate raw input to a string
31.         SStream.size := 0;
32.         SStream.Write(Buffer^, nread);
33.         // And add the raw stringdata to the memo
34.         Memo1.Lines.Text := Memo1.Lines.Text + SStream.DataString;
35.       end;
36.   end;
37.  
38.   // Capture remainder of the output
39.   repeat
40.     nread := Hbk.Output.Read(Buffer^, C_BUFSIZE);
41.     if nread > 0 then
42.     begin
43.       SStream.size := 0;
44.       SStream.Write(Buffer^, nread);
45.       Memo1.Lines.Text := Memo1.Lines.Text + SStream.Datastring;
46.     end
47.   until nread = 0;
48.  
49.   // Clean up
50.   Hbk.Free;
51.   Freemem(buffer);
52.   SStream.Free;
53. end;

[j0x]

it worked! i just added poStderrToOutPut on the TProcess.Options

im gonna try to understand how it works though and thanks a lot 

[bmatthewshea]

it worked! i just added poStderrToOutPut on the TProcess.Options
im gonna try to understand how it works though and thanks a lot 

Yes that 'works' but this doesn't update the TMemo in 'real time' for me. Just add this option for instance so the DIR command isn't 'instantaneous':

Code: [Select]

Hbk.CommandLine := 'c:\windows\system32\cmd.exe /c dir /s c:\windows\*.*';  // (/s option added to increase data)
For me at least this does the same thing as the"Reading large output" code from Wiki.. just hangs for a bit and then adds to my  Memo all at once.

I need to use a memo for some possibly very large output... depending on file and I am using "poNoConsole" option to hide console, so I need "real time" output if possible. Thanks for any help.

[Arbee]

@bmatthewshea ... maybe it helps if you intersperse it with "Application.ProcessMessages" calls?

[bmatthewshea]

@bmatthewshea ... maybe it helps if you intersperse it with "Application.ProcessMessages" calls?

I had tried that early on, but just tried it again since code had changed quite a bit. No luck (app is Win32 only BTW)

[eny]

Yes that 'works' but this doesn't update the TMemo in 'real time' for me. Just add this option for instance so the DIR command isn't 'instantaneous':

The TMemo is updated in real time, just not displayed in real time hence the required sync with the LCL.
How do you synchronize with the LCL?

[bmatthewshea]

The TMemo is updated in real time, just not displayed in real time hence the required sync with the LCL.
How do you synchronize with the LCL?

? "Application.ProcessMessages;"

Doh. Just looked through code. Had a 'BeginUpdate' sticking around I thought was commented out. :| Anyway, thanks.. So.. some of my code now:

Code: [Select]

      begin
        SStream.size := 0;
        SStream.Write(Buffer^, nread);
        Form2Output.MemoOutput.Lines.Text := Form2Output.MemoOutput.Lines.Text + SStream.DataString;
      end;
// Added this--
  Application.ProcessMessages;
  Form1.MemoOutput.SelStart := Length(Form1.MemoOutput.Lines.Text);
//
  end;

Memo is now updating correctly (and quickly) and scrolling as it does...
Thanks all...