From graphics-request at octave dot org Sat Mar 11 12:10:27 2006 Subject: Re: GUI thoughts From: "Sebastien Loisel" To: "John W. Eaton" Cc: "octave graphics and gui mailing list" Date: Sat, 11 Mar 2006 19:08:33 +0100 ------=_Part_10571_15064187.1142100513834 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline John, I just returned from Italy and I've started digging around for your method of communicating with Octave. It looks as if openpty does not exist in MinGW, so I won't be able to use it. I don't know if there's going to be an issue with the DLL's fd 1 being different from the EXE's fd 1, I'll see whe= n I try it. I think MinGW has pipes, I don't know what having a PTY instead o= f a pipe for Octave will do, do you suppose that's ok? >From time to time, the GUI wants to go and pry in Octave's innards. While ideally Octave would be thread-safe, for now it would be nice to be backwar= d compatible and not rely on this. One simple workaround is to only pry into Octave's innards when it is safe to do so. One way of doing this would be to have Octave call a function before each prompt. From within this function, I could dump all the info from Octave into my GUI. Another method would be to be to read a variable or call a function which would tell me whether Octave is running code or waiting for input. Or, if Octave has an idle() function (like GLUT) which it calls whenever it has some spare cycles, I could hook into that to update the GUI= . Are there backwards-compatible ways of doing this? Cheers, S=E9bastien Loisel On 2/28/06, John W. Eaton wrote: > > I have some thoughts and questions about how we would like a GUI for > Octave to work. Mostly, they are about how the GUI and Octave should > be linked together, who should be in charge of events, etc. > > In Sebastien's Octave Workshop, I think the GUI handles all events > (correct me if I'm wrong). Octave is embedded in the GUI and gets > input via Octave's eval_string function. Output is captured by > grabbing std::cout in some way. This sort of works, but misses a few > things like handling the diary, automatic function updating based on > the last prompt time, the actual readline library for command-line > editing, etc. If you want to stick with eval_string, then it would > still be possible to handle the all these things, but you have to > duplicate a lot of functionality that is already in Octave. The > eval_string function was not really meant for embedding an interactive > Octave session (for one thing, embedding Octave this way turns off the > internal flag that tells it that it is running in an interactive > session). You could probably work around that too, but I think there > must be a better way. > > Also, if you are embedding Octave with eval_string, I think your GUI > will be unresponsive while Octave is performing computations. Unless > you are running Octave in a separate thread, I think you have to wait > until eval_string returns to continue with the GUI operations. If you > are running Octave in a separate thread and then having the GUI touch > Octave internals while eval_string is running, you will likely have > unexpected results since Octave is not currently thread safe (there > is no mechanism to prevent global data from being modified at the > wrong time). > > To avoid threads, you could start Octave as a coprocess to your GUI, > but you will still need some modifications to Octave so that it can > process requests while it is off running user code. You won't want to > use a pipe for the communication though, because that will not be > recognized as tty, so the interactive features won't work properly. > Opening a pty connection to a separate process is better (Octave will > see the pty as a tty, so it will run in interactive mode). > > There is a terminal widget for GTK (VTE) that would make it easy to > start Octave as a separate process and communicate with it using a > pty. It provides a terminal window that is (more than) sufficient for > running Octave. Running Octave this way, you get readline, Octave > thinks it is interactive, diary works, etc. Perhaps there is > something similar for Qt? > > The difficulty with this approach is that if you want the GUI to do > more than you can get by running Octave in a terminal window, then the > GUI and Octave have to pass some extra information around. I'm sure > that this can be done, but it will mean some extra effort. Will you > parse text data or design a protocol for passing binary data? Will > Octave or the GUI be in charge of that, and how much overhead will it > add? In terms of passing data back and forth, it would be much > simpler if the GUI and Octave were in the same process and the GUI > could just call internal Octave functions to do its thing. > > At least with GTK on Unixy systems, it should be possible to embed > Octave in a terminal widget and make Octave think it is talking to a > tty without having to run Octave as a separate process. To do this, > you will need a thread-safe Octave (but you need that anyway with the > eval_string approach) and a pty implementation. The idea is that > instead of spawning a subprocess, you set up some ptys, dup some file > descriptors, start Octave in a separate thread, then attach the master > pty to the terminal widget. > > Unfortunately, the VTE widget will not allow this mode of operation > out of the box. Fixing it requires the addition of one small > function, but it must be added to the core library because it relies > on some private internal data to work. > > So, some questions and things to think about. > > * Should we work to make Octave thread-safe? I think I would prefer > to work on this instead of a protocol that could be used to > communicate with a separate Octave process. > > * Is there a terminal widget for Qt that could do the same thing as > VTE? I don't really want a GUI for Octave to be useful with just > one toolkit. > > * Would any of this work in a Windows environment? > > I'm appending a simple example program that illustrates this. It > depends on the modified VTE terminal widget and GTK. Building it is a > bit tricky since you need to first patch the VTE sources and build the > modified VTE library. Then you need to ensure that you use that > library for building and running the example program. > > I'm assuming that if we choose this sort of approach, we can get the > necessary changes rolled into the default vte (or other) terminal > widget(s). If not, then I suppose we could provide our own terminal > widget with Octave (though that would definitely not be my > preference). > > jwe > > > First, here is the function that must be added to vte: > > /* We need the following additional function for vte: > > int vte_terminal_set_pty(VteTerminal *terminal, int pty_master); > > It must be added to vte because it calls static private functions > in the vte library. > > This function was written by Eric Smith , > archived here: > > http://www.brouhaha.com/~eric/software/vte/vte-0.11.10-add-pty.patch > > as a patch and was submitted to the Gnome project here: > > http://bugzilla.gnome.org/show_bug.cgi?id=3D135230 > > (that was Feb 2004, and still not accepted). > */ > > int > vte_terminal_set_pty(VteTerminal *terminal, int pty_master) > { > GtkWidget *widget; > > g_return_val_if_fail(VTE_IS_TERMINAL(terminal), -1); > widget =3D GTK_WIDGET(terminal); > > if (terminal->pvt->pty_master !=3D -1) { > _vte_pty_close(terminal->pvt->pty_master); > close(terminal->pvt->pty_master); > } > terminal->pvt->pty_master =3D pty_master; > > /* Open channels to listen for input on. */ > xvte_terminal_connect_pty_read(terminal); > > /* Open channels to write output to. */ > xvte_terminal_connect_pty_write(terminal); > } > > Now the main program. This is just a much-simplified version of an > example program that is distributed with readline. I installed the > modified vte library in /home/jwe/vte, so I compile the example > program with the following command > > gcc -I/home/jwe/vte/include -g $(pkg-config gtk+-2.0 --cflags) \ > example.c -o example \ > -L/home/jwe/vte/lib -lvte $(pkg-config gtk+-2.0 --libs) \ > -lreadline -lpthread -lutil > > and run it with > > LD_LIBRARY_PATH=3D/home/jwe/vte/lib ./example > > This example is extremely simple and does almost nothing. The point > is to demonstrate using the terminal widget to embed an interactive > application without using a coprocess. The next step would be to add > a button or two that modifies some global data and protect the > accesses to the global data as needed. Add in some signal handling > and I think that would demonstrate most of what Octave does. > > #include > #include > #include > #include > #include > #include > #include > > #include > #include > > void > execute_line (char *line) > { > static char *err_hilite_on =3D "\e[01;31m"; > static char *err_hilite_off =3D "\e[0m"; > > if (! strcmp (line, "ls")) > system ("ls -FClg"); > else if (! strcmp (line, "quit")) > exit (0); > else > fprintf (stderr, "%sunknown command `%s'%s\n", > err_hilite_on, line, err_hilite_off); > } > > void * > fileman_main (void *dummy) > { > rl_readline_name =3D "FileMan"; > > /* Loop reading and executing lines until the user quits. */ > while (1) > { > char *line =3D readline ("FileMan: "); > > if (! line) > break; > > if (*line) > { > add_history (line); > execute_line (line); > } > else > free (line); > } > > return 0; > } > > int > main (int argc, char **argv) > { > #ifndef FORK_COMMAND > pthread_t fileman_thread; > > int fdm; > int fds; > #endif > > /* GtkWidget is the storage type for widgets */ > GtkWidget *window; > GtkWidget *terminal; > > /* This is called in all GTK applications. Arguments are parsed > * from the command line and are returned to the application. */ > gtk_init (&argc, &argv); > > /* create a new window */ > window =3D gtk_window_new (GTK_WINDOW_TOPLEVEL); > > terminal =3D vte_terminal_new (); > > #ifdef FORK_COMMAND > vte_terminal_fork_command (VTE_TERMINAL (terminal), > "/usr/bin/octave", 0, 0, 0, 0, 0, 0); > #else > if (openpty (&fdm, &fds, 0, 0, 0) < 0) > fprintf (stderr, "oops!\n"); > > dup2 (fds, 0); > dup2 (fds, 1); > dup2 (fds, 2); > > pthread_create (&fileman_thread, 0, fileman_main, 0); > > vte_terminal_set_pty (VTE_TERMINAL (terminal), fdm); > #endif > > /* This packs the terminal into the window (a gtk container). */ > gtk_container_add (GTK_CONTAINER (window), terminal); > > vte_terminal_set_font_from_string (VTE_TERMINAL (terminal), "Fixed 11")= ; > > vte_terminal_set_size (VTE_TERMINAL (terminal), 80, 24); > > vte_terminal_set_scrollback_lines (VTE_TERMINAL (terminal), 1024); > > /* The final step is to display this newly created widget. */ > gtk_widget_show (terminal); > > /* and the window */ > gtk_widget_show (window); > > /* All GTK applications must have a gtk_main(). Control ends here > * and waits for an event to occur (like a key press or > * mouse event). */ > gtk_main (); > > return 0; > } > > ------=_Part_10571_15064187.1142100513834 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline John,

I just returned from Italy and I've started digging around for= your method of communicating with Octave. It looks as if openpty does not = exist in MinGW, so I won't be able to use it. I don't know if there's going= to be an issue with the DLL's fd 1 being different from the EXE's fd 1, I'= ll see when I try it. I think MinGW has pipes, I don't know what having a P= TY instead of a pipe for Octave will do, do you suppose that's ok?

From time to time, the GUI wants to go and pry in Octave's innards.= While ideally Octave would be thread-safe, for now it would be nice to be = backward compatible and not rely on this. One simple workaround is to only = pry into Octave's innards when it is safe to do so.

One way of doing this would be to have Octave call a function befor= e each prompt. From within this function, I could dump all the info from Oc= tave into my GUI. Another method would be to be to read a variable or call = a function which would tell me whether Octave is running code or waiting fo= r input. Or, if Octave has an idle() function (like GLUT) which it calls wh= enever it has some spare cycles, I could hook into that to update the GUI. = Are there backwards-compatible ways of doing this?

Cheers,

S=E9bastien Loisel

On 2/28/06, John W. Eaton <jwe@bevo dot che dot wisc dot edu> wrote:
I have some thoug= hts and questions about how we would like a GUI for
Octave to work. = ; Mostly, they are about how the GUI and Octave should
be linked together, who should be in charge of events, etc.

In S= ebastien's Octave Workshop, I think the GUI handles all events
(correct = me if I'm wrong).  Octave is embedded in the GUI and gets
inpu= t via Octave's eval_string function.  Output is captured by
grabbing std::cout in some way.  This sort of works, but miss= es a few
things like handling the diary, automatic function updating bas= ed on
the last prompt time, the actual readline library for command-line=
editing, etc.  If you want to stick with eval_string, then it= would
still be possible to handle the all these things, but you have to
du= plicate a lot of functionality that is already in Octave.  Theeval_string function was not really meant for embedding an interactive
= Octave session (for one thing, embedding Octave this way turns off the
internal flag that tells it that it is running in an interactive
ses= sion).  You could probably work around that too, but I think ther= e
must be a better way.

Also, if you are embedding Octave with ev= al_string, I think your GUI
will be unresponsive while Octave is performing computations. &nbs= p;Unless
you are running Octave in a separate thread, I think you have t= o wait
until eval_string returns to continue with the GUI operations.&nb= sp; If you
are running Octave in a separate thread and then having the GUI touch
Oc= tave internals while eval_string is running, you will likely have
unexpe= cted results since Octave is not currently thread safe (there
is no mech= anism to prevent global data from being modified at the
wrong time).

To avoid threads, you could start Octave as a copro= cess to your GUI,
but you will still need some modifications to Octave s= o that it can
process requests while it is off running user code. &= nbsp;You won't want to
use a pipe for the communication though, because that will not be
re= cognized as tty, so the interactive features won't work properly.
Openin= g a pty connection to a separate process is better (Octave will
see the = pty as a tty, so it will run in interactive mode).

There is a terminal widget for GTK (VTE) that would make it easy to=
start Octave as a separate process and communicate with it using a
p= ty.  It provides a terminal window that is (more than) sufficient= for
running Octave.  Running Octave this way, you get readline, Octav= e
thinks it is interactive, diary works, etc.  Perhaps there i= s
something similar for Qt?

The difficulty with this approach is = that if you want the GUI to do
more than you can get by running Octave in a terminal window, then the<= br>GUI and Octave have to pass some extra information around.  I'= m sure
that this can be done, but it will mean some extra effort. &= nbsp;Will you
parse text data or design a protocol for passing binary data?  Wi= ll
Octave or the GUI be in charge of that, and how much overhead will it=
add?  In terms of passing data back and forth, it would be mu= ch
simpler if the GUI and Octave were in the same process and the GUI
could just call internal Octave functions to do its thing.

At le= ast with GTK on Unixy systems, it should be possible to embed
Octave in = a terminal widget and make Octave think it is talking to a
tty without h= aving to run Octave as a separate process.  To do this,
you will need a thread-safe Octave (but you need that anyway with theeval_string approach) and a pty implementation.  The idea is th= at
instead of spawning a subprocess, you set up some ptys, dup some file=
descriptors, start Octave in a separate thread, then attach the master
pty to the terminal widget.

Unfortunately, the VTE widget will n= ot allow this mode of operation
out of the box.  Fixing it req= uires the addition of one small
function, but it must be added to the co= re library because it relies
on some private internal data to work.

So, some questions and th= ings to think about.

  * Should we work to make Octave thr= ead-safe?  I think I would prefer
    to w= ork on this instead of a protocol that could be used to
    communicate with a separate Octave process.
=
  * Is there a terminal widget for Qt that could do the same = thing as
    VTE?  I don't really want a G= UI for Octave to be useful with just
    one toolkit= .

  * Would any of this work in a Windows environment?
<= br>I'm appending a simple example program that illustrates this.  = ;It
depends on the modified VTE terminal widget and GTK.  Buil= ding it is a
bit tricky since you need to first patch the VTE sources an= d build the
modified VTE library.  Then you need to ensure that you use t= hat
library for building and running the example program.

I'm ass= uming that if we choose this sort of approach, we can get the
necessary = changes rolled into the default vte (or other) terminal
widget(s).  If not, then I suppose we could provide our own t= erminal
widget with Octave (though that would definitely not be my
pr= eference).

jwe


First, here is the function that must be a= dded to vte:

/* We need the following additional function for vte:

 =     int vte_terminal_set_pty(VteTerminal *terminal, int pty_= master);

   It must be added to vte because it calls stati= c private functions
   in the vte library.

   This function was written by Eric Smith <eric@brouhaha dot com>,
   archi= ved here:

     http://www.brouhaha.com/~eric/software/vte/vte-0.11.10-add-pty.patch
   as a patch and was submitted to the Gnome project here:
     http://bugzilla.gnome.org/show_bug.cgi?id=3D135230

   (that was Feb 2004, and still not accepted).
*= /

int
vte_terminal_set_pty(VteTerminal *terminal, int pty_master)=
{
        GtkWidget *widget;=

        g_return_val_if_fai= l(VTE_IS_TERMINAL(terminal), -1);
        widget =3D GTK_WIDGET(t= erminal);

        if (termin= al->pvt->pty_master !=3D -1) {
      = ;          _vte_pty_close= (terminal->pvt->pty_master);
      &= nbsp;         close(terminal-&= gt;pvt->pty_master);
        }
   =      terminal->pvt->pty_master =3D pty_maste= r;

        /* Open channels = to listen for input on. */
       &nb= sp;xvte_terminal_connect_pty_read(terminal);

    = ;    /* Open channels to write output to. */
        xvte_terminal_connect_p= ty_write(terminal);
}

Now the main program.  This is ju= st a much-simplified version of an
example program that is distributed w= ith readline.  I installed the
modified vte library in /home/j= we/vte, so I compile the example
program with the following command

  gcc -I/home/jwe/v= te/include -g $(pkg-config gtk+-2.0 --cflags) \
    = example.c -o example \
    -L/home/jwe/vte/lib -lvte= $(pkg-config gtk+-2.0 --libs) \
    -lreadline -lpt= hread -lutil

and run it with

  LD_LIBRARY_PATH=3D/home/jwe/vte/= lib ./example

This example is extremely simple and does almost nothi= ng.  The point
is to demonstrate using the terminal widget to = embed an interactive
application without using a coprocess.  The next step would b= e to add
a button or two that modifies some global data and protect the<= br>accesses to the global data as needed.  Add in some signal han= dling
and I think that would demonstrate most of what Octave does.

#include <stdio.h>
#include <sys/types.h>
#includ= e <unistd.h>
#include <gtk/gtk.h>
#include <vte/vte.h&= gt;
#include <pthread.h>
#include <pty.h>

#include= <readline/readline.h>
#include <readline/history.h>

void
execute_line (char *= line)
{
  static char *err_hilite_on =3D "\e[01;31m&qu= ot;;
  static char *err_hilite_off =3D "\e[0m";
<= br>  if (! strcmp (line, "ls"))
    system ("ls -FClg");
  e= lse if (! strcmp (line, "quit"))
    exit = (0);
  else
    fprintf (stderr, "= %sunknown command `%s'%s\n",
      &n= bsp;      err_hilite_on, line, err_hilite_off);
}

void *
fileman_main (void *dummy)
{
  rl_re= adline_name =3D "FileMan";

  /* Loop reading and= executing lines until the user quits. */
  while (1)
 = ;   {
      char *line =3D = readline ("FileMan: ");

      if (! line)
  &nbs= p;     break;

     = ; if (*line)
        {
&= nbsp;         add_history (lin= e);
          execute_= line (line);
        }
 =      else
      &= nbsp; free (line);
    }

  ret= urn 0;
}

int
main (int argc, char **argv)
{
#ifndef FORK_COMMA= ND
  pthread_t fileman_thread;

  int fdm;
=   int fds;
#endif

  /* GtkWidget is the stora= ge type for widgets */
  GtkWidget *window;
  GtkWidget *terminal;

  /* This is called i= n all GTK applications. Arguments are parsed
   * from the com= mand line and are returned to the application. */
  gtk_init (= &argc, &argv);

  /* create a new window */
  window =3D gtk_window_new (GTK_WINDOW_TOPLEVEL);

&nb= sp; terminal =3D vte_terminal_new ();

#ifdef FORK_COMMAND
&n= bsp; vte_terminal_fork_command (VTE_TERMINAL (terminal),
 &nbs= p;            &= nbsp;           &nbs= p;  "/usr/bin/octave", 0, 0, 0, 0, 0, 0);
#else
  if (openpty (&fdm, &fds, 0, 0, 0) < 0)<= br>    fprintf (stderr, "oops!\n");

&n= bsp; dup2 (fds, 0);
  dup2 (fds, 1);
  dup2 = (fds, 2);

  pthread_create (&fileman_thread, 0, filema= n_main, 0);

  vte_terminal_set_pty (VTE_TERMINAL (terminal), fdm);#endif

  /* This packs the terminal into the window (a gt= k container). */
  gtk_container_add (GTK_CONTAINER (window), = terminal);

  vte_terminal_set_font_from_string (VTE_TERMIN= AL (terminal), "Fixed 11");

  vte_terminal_set_size (VTE_TERMINAL (terminal), 80, 24)= ;

  vte_terminal_set_scrollback_lines (VTE_TERMINAL (termi= nal), 1024);

  /* The final step is to display this newly = created widget. */
  gtk_widget_show (terminal);

  /* and the window */
  gtk_widget_show (wi= ndow);

  /* All GTK applications must have a gtk_main(). C= ontrol ends here
   * and waits for an event to occur (like a = key press or
   * mouse event). */
  gtk_main ();

  return 0;
}


------=_Part_10571_15064187.1142100513834--