Abstract. Fruit is a new user interface toolkit based on a formal model of user interfaces. This formal basis enables us to write simple, concise executable specifications of interactive applications. This paper presents the Fruit model and several example specifications. We consider one example (a basic media controller) in detail, and contrast the executable specification style of Fruit with a more traditional “rapid prototype” implementation using an imperative, object-oriented toolkit (Java/Swing) to show the benefits of our approach.

[Haskell-cafe] Re: Why binding to existing widget toolkits doesn’t make any sense

Hi Conal (and Paul and Henrik and everyone else!),

Apologies for the delay in replying.  Paul is quite right about my day
job keeping me quite busy, although I've been enjoying watching the
discussion.

I think there are several reasons that Fruit never really caught on.

First and foremost, Fruit never provided anything close to the full
suite of GUI components found in modern UI toolkits (buttons, sliders,
scrollbars, check-boxes, text fields, combo boxes, split panes, tabs,
menus, toolbars, etc. etc.).  I never got much further than
implementing labels, buttons, text fields and maybe one or two other
components.   This wasn't quite as limiting as one might think as
Fruit subsumed everything that was possible to do in Fran and had a
very powerful vector graphics library underneath (Haven) so one could
use Fruit to implement quite sophisticated interactive animations and
combine them with the library of other GUI components.  But one
couldn't realistically expect to download and install Fruit and just
start using it to write the next great full-fledged GUI app. in
Haskell.

This is related to the second reason Fruit didn't catch on, which is
that Fruit was built directly on an interactive graphics library, not
a standard GUI toolkit.  I think that this was the right first step
for our research goals of giving a rigorous answer to the question of
"What is a formal model of a GUI?".  But from both a pragmatic and
research point of view, the next interesting research problem to solve
would be:  "How can we present the Fruit programming interface (which
offers a clean, simple, pure semantic model of a GUI), but implement
the individual GUI components using a standard GUI toolkit?".  This is
actually an interesting implementation challenge since Yampa's
switching and dynamics collections mean that absolutely anything can
happen to the widget hierarchy from one time step to the next.  You
really, really don't want to destroy and rebuild the entire widget
hierarchy of the application at every time step, so figuring out how
to do some kind of sample-to-sample differencing of the widget
hierarchy would be essential.  It's not obvious how to do this, which
is probably why FranTk and wxFruit took a bit more of a pragmatic
approach and exposed bits of the imperative programming model of the
underlying toolkit for widget creation and destruction.

Another important point is that even the bits of fruit that were
implemented involved a number of different dependent parts, so it was
always a bit more work to download and install than I would have
liked.  I was hell-bent on using a vector graphics library as the
foundation. Unfortunately there was no widely-available cross-platform
vector graphics library implemented in C.  So I chose to use Java2D,
which meant that I had to build my own system for Haskell/Java
interoperability (GCJNI, later hsjni) and a Haskell vector graphics
library (Haven) on top of that.  So if one wanted to play with fruit,
it was necessary to download and install Java, gcjni, haven, yampa and
fruit.  And in the early days one also had to grab greencard and the
arrows preprocessor!  I think we did a reasonable job of packaging
these given the severe limits of Haskell packaging and build tools at
the time, but it was a ton of work and still never made it easy enough
for others to experiment casually.

Apart from the practicalities of the implementation, I also think
there are legitimate pragmatic concerns about using the FRP model to
write a full-fledged application.  The benefit of using FRP is that it
forces the programmer to be explicit about input and output
connections, which enables precise reasoning about exactly what a
particular signal function may or may not do.  But there is an
inevitable tension between reasoning power and easy extensibility.
If, for example, we wanted to add a "File...Open" menu item to a Fruit
application that didn't have this originally, we would have to very
explicitly thread signals that provides access to the filesystem
through the entire application to make it available in the menu.  This
is always going to be more work than in a conventional imperative
toolkit in which every component always always has implicit access to
the World.  A related point here is that if you look closely at the
dynamic collections work (pSwitch and friends), you will see that they
operate on collections of signal functions with homogenous types of
input and output signals.  This means that a collection of signal
functions that have heterogenous input and output signals will all
have to be "widened" to some input and output type that is the union
of all the individual signal types in the collection, and then the
individual members of the collection will just ignore the signals they
don't need.  Taken to it's logical extreme we end up with a model
where we have a time-varying collection of SF World World, which seems
disturbingly close to the overly-general semantic model we were trying
to escape from in the original Fruit paper.

But I agree that Fruit does offer some nice practical benefits as a
programming model, and I think it would be great to focus on fruit's
ability to provide interesting and novel animated vector graphics UIs
in a modular way.  Have you played with an iPhone?  It is quite simply
one of the best UIs I have ever used, precisely because of the
pervasive use of vector graphics (spatially continuous zooming) and
pervasive, uniform and subtle animation effects.  See:
      http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=00036T=1
for a video of the iPhone interface and the veritable Ed Tufte
offering some critique and design suggestions.

I think it would be quite interesting and relatively straightforward
to express a number of the standardized UI effects from the iPhone as
a library of GUI transformers in Fruit.  I also suggest checking out
Piccolo, a 2-D structured graphics framework for .Net and Java:
    http://www.cs.umd.edu/hcil/jazz/
Piccolo is based on the idea of using a scenegraph for 2-D graphics,
and is able to leverage that to provides a nice standard library of
animations and effects.  Again, I think many of the ideas would fit
very naturally into a Fruit-like programming model.

One issue I think you may encounter, though, is that last time I
looked, there was still no high-quality, widely available
cross-platform 2-D vector graphics library in C or C++!  So that means
either building on something like Java2D like I did and ending up with
lots of moving parts and a fat runtime (an entire JVM), giving up on
cross-platform compatibility (and losing important Linux-only folks
like Henrik!), or rolling up your sleeves and writing your own vector
graphics library in C before you even start (a lot of work!).  But
maybe something has changed on this front since I was actively working
in the area.

Hope that helps.  Good luck with making another foray into purely
functional GUIs.  Please keep me and the rest of the list informed of
your progress!

Kind Regards,

       -Antony