linux: shared memory interface - link with librt

[supercollider.git] / HelpSource / Guides / GUI-Introduction.schelp

title:: Introduction to GUI
summary:: An introduction to writing graphical user interface code
categories:: GUI



SECTION:: Platform indepedent GUI code

SuperCollider provides for writing graphical user interface code that may be executed by different GUI kits implementing the same functionality. A user may not worry about the different implementations and under most circumstances generic GUI code will have the same effect in any of them. Nontheless, there may appear slight differences between them, therefore it's worth mentioning them here.

subsection:: Different GUI kits

Various GUI kits are managed by the link::Classes/GUI:: class. See it's documentation for details of how to switch the active GUI kit.

At the moment of this writing, there exist three GUI kits:

table::

## strong::Name:: || strong::Implementation:: || strong::Availability::
## strong::Cocoa:: || Mac OS Cocoa toolkit || Only on Mac OS, within the SuperCollider application
## strong::SwingOSC:: || Java || Cross-platform, running as a separate program and commuincating with SuperCollider via OSC.
## strong::Qt:: || Qt framework || Cross-platform.

::

subsection:: Generic GUI code

There is a set of classes available for writing generic GUI code. When they are instantiated an instance of the kit-specific equivalent class is returned instead. The equivalent classes of all the GUI kits implement a large set of common methods which can therefore be used in generic code, and are documented as belonging to the generic class.

The kit-specific classes typically have the same name as their generic equivalents, but prefixed with an identifier of the implementation: "SC" for Cocoa, "JSC" for SwingOSC and "Q" for Qt.

For a list of all the generic GUI classes and their kit-specific equivalents see link::Overviews/GUI-Classes::.

In the rest of this document we will refer to GUI classes by their generic name, adding notes where important differences between GUI kits are of concern.





SECTION:: Basic elements: Windows, views and containers

The most fundamental element of the GUI is the strong::Window::. It occupies a rectangular space on screen within which other GUI elements are displayed. It usually has a bar that displays the window's title and allows for moving it, resizing it and closing it with the controls it displays or through mouse and keyboard interaction. Some of these aspects may be controlled within SuperCollider GUI code, though it is largely platform-dependent how precisely interaction with a window happens and is visually indicated.

The GUI elements contained within a Window are called strong::views::. They all inherit from the basic View class. The view occupies a rectangular space of the window within which it draws itself to display some data or to indicate a mode of interaction between the user and the program. Views receive keyboard and mouse events generated by the user and respond to them by controlling the behavior of the program. They also display information about the state of the program and the data on which it operates.

There are also special types of views that can contain other views and are thus called strong::containers::, for example the CompositeView. They allow for structuring GUI in a hierarchical way. A container view is called a strong::parent:: of the views it contains, and they are called its strong::children::. Hierarchical organization allows to easily change aspects of all the views within a container: if the parent view is hidden, so are all the children; if the parent view is moved, so are they. Children are positioned with coordinates relative to their parent.

note::
In many aspects, a Window is also considered to be a parent of the views it contains, and can functionally take the same place in code as container views, although that is not true in all cases. When a Window is created it implicitely creates a container view occupying its entire space. When a view is created with a Window as its parent it will actually become a child of that container. See Window's link::Classes/Window#-view#view:: method and View's link::Classes/View#*new#constructor:: for details.
::

note::
In strong::Qt GUI:: there is no distinction between windows, views, and containers. An instance of the View class itself can be displayed directly on screen, and can contain other views, so the same applies to all its subclasses. Most of the methods that are specific to Window and containers in other GUI kits are shared by all views in Qt.
::

The following example shows a window containing a Button, a Slider and a group of StaticText views contained in a CompositeView. When the button is clicked the visibility of the CompositeView is toggled, while interacting with the Slider will move the CompositeView (and consequently all its contents) in horizontal direction.

code::
w = Window.new("GUI Introduction", Rect(200,200,255,100));
b = Button.new(w,Rect(10,0,80,30)).states_([["Hide"],["Show"]]);
s = Slider.new(w,Rect(95,0,150,30));
c = CompositeView.new(w,Rect(20,35,100,60));
StaticText.new(c,Rect(0,0,80,30)).string_("Hello");
StaticText.new(c,Rect(20,30,80,30)).string_("World!");
b.action = { c.visible = b.value.asBoolean.not };
s.action = { c.bounds = Rect( s.value * 150 + 20, 35, 100, 100 ) };
w.front;
::



SECTION:: Automatic positioning and resizing of views

As a handy alternative to specifying all the dimensions and positions of views explicitely in code, SuperCollider allows for automatic positioning and resizing of views in relation to each other and in relation to window size - at the view creation and dynamically, when window is resized. There is several mechanisms for this purpose.

subsection:: View's resize options

Views can automatically resize or move when their parent is resized, in one of the nine different ways that define how each of the view's edges will move along with the parent's edges. For documentation see the view's link::Classes/View#-resize#resize:: method and link::Reference/Resize:: document.

code::
w = Window.new("GUI Introduction", Rect(200,200,200,200));
TextField.new(w,Rect(0,0,200,30)).resize_(2);
Slider.new(w,Rect(0,30,30,170)).resize_(4);
TextView.new(w,Rect(30,30,170,170)).resize_(5);
w.front;
::

subsection:: Decorators

Decorators are objects that can be assigned to container views to carry the task of positioning the container's child views (currently there exists only one: FlowLayout). After a decorator is assigned to a container, the views created as its children will automatically be positioned in a specific pattern. See documentation of link::Classes/FlowLayout:: for details.

code::
w = Window.new("GUI Introduction", Rect(200,200,320,320)).front;
// notice that FlowLayout refers to w.view, which is the container view
// automatically created with the window and occupying its entire space
w.view.decorator = FlowLayout(w.view.bounds);
14.do{ Slider(w, 150@20) };
::

subsection:: Layouts

Layout classes make part of a complex system to manage both position and size of views. Using layouts, only relations of views within a pattern of organization need to be specified and their exact positions as well as sizes will automatically be deduced based on their type (the content they display and the type of interaction they offer) and in accord with principles of good GUI usability. Layouts also position and resize views dynamically, whenever their parent is resized or their contents change.

See the link::Guides/GUI-Layout-Management:: guide for detailed explanation.

note::
Layouts are currently implemented strong::only in Qt GUI::. The following example will not work in other GUI kits.
code::
w = Window.new("GUI Introduction").layout_(
        QVLayout(
                QHLayout( Button(), TextField(), Button() ),
                TextView()
        )
).front;
::
::

note::
Layouts are not compatible with decorators and will ignore view resize options. The effect of combining layouts and decorators is undefined.
::

SECTION:: Customizing appearance

Views offer various ways to customize their appearance. This ranges from decorative changing of colors they use to draw themselves to controlling how they display various kinds of data.

subsection:: Colors

Colors are represented in GUI code by the link::Classes/Color:: class.

A typical color that can be customized is background color - a color of choice can be applied to whatever is considered to be the background of a particular view. Views that display some text will typically also allow customizing its color as well.

Custom colors may be associated with different changing states of views or data they display, for example: Button allows to associate background and text colors with each one of its states, and will thus switch colors together with state when clicked; ListView allows to set a different background color for each of its items, as well as special background and text colors applied only to the item currently selected.

Whenever you execute the following example, a random palette of colors will be applied to different aspects of the Window, the Button and the ListView:

code::
w = Window.new("GUI Introduction").background_(Color.rand).front;
b = Button.new(w,100@30).states_([
        ["One",Color.rand,Color.rand],
        ["Two",Color.rand,Color.rand],
        ["Three",Color.rand,Color.rand]
]);
l = ListView.new(w,Rect(0,40,200,100))
        .items_(["One","Two","Three"])
        .colors_([Color.rand,Color.rand,Color.rand])
        .hiliteColor_(Color.blue)
        .selectedStringColor_(Color.white);
::

subsection:: Fonts

Views that display some text will typically allow you to specify a custom font for it. Fonts are represented by the link::Classes/Font:: class, which can also be queried for the default font used in general, as well as the default font specifically for the "serif", "sans-serif" and "monospace" font types. It can also be queried for all available fonts on the system.

code::
(
w = Window.new("GUI Introduction",Rect(200,200,200,70)).front;
a = [Font.defaultMonoFace, Font.defaultSansFace, Font.defaultSerifFace];
b = Button.new(w,Rect(10,10,180,50))
        .states_([["Monospace"],["Sans serif"],["Serif"]])
        .font_(a[0])
        .action_({|b| b.font = a[b.value]});
)
::

subsection:: Other visual properties

Complex views may have many other ways to customize how they display the same data. link::Classes/MultiSliderView:: and link::Classes/EnvelopeView:: are good examples.






SECTION:: Actions and hooks: Make that button do something!

Views and windows can be assigned strong::actions:: that they will perform whenever a specific event occurs as a result of user's interaction. Technically, an action can be any Object, and when the relevant event occurs, it's link::Classes/Object#-value#value#:: method will be called. For example, it is useful to assign a Function as an action, which allows one to define an arbitrary chunk of code to be performed in response to a GUI event.

Objects can also be given to views and windows to evalute on events that are not a direct result of user's interaction, but convey useful information about the view's operation and the state it moved in. In this case they are often differentiated from actions and called strong::hooks::.

Here, we will give an overview of different kinds of actions and hooks. See link::Classes/View#actions_in_general:: and following sections for precise explanation of how to assign and make use of them.


subsection:: Default actions

Views can typically be assigned a default action with their link::Classes/View#-action#action:: setter method, which will be performed when the view's primary mode of interaction is invoked. The default action for a Button for example occurs when it is clicked, for a Slider when its handle is moved.

In the following example, pressing the button will open an exact same window but at different position.

code::
~makeWindow = { var w;
        w = Window.new("Evader",Rect(500.rand + 100, 500.rand + 100, 200,50)).front;
        Button.new(w,Rect(10,10,180,30)).states_([["Evade"]]).action_(~makeWindow);
};
~makeWindow.value;
::

subsection:: Keyboard and mouse actions

All the views can be assigned actions to specific mouse and keyboard events, no matter what other effects those events might have on the view or what other specialized actions or hooks the view might trigger on these events.

You can assign actions to mouse events generated when the mouse pointer enters the space of a view, when it moves over them, and when a mouse button is pressed or released.

See link::Classes/View#mouse_actions:: for details.

In the following example the StaticText will report whether the Button is pressed or released.

code::
w = Window.new(bounds:Rect(200,200,200,50)).front;
b = Button.new(w,Rect(10,10,80,30)).states_([["Off"],["On"]]);
t = StaticText(w,Rect(100,10,90,30)).string_("Button released");
b.mouseDownAction = { t.string = "Button pressed" };
b.mouseUpAction = { t.string = "Button released" };
::

You can assign actions to keyboard events generated whenever a key is pressed or released while the view has keyboard focus. Keyboard focus is a state of a view in which it has exclusive priority to respond to keyboard events. A view that has keyboard focus typically in a way visually indicates so. On most platforms, pressing the Tab key will switch the keyboard focus between views in the active window and clicking on a view will give it focus.

See link::Classes/View#key_actions:: for details.

Typing text into any of the TextFields in the following example will change the color of the rectangle bellow, for each TextField a different color.

code::
w = Window.new(bounds:Rect(200,200,200,100)).front;
x = TextField(w,Rect(10,10,80,30));
y = TextField(w,Rect(110,10,80,30));
t = StaticText(w,Rect(10,40,180,50));
~reset = {t.background = Color.red};
x.keyDownAction = {t.background = Color.green};
x.keyUpAction = ~reset;
y.keyDownAction = {t.background = Color.blue};
y.keyUpAction = ~reset;
~reset.value;
::

subsection:: Drag and drop actions

When a mouse button is pressed on a view together with Cmd(Mac OS) or Ctrl(Other OS) key and the mouse pointer is moved while holding the button, a strong::drag-and-drop:: operation is initiated - in case the view supports it. Most views have a default object that they export when a drag is attempted. For a Slider it is its value, for a List it is the numeric index of the currently selected item, etc. It is said that the exported object is being strong::dragged::. When the dragging gesture ends on another view by releasing the mouse button on top of it, it is said that the dragged object was strong::dropped:: on another view. A view may respond to various objects dropped on it in different ways.

It is possible to customize what object a view exports when dragged from and how a view reacts to objects dropped by assigning custom drag and drop actions.

See link::Classes/View#drag_and_drop:: for details.

code::
(
        w = Window.new.front;
        a = Button(w, Rect(10, 10, 200, 20)).states_([["Hi There!"]]);
        a.beginDragAction = { a.dragLabel ="I'm dragging: \""++ a.states[0][0]++"\""; a.states[0][0] };
        DragSink(w,Rect(10,40,200,20)).align_(\center).string="Cmd-drag from Button to here";
)
::

subsection:: Other specialized actions

Some views can be assigned actions on other events specific to their mode of interaction with the user which you are invited to discover by consulting their documentation.

subsection:: Hooks

Hooks are various events that signify important changes of state of GUI elements. Technically they are used the same way as actions, but are distinguished from them to denote events that are not a direct result of the user's interaction. Methods of GUI classes used to assign hooks are usually prefixed with "on". (You will also find this naming pattern in methods of other SuperCollider classes, that have hooks in the same sense).

For example, one hook that every view as well as Window has is onClose, which is triggered when the window is closed or the view is removed. Other hooks for example exist for the case when a Window becomes or ceases to be the active one.




SECTION:: Custom views

The UserView is a view that displays and does nothing on itself, but allows emphasis::you:: to define how it will be drawn, and for which you can  define the entire behavior using mouse, key, and drag and drop actions. For documentation on all of these aspects, see link::Classes/UserView::, link::Classes/View::, and link::Classes/Pen::. The explanation below, however, will demonstrate the basic techniques for designing a custom view.

There is two ways of using the UserView class: either creating it and using its methods to define its behavior, or subclassing it, which gives you more freedom but is more complex. Either way, you will be using the link::Classes/Pen:: class to draw the view. Pen is a powerful class that allows you to algorithmically draw using simple visual primitives like lines, arcs, curves, rectangles, ellipses, etc. and fill the shapes with colors and gradients.

subsection:: The simple way: using the UserView class

The simple way comprises the following steps:

numberedList::
## create a User View
## define a draw function
## define the default action
## define mouse actions
## define key actions
## define drag and drop actions
::

You can omit steps which you don't need.

code::
(
var value = 0.5;
w = Window.new.front;

// (1) create a UserView
v = UserView(w,Rect(50,50,200,20));

// (2) define a drawing function using Pen
v.drawFunc = {
        // Draw the fill
        Pen.fillColor = Color.grey;
        Pen.addRect(Rect(0,0, v.bounds.width*value,v.bounds.height));
        Pen.fill;
        // Draw the triangle
        Pen.fillColor = Color.red;
        Pen.moveTo(((v.bounds.width*value)-5) @ v.bounds.height);
        Pen.lineTo(((v.bounds.width*value)+5) @ v.bounds.height);
        Pen.lineTo(((v.bounds.width*value)) @ (v.bounds.height/2));
        Pen.lineTo(((v.bounds.width*value)-5) @ v.bounds.height);
        Pen.fill;
        // Draw the frame
        Pen.strokeColor = Color.black;
        Pen.addRect(Rect(0,0, v.bounds.width,v.bounds.height));
        Pen.stroke;
};

// (3) set the default action
v.action = {value.postln; v.refresh};

// (4) define mouse actions
v.mouseDownAction = { arg view, x = 0.5,y, m;
        //m.postln;
        ([256, 0].includes(m)).if{ // restrict to no modifier
        value = (x).linlin(0,v.bounds.width,0,1); v.doAction};
};

v.mouseMoveAction = v.mouseDownAction;

// (5) (optional) define key actions
v.keyDownAction = { arg view, char, modifiers, unicode,keycode;
        if (unicode == 16rF700, { value = (value+0.1).clip(0,1) });
        if (unicode == 16rF703, { value = (value+0.1).clip(0,1) });
        if (unicode == 16rF701, { value = (value-0.1).clip(0,1) });
        if (unicode == 16rF702, { value = (value-0.1).clip(0,1) });
        v.doAction;
};

// (6) (optional) define drag and drop behavior
v.beginDragAction = {value}; // what to drag
v.canReceiveDragHandler = {View.currentDrag.isNumber}; // what to receive
v.receiveDragHandler = {value = View.currentDrag; v.doAction }; // what to do on receiving


// just for testing drag and drop
Slider(w,Rect(50,100,200,20));

StaticText(w,Rect(50,150,350,50)).string_("To Test Drag and Drop,\nHold down Cmd (Ctl) Key");

)
::

subsection:: The advanced way: subclassing the UserView

Subclassing the UserView differs in many aspects from the example above. For a subclassing template and a quick tutorial on how to write a custom widget as a UserView subclass, see the link::Guides/UserView-Subclassing:: guide.






SECTION:: Caution: GUI and timing

warning::
Executing code that uses the GUI system is restricted to main application context. There are many ways in SuperCollider for code to be executed in other contexts that run in parallel with the main one, and interacting with GUI objects is not allowed there. This includes:

list::
## Code scheduled on the SystemClock and the TempoClock
## Code executed in response to OSC messages
::

::

If you attempt to interact with a GUI object in the contexts listed above, an error will be thrown.

Therefore, if you want to use Functions, Routines, Tasks and other similar objects to schedule code that interacts with GUI elements, you must do so using the AppClock, since code scheduled on the AppClock is performed in the main application context. You can of course also reschedule GUI code to the AppClock from within code performed in other contexts, and the link::Classes/Function#-defer#'defer':: mechanism is a convenient shorthand for this.

An example of scheduling GUI code on the AppClock:

code::
w=Window.new.front;
Routine{
        20.do{
        w.bounds=Rect(200.rand, 200+200.rand, 300,300);
        0.1.wait;
        };
        w.close;
}.play(AppClock)
::

The same thing using the SystemClock in combination with the defer mechanism:

code::
w=Window.new.front;
Routine{
        20.do{
        {w.bounds=Rect(200.rand, 200+200.rand, 300,300) }.defer; // you must defer this
        0.1.wait;
        };
        {w.close}.defer; // you must defer this
}.play(SystemClock)
::

As mentioned above, using the GUI system is also not allowed in code performed directly in response to OSC messages (this includes functions given to all kinds of OSC reponder classes). The same solutions as above apply: