Smart Hand Computer

TECHNOLOGY

Concept : algorithmic gestural music application for smartphone

The smartphone contains a program that generates sounds from the gesture of the interpreter. This program consists of different sound states, interconnected to form a global algorithm.

The interpreter can change state from a predefined particular gesture.

Algorithmic structures and sound content are infinitely available. The smartphone must be able to withstand the computing power required for one state at a time. It must also integrate 9 sensor axes: 3 accelerometers, 3 gyroscopes, 3 magnetometers.

The amplification of the smartphone in concert must be done via a standard or wired UHF system, in order to transmit the integrity of the audio bandwidth to the broadcast system, without any discernible latency.

Hybrid interface: combination of a tactile and gestural interface.

-Tactile -> the smartphone screen is made up of 2 zones:

  • The first allows you to modulate finger position settings on the screen in x and y.
  • the second allows you to edit state volume settings to adapt to new broadcast contexts. This access must be hidden during the game.

An SHC shell is recommended to limit access to the interface during the game and secure the connection of the mini-jack. A 3D printing model is available below in the “Download” section. It was designed for iPhone type 5s or SE with a 4-point mini-jack connector from HICON.

-Gestural -> the 9 axes of the sensors built into the smartphone allow to discriminate the following gestures:

  • shock: rapid acceleration on a particular axis of the smartphone
  • gyrospeed: smartphone rotation speed
  • XY projection: two-dimensional projection of the smartphone’s orientation in relation to its 3D orientation
Sound Broadcast: quadraphonic
  • The smartphone is connected via its stereo output to 2 UHF transmitters. This allows the sound to be broadcast without latency to any type of sound broadcast device. Wireless systems can be replaced with a wired connection.
  • It is necessary to make a specific cable to connect to THE UHF transmitters according to the brand of the systems used.
  • It is advisable to make a shell for the smartphone in order to secure the mini-jack connection during the game and reduce the access of the touch screen to the bare minimum.
  • In rehearsal just plug in a headset. If using 2 Smartphones simultaneously the use of a small mixer will be necessary (DJ type for smartphone)

Software: iOS compatible application
  • An SHC app once downloaded and installed on the smartphone does not require any wifi connection to work. it is completely autonomous.
  • For some achievements, an OSC (Open Sound Control) message transmission can be used to control video or any other device.
Sound status:
  • a sound state or situation is a program made from the open source FAUST (GRAME) language. This one contains sounds and descriptionof interactions. Sounds can be synthesis or audio files on which real-time processing can be applied.
  • Changes in settings can be either related to the gesture interface and/or pre-programmed.
  • A change of sound state, goes through a very short fade-out of the current state, its mute, then the activation of the next state and its fade-in. This helps to limit the CPU overload between two states.
Creating a Smart Hand Computer Application :
  • Contact LisiLoG for a study of your project.
  • Depending on the complexity of the project one or more residences may be required.
Download :
iPhone SE or 5-plus case gcode with 4 pts HICON mini-jack: 3D printer iPhone SE SHC VR04

Cable schematics iPhone to UHF: Cable iPhone to UHF

ARTISTIC

On stage, it’s about living a moment and not playing it, that’s the difference between a performer, even a virtuoso, and a musician. It is the difference between thinking of the stage as a space of representation, or the stage as a space for encounters. Meetings between musicians or artists of all disciplines, but also encounters with the (the) public.

As an interpreter it is always possible to “hide” behind our know-how, our virtuosity… Here, whether with the LWS or the use of SHC, know-how and virtuosity are transformed into listening, presence, conviction, incarnation, precisely because there is no technique, tradition, school, or method. These are to be invented, to be re-invented constantly, as every time we play or replay a work. Rethinking our relationship to the body, to movement, to carry and to be carried by the sound we produce, to surprise ourselves with this renewed listening.

September October 2019

Transfigured Energy 3 / LiSiLoG Lebreton-Geoffroy / Taipei-Research-Lab

Transfigured Energy – performance around 6 interactive and performative installation projects coordinated by Jean Geoffroy.

Facilities created by the TRL and bringing together more than 50 students from different university dpt:

New Media dpt – Design dpt – Dance dpt – Music and Composition dpt…

In the end 3 days of performances with performances / improvisations / dance games with the audience …

May 2019

Transfigured Energy #2 / LiSiLoG Lebreton-Geoffroy / Taipei-Research-Lab

Residence and concert with the Taipei-Research-Lab collective

The TRL is a variable geometry ensemble of more than 50 composers, dancers, performers designers, designers etc. Following the February residency with Christophe Lebreton around the writing for SHC and the Faust programming language, the first production of interactive installations with the collaboration of Shih-San Wu (percussion teacher) Mei- Fang Lin (professor of Mr. Wang (director of the new media dpt).

First restitution of the 6 projects carried out by the students for the interactive exhibition in September 2019

April 2019

Taipei Opening Concert of the exhibition at Taipei Fine Arts Museum /

LiSiLoG Lebreton-Geoffroy / “Musica Mobile”

Concert at the opening of the exhibition Musica Mobile, an exhibition created and designed by Christophe and James Giroudon.

On the program: Collective improvisation with the public around an application SHC, Virtual Rizhomes and Touch by Vincent Carinola

with Christophe Lebreton – Vincent Carinola and Jean Geoffroy

April 2019

Festival Why Note Dijon / LiSiLoG Lebreton-Geoffroy / Dijon Higher School of Music

New technologies and interpretation

Started in December 2018 with a first day of meetings, supervised by Jean Geoffroy, the students of the ESM in Dijon offer pieces with the LWS and the SHC (Embedded studies by Vincent Carinola)

Concert at the Why Note festival in Dijon at the Atheneum

On the program:

Six student creations (4 around the LWS and 2 with the new SHC applications) Squash by Bertrand Dubedout, Virtual Rizhomes by Vincent Carinola, Toucher by Vincent Carinola performed by Jean Geoffroy

March 2019

Concert and study day at the Hear / LiSiLoG Lebreton-Geoffroy / HEAR Strasbourg

Concert at the HEAR of Strasbourg  Table music by Thierry de Mey / Percussions de Strasbourg / Atau 1 Le Loup by and with Atau Tanaka / Tippeke by Thierry de Mey with David Poro (cello) / Atau 2 Lifting by and with Atau Tanaka / Silence must be by Thierry de Mey / Perc Strasbourg ussions / Atau 3 Myogram by and with Atau Tanaka / Virtual Rizhomes by Vicent Raphaël Carinola with Jean Geoffroy  for two SHC

This residency ended with a day of study around the new approaches to the stage, the place of the performer in the face of the new opportunities offered by the new violins.

With the participation of Atau Tanaka – Georges Gagneré (director and researcher virtual scenes and motion capture) – Marie Lancelin (performer) – Jean Geoffroy – Christophe Lebreton – Thierry de Mey

“Virtual Rhizome”

Virtual Rhizome is akin to an open work, i.e. a work whose score would be closer to a navigation map than to the fixed time unfolding specific to the traditional musical scope.

The interpreter goes through all situations and explores the possibilities of combinations offered by the use of the two smartphones. While the choice of sound material, types of processing and their succession respond to a clear concern for writing, an important part of the result depends on the acuity of the performer and his ability to reveal a world that he must explore.

The graphic score describes a driving pattern, among other possible ones, going through the 23 situations, indicating their succession for each smartphone. It comes from the interpretation of Jean GEOFFROY, dedicatee and creator of the work. These are suggestions for situational arrangements, from which the interpreter can depart depending on the choices of expression derived from the personal exploration of the architecture of the work.

Several other readings remain open to exploration and it is these readings that constitute an essential part of Virtual Rhizome‘s interpretation. It is thus quite possible to play several cycles (a cycle is a course of 1 to 23) by favouring, in each cycle, the exploration of certain situations (such as a round form). Or, to go through the situations start with the 23 and finish at the 1. Or combine a reading from 1 to 23 with one hand and a 23-1 reading with the other, exploring situations far apart, in a path that produces a symmetrical shape (like a “crayfish” cannon).

Vincent Raphaël CARINOLA

First achievement based on the “Smart Hand Computer” concept. (State commission made to Vincent Raphael CARINOLA, creation of the piece at the Biennale Musique en scène 2018 in Lyon with the GRAME, performed by Jean GEOFFROY). Development: Christophe LEBRETON

Virtual Rhizome is for a performer holding in each hand an SHC containing the same “Virtual Rhizome” application. (available on Apple Store – iOS version 10 or more compatible with iPhone 5s or newer models)

The algorithm consists of 23 states (situations) connected to each other to form a circle (see diagram below)

The content of each state is described in the spreadsheet provided with the partition.

As a general rule, most of the time:

          • a fixed loop sound without interaction chosen at hazard in a predefined list
          • a brief sound when changing state chosen at hazard in a predefined list
          • a controllable sound:
                • volume via the Y touch interface.
                • If you don’t touch the screen the sound is off.
                • real-time sound variation via XY and Gyrospeed projection, depending on the case.

Score “Virtual Rhizome” : Download

“Virtual Rhizome” Technical Sheet : Download

Website of composer Vincent Raphaël Carinola : Link

Note: In partition the vocabulary used to describe the movements to be performed is dependent on an initialization position

transmission

Embedded Studies (2019), for Smart Hand Computer

Duration: Indeterminate

Created on April 11, 2019 at the Atheneum (Dijon) by ESM-BFC students in collaboration with Vincent Raphael CARINOLA, Jean GEOFFROY and Christophe LEBRETON

Embedded Studies are a set of pieces from the composition process developed for Virtual Rhizome (2018).

At the Smartgestes concert on April 11, 2019 at the Atheneum (Dijon) by ESM-BFC students, three different applications on three SHCs (smartphones) were used, for a version of Embedded Studies as a smartphone duo and a sHC duo version and electric bass.

WORKSHOP “Smart Hand Computer”

Public:

Any audience interested in new violins and musical approaches through gesture

Primary and secondary school students / Students of the Schools of Music and Dance, Conservatories, CRR, CRD, CNSM / Students of arts schools, Choreographic Centre, Choreographers, Plasticians, Scenographers…

Program: Embedded Studies

The aim of these pieces is to explore all the questions opened by digital violins, questions that concern both composition and interpretation.

The process at work for Embedded Studies is as follows:

  • Introducing the Smart Hand Computer developed by Christophe Lebreton and its use in Virtual Rhizome.
  • Making the instrument, starting from workshops around the sound recording, which are essentially workshops around instrumental gesture. The aim is to prepare a reservoir of samples from the sound universe specific to each participant.
  • Making apps for smartphones. These are based on the formal architecture of Virtual Rhizome
  • Interpretation/improvisation/composition workshops. This is the most important step, whereby participants will appropriate interfaces in order to develop a musical form that, depending on the case, can integrate other instruments and be articulated in several movements.

For more information, please contact : contact@lisilog.com