A pdf version of the user guide can be downloaded here

User guide

Author: Slim Ouni
Main programmer: Loïc Mangeonjean
Senior programmer: Ilef Ben Farhat

VisArtico V 0.9.9 - February 2015

Introduction

VisArtico is intended to visualize articulatory data acquired using an articulograph. It is intended for researchers that need to visualize the data acquired mainly from the articulograph with no excessive processing. It's also possible to use this software to display speech-related data. VisArtico is well adapted to the data acquired using the AG500 and AG501,, developed by Carstens (Carstens Medizinelektronik GmbH). Since the version 0.8, VisArtico supports the articulograph NDI Wave, developed by by Northern Digital Inc. Now, it is possible to read adhoc format, which allows displaying motion capture data for instance.

In addition to displaying the raw data, VisArtico provides additional information that helps to make more sense of the data. In fact, VisArtico can help to visualize the tongue shape, the lips, the jaw and the palate. This information is very useful to researchers working in the field of speech production, as phoneticians for instance.

VisArtico provides several possible views: (1) temporal view, (2) 3D spatial view and (3) 2D midsagittal view. In the temporal view, it is possible to display different articulatory trajectories in addition to the acoustic signal and eventually labels. The midsagittal view can display the tongue contour, the jaw, the lips and the palate, when this information is available.

VisArtico provides several tools to help to improve the quality of interpreting the data, in addition to animating the articulatory data synchronously with speech signal.

VisArtico is cross-platform software as it is developed in JAVA and does not need any external library or framework to be additionally installed. It was tested and worked on Windows, Mac OS, and Linux. It should work on any system having JAVA installed.

Terms of use

VisArtico is free for non-commercial and non-military use. We highly encourage academic to use it and we appreciate any feedback that can improve the future version.This software is provided as it is. It is highly recommended to backup your data and to work on a copy. We are not responsible of the loss of any of your data.

Articulatory Data Format

VisArtico was initially intended to visualize the data acquired by the articulograph AG500. For this reason, the application is using internally the data structure of the pos files generated by Calcpos (an application provided by Carstens, to compute the 5D positions from the raw data of the articulograph). Recently, Carstens released a new format generated by the articulograph AG501. VisArtico is now able to read .pos files written in this format. Moreover, VisArtico is able to read MOCHA-TIMIT EMA database files which are also generated by the articulograph AG500. These are 2D Data written in a format different from the Carstens format. And the same with the Mngu0 database which contains 5D data.

For all the previous formats, the data files should be organized as one of the 2 next organizations:

• (1) One folder containing only valid .pos files (Carstens) or .ema files (Mocha or Mngu0), (2) one folder denoted wav containing the audio files (wav format) and (3) if available, one folder denoted lab containing the segmentation files (.lab, .ali or .seg format). It is important to have only valid pos or ema files in the folder, as VisArtico usually uses all of them to compute midsagittal plan. If some folders contain corrupted or invalid files, the result can be completely wrong.

• One folder containing all of those files, the .pos/.ema, .wav and eventually the segmentation files.

We note that for each .pos/.ema file, the corresponding audio and segmentation files must have the same name so that VisArtico can be able to recognize them.

In addition, VisArtico allows importing NDI Wave data. The tsv files and the sound files (wav format) should be in the same folder.

VisArtico now also offers the possibility to export to all those different formats. So for exemple, if one user of MOCHA wants to share his data with a Carstens format user, he is now able to convert his data from his own format to the Carstens format. If the data is modified in VisArtico (using some filtering), the output file to be saved, can be in the same format, in another format with the constraint that the exported format channel number must be at least equal or higher than the imported one, or in plain text. We consider VisArtico support to NDI Wave format as Beta. In fact, all the implementation for the format is based on the information kindly provided by the technical stuff of Northern Digital Inc. We validated the format on a very limited number of file examples. We will appreciate if some NDI Wave users send us some of their data, or if they can share their experience with us regarding the NDI Wave file format.

File format for Carstens AG500 and AG501

VisArtico reads and writes the pos files of AG500 and AG501 articulographs. The pos files are binary files. The Carstens introduced the new format for the AG501. To differ between their formats, Carstens uses the V001 label for AG500, V002 and V003 for AG501. The format of the V001 file data is as follow :

• Sample: articulatory data for the current time stamp:
• Channel1, Channel2, ..., Channel12: The 12 sensors of the articulograph AG500. VisArtico accepts also higher number of sensors (16 and 24 for AG501, for instance). Each channel has the following information:
  • x, y, z , phi, theta: spatial position in 5D (3D + 2 angles).
  • Rms: the root mean square error.
  • Extra: still not used.

Each of these entries is coded using 4 bytes.

As for the V002 files, we have the same data organization as V001 files. The unique difference is that at the beginning of the file, there is a header as shows the next exemple:

The first line of the header starts with the signature “AG50xDATA$\_$V” followed by a 3-digit number (“002”) which is the file format version.

The second line contains an 8-digit number which is the full size of the header in bytes. It is at the same time the absolute file position of the binary data.

The V003 pos files format is almost the same as V002. The difference are that the channel number can go up to 24, as showing in the following example, and that we have extra information in the header that we don't use in VisArtico but that may be useful for other purposes.

The data in the pos folder (0001.pos, 0002.pos, XXXX.pos, etc.) is directly readable in VisArtico. When the pos folder, the wav folder and the lab folder are available, VisArtico reads the wave file and the segmentation file associated to the currently opened pos file from the wav and lab folder. If such folders don’t exist, VisArtico can still read the pos file. The user can specify at any time, a particular wav sound file or segmentation file to open it.

File format for MNGU0

VisArtico reads the .ema files obtained from the AG500 articulograph. These EMA data are stored in Edinburgh Speech Tools Trackfile format. They are binary files which also contain a header as shows the following example:

The channels having the label T1,T2 or T3 means that the corresponding sensors are glued on the tongue and those with the ref label are reference sensors which are usually glued behind the ears. After this header, we find the data organised as follows:

• px, py, pz, ox, oy, oz
• rms: the root mean square error
• newflag: still not used

Each of these entries is coded using 4 bytes in little endian.

This corpus is using 11 sensors and 200Hz as a frequency. It contains audio files (.wav) in a separate folder and segmentation files (.lab) in a separate folder too. VisArtico is able to load .ema files simultaneously with the corresponding .wav and .lab files. VisArtico offers also the possibility to save files in this format.

File format for MOCHA-TIMIT

This EMA database is also acquired via the AG500 articulograph and is also stored in Edinburgh Speech Tools Trackfile format. It's using 10 sensors and a sampling frequency equal to 500Hz. The same as the previous EMA databases, VisArtico is able to read and to write files in this format. The position files have a .ema extension. These files start with a header as the following example shows:



These EMA data are stored in Edinburgh Speech Tools Trackfile format consisting of a variable length ASCII header and a 4 byte float representation per channel. The first channel is a time value in seconds, the second value is always 1 (used to indicate if the sample is present or not) subsequent 5 values are coil 1-5 x-values followed by coil 1-5 y-values followed by coil 6-10 x-values and finally coils 6-10 y-values (see link)

Those are the different explanation for the channel flags :

• ui : upper incisor
• li : lower incisor
• ul : upper lip
• ll : lower lip
• tt : tong tip
• tb : tong bottom
• td : tong dorsum
• v : velum

This corpus contains also audio files in NIST format, and segmentation files. Even if those formats are different from what VisArtico was supporting in its latest version, now it is able to read those data.

File format for NDI Wave

VisArtico reads the tsv files NDI Wave articulograph. The tsv files are plain text files. VisArtico can read the tsv files but cannot write them back. The only possible output is either pos-files or plain text files. The format of the pos file data is as follow:

• WavID: sample number
• Time: in seconds (this is also WavID/22050)
• MeasID: --
• For each used sensor:
• Sensor n Id: the sensor Id where n is the sensor name.
• Sensor n Status: --
• X, Y, Z: sensor position
• Q0, Qx, Qy, Qz: sensor rotation, in quaternion format (Qz=0 for 5D sensors).

VisArtico extracts only the two angles, and doesn't use the full information that can be provided by the quaternion. For instance, for now, VisArtico doesn't handle the reference sensors used in NDI Wave. The full support of the NDI Wave is projected in future realises.

Motion capture data

In order to enhance Visartico usability, we recently added a function allowing the user to open a text file containing any kind of 3D data. This may be useful to display motion capture data if it's structured as above :

t₀ x₀ y₀ z₀ x₁ y₁ z₁x₂ y₂ z₂...x_n y_n z_n

t₀ is the sample time in milliseconds and x_i, y_i ,z_i are the coordinates of the vector i in a 3D space in cm.

General Description of the VisArtico GUI

Figures 1-5 show snapshots of the graphical interface of VisArtico.





























Getting Started

Installation and running

VisArtico comes in one single "executable" file. It is a JAR file. Almost on all the systems, you need just to double click on its icon to launch the software. That's it! In the latest version, we added a wrapper for two operating systems: Mac OS and Windows.

Open your first file

This section will tell you how to open your first ema, audio or segmentation file

Audio and segmentation data

Before talking about articulatory data, let's start with audio. VisArtico is not only a tool for visualizing articulatory data, it is also a software allowing the user to open an audio file even without an EMA data. For this, the user should select File > Open audio file. A file chooser will appear allowing him to select an audio file. After that, the file will be visible in the temporal view. You can fix the audio file size with your screen size by pressing CTRL+R.

Once your audio file is there, you can press the play button to read it. You can also open a segmentation file that corresponds to this audio file by choosing Open a segmentation file in the menu segmentation.

Articulatory data

At the first run, probably it is a good idea to configure VisArtico to use either AG500/AG501 or NDI Wave File. To do this, check in the menu "Preferences" and you can choose an articulograph or add a new one. In this same menu you can configure the language of VisArtico (for now, it supports English and French).
Even if you don't do this step, VisArtico is capable of recognizing and adjusting the configuration to the right articulograph. So, usually, you don't have to do much to make it work... so just launch VisArtico and let's start!!
VisArtico can open any pos file or tsv file or ema file. The files should respect the correct data format. For AG500/AG501, the data is expected to be organized in two folders:
pos: this folder contains all the pos files (the actual articulatory data)
wav: this folder contains all the wave files associated to the pos files of the folder pos.
lab: this folder contains all the segmentation files associated to the position files of the pos folder



It is also possible de put all the files (position, wav and lab) in the same main folder.

For the NDI Wave, the data is expected to be in the same folder for both articulatory data (tsv files) and acoustic data (wav files).
If you are using both articulographs, it is highly recommended to not mix all the data together. Some functionalities of VisArtico is using the folder structure to optimize its use.


To open a file, select in the menu File > Open experience file. Then select the file you want to open.




After selecting the position file to open, a warning dialog box tells you that there is no sensors configuration associated to this articulatory data file. The configuration step is important to make more sense of the data and be able to fully use the midsagittal view. Thus, you probably need to hit yes button to make a new configuration file. If you don't want to use the midsagittal view, you can ignore the configuration step. You can always make the configuration later, by selecting in the menu Experiment > Sensors configuration.

Sensor Configurations

To be able to interpret the articulatory data, the midsagittal view is very helpful. However, VisArtico needs to know how to associate sensors to a particular articulator. For instance, one needs to know what are the sensors associated to the tongue, or to the lips, etc. You can access and modify at any time the configuration panel by selecting in the menu Experiment > Sensors configuration. The configuration panel is showed below:











Configuring the sensors

You can use the 3D view to have a better idea of the position of each sensor. Then, in configuration panel, you have to drag the sensor (by its number) and drop it on the appropriate place on the vocal tract (the possible positions are highlighted in red). Some positions are represented by a region. This means either you can specify exactly the position of the sensors (this the case for the lips), or you can specify the order of the sensors in that region (the case for the tongue). In fact, for the tongue, the highlighted region helps to specify the order of the tongue sensors (to generate the connecting lines). The positions where you put the sensors will not be respected, as VisArtico will reproduce the real measurements. You can drag and drop the sensors in the positions indicated in the vocal tract image, from either the 3D view, the midsagittal view or the configuration panel itself.
The configuration tool provides by default names to the sensors depending on their position in the vocal tract. These names can be modified in the configuration panel. You have just to double click on the current name, and modify it.




Mid-sagittal plane detection

It is possible that the midsagittal plane of the recorded subject doesn't coincide with the reference axes of the articulograph.For this reason, VisArtico try to find the midsagittal plane using some sensors. The sensors attached to the tongue (if they have been put in the middle of the tongue) will be used to determine the midsagittal plane, and the data will projected in this plane. Thus, The 2D View (midsagittal view) shows the projection of the data. By default, in the sensor configuration panel, VisArtico computes automatically the midsagittal plane relative to the sensor attached to the tongue (the check button "detect" is checked by default. By unchecking this button, you can modify the orientation of the had as you want. To do so, you just have to rotate one of the heads according the appropriate plane.




After finishing this step, the 2D view (midsagittal view) should display something similar to the following drawing (except for the palate tracing, that will be presented later):









Now, you are ready to visualize your data. Just hit the play button, and the different articulators will be animated simultaneously.





VisArtico Controls

The three different views can be controlled using the mouse. The right click shows the context menu specific to each view.

2D View (midsagittal view)

The possible actions on this view are mainly:

• Moving the front-view of the lips window.
• Moving the tongue-jaw-lips (for a better visualization)
These two actions can be done by pressing the mouse left button and moving it around.

• Resizing the view: by scrolling the mouse wheel it is possible to resize the tongue-jaw and the lips.
• Showing all the sensors (context menu)
• Modifying the interpolation mode of the tongue and the thickness of the curve. By default, bicubic interpolation is used. This can be changed to linear.
• Displaying the palate when available.
• Displaying or hiding some elements of the view (lips, sensor numbers, scale, etc.)
• Inserting an image in the 2D view background.

This functionality was recently added in VisArtico and it may be useful for people combining EMA data with MRI for example. It allows them to overly EMA data on MRI. This function is accessible via the pop up menu of the 2D View. So you just right click on the 2D view and choose "Insert image". A file chooser will appear allowing you to choose an image having one of the following extensions (bmp, jpg, jpeg, wbmp, png, gif). Once you choose your image, you click open and that's it. You will see your chosen picture on the 2D view with a new tool panel on the bottom of the view, as shown in the figure. You are now in edition mode, this means that the only thing you can control is the image.



VisArtico allows you to make some modifications on your imported picture via the tool panel shown in the figure 9.



As the figure 9 shows, the tool panel enable the user to :

• [1] Resize the Image
• [2] Flip it horizontally
• [3] Control the brightness
• [4] Apply filter color
• [5] Rotate it from 0 to 360 degrees
• [6] Flip it vertically
• [7] Control the transparency
• [8] Remove all the modifications
• [9] Remove the image and quit the edition mode
• [10] Save all the applied changes and quit the edition mode




We note that the image is controlled via the mouse in the same way the 2D view is controlled but in an exclusive way. So you can move it by draging or zoom it by scrolling the mouse. You can also control the size from the corners. Moreover, you can rotate the image by double clicking on it and moving your mouse as shown in the figure 10. Once you finish your modifications, you can save them by clicking on the component number 10 in the figure 9. This will be saved even for the next time you open the same experience in VisArtico. After clicking, you will control again the 2D view without controlling the image, but you can always recontrol you image by choosing "edit image" in the pop up menu.




N.B. In the 2D view, some animation effects were added (this usually happens when changing the size, or displaying all the sensors). This effect can be disabled by checking in the context menu "view blocked".


It is possible to detach this view and make it in a separate window. This is possible for the 2D, 3D and the temporal view windows. Closing the window will put it back in the main program window. It is always possible to change the size of each window.

3D View

The possible actions on this view are:

• Zooming : by scrolling the mouse wheel.
• Rotating : pressing the left mouse button and moving the mouse.
• Moving right and left (using right and left arrows of the keyboard)
• Displaying the palate when available
• Showing the traces for each sensor. When only an interval is selected (in the temporal view), the traces belonging to this segment only will be displayed.



Extra Features

In the 3D view, it is possible to associate different sensors together. For instance, in case the midsagittal view has no real sense for a particular acquisition, it is possible to use the 3D view directly and display some segments in 3D. To do this, you need to drag a given sensor and drop it on another sensor that you want to connect to the first one. To remove a link, you can use Undo in the menu Edit.



Temporal View

The temporal view allows displaying the trajectories of the different sensors across the different component. To display a trajectory of a component of a given sensor, you have do drag the sensor from the 2D view or the 3D view and drop it to the left panel of the temporal view. Then a dialog box will appear to specify which component you want to display. It is possible to add several trajectories at once.
VisArtico allows displaying all the trajectories together in one panel, but it is possible to organize them in different panels. To add a new panel, click on Add button, on the top left temporal view. The new panel will be added to the bottom. It is very easy to move a trajectory from one panel to another. You have just to drag the colored rectangle (legend of the trajectory) to the targeted panel.
To remove a trajectory, right click on the colored rectangle associated to the trajectory to be removed and choose in the context menu Remove. If there is no trajectory in the panel, the panel will be removed.
It is possible to add a trajectory, by right clicking on left panel of temporal view, and select Add Curve.
Another way to add all the sensors for a given component is to select in the main menu, Curves > Add Curves. All the trajectories will be added to a new panel.

Derivatives

Visartico allows displaying the trajectories of the speed and acceleration. Actually it is possible to do as many derivatives as one wants. The best way to do this is to add the curve of the sensor we are interested in its derivative (a good way is to duplicate an existing one), and right click (context menu) on the legend and choose "derive".

Selecting and Zooming

It is possible to select a temporal segment in the temporal view to apply some processing on it. For instance, one can select a segment to smooth, or for which one wants to display the traces for the selected segment (in the 3d view). It is possible also to play (animate) the selected segment. Selecting a segment is done by right click and dragging the mouse in the temporal view. The selected area will be highlighted in blue.
While selecting a segment , an information window provides some information about the selection (start and end time, duration).




When pressing Ctrl + w (or ⌘ + w), the selected area will be zoomed in. To go back to the full trajectory, hit Ctrl + r (or ⌘ + r).

It is also possible to zoom by selecting the temporal range.




Finally, it is also possible to zoom in by pressing (+), or to zoom out by pressing (-), where the zooming factor is a multiple of 2.

Acoustic View

By default, VisArtico displays simple acoustic signal. However it is possible to switch between acoustic signal view and spectrogram view. To switch between the two views, just hit the toggle button on left side of the temporal panel :

The acoustic panel can be resized to have a better view of the spectrogram (the cursor changes to ↕ when approaching the top edge of the acoustic panel). As for any element of the temporal window, it is possible to zoom in/out the spectrogram.
Note on the bottom, there is a small window that shows a spectral slice, of the current cursor position. This view is just for illustration and intended, for now, to provide very precise numerical value. Each tick represents 1kHz, in the frequency axis. The two long horizontal line, delimit the area where the spectrum is visible: below the lower horizontal line, all the amplitude are zeros (not visible in the spectrogram), and the top horizontal line represents the amplitude limit of the spectrogram.



Segmentation

It is possible to create, read and modify a segmentation file in VisArtico. It is possible to display several levels of labeling (for instance, one for phonemes, one for words, etc.).

Reading a segmentation file

VisArtico offers the possibility to manipulate segmentation files with or without EMA data. Thus, it can be used as a segmentation tool.

It is possible to read a segmentation file made by other software as WinSnorri and Praat. VisArtico can read the ali format (Winsnorri), textGrid format (Praat) and lab format (x-lab). We also added a simple format seg (each line has : start_time end_time label).
To open a file, in menu Segmentation ► Open segmentation file , and choose the right format. If the length of the segmentation file mismatches audio/EMA file length, VisArtico will not open it and gives an error message.
When labels are displayed it is possible to play one or more labels together. You have just to select the corresponding labels in the label strip.



Saving a segmentation file

To save a segmentation file, you can proceed in two ways. If you have one single label level, you can either save it using the menu Segmentation, or click on the button .
If you have multiple label levels, you can save them together using TextGrid format (menu: Segmentation► Save segmentation). You can also save them level by level in one the supported formats in VisArtico. In this case, you should save it by clicking on the button for each label level.

Creating a segmentation file

To create a segmentation file, you can add label levels, one by one, using the menu Segmentation and ► Add a segmentation. A label level (that you can give it a name, mainly for TextGrid format) that is presented by an empty strip is added, with a unique label "UKN". When the cursor is on the segmentation strip, all the interaction will be captured by the strip. This means, for instance, if you left click the current label segment will be selected, if you keyboard-type, the current label will capture keys, etc.
When Shift + left click on the strip will add a boundary at the position of the cursor (you can use the keyboard and hit INSERT). You have just to type with the keyboard to put your own label. It is that simple.
To remove a segment, just put the cursor on that segment, and hit SUPPR or right click on the segment and choose REMOVE.
To modify the boundaries of a segment, just drag one boundary with the cursor, by pressing the left mouse button, and dragging it.


N.B. When the cursor is on the segmentation strip, when you type the current segment will capture all the typed keys. For instance, if you want to play the segment by hitting SPACE BAR, you should move the cursor outside the segmentation area and hit SPACE BAR.

After trying for few times, you will see that this technique of segmentation is very natural and easy to perform.

It is possible, to display phonetic symbols, as VisArtico handles UNICODE correctly. When you want to insert a phonetic symbol, double-click on the label, and a virtual keyboard will appear. This keyboard allows inserting almost a large variety of phonetic symbols.

When you are using more than one label level, the boundaries are "sticky". This means when you are adjusting the boundary of one segment, this one try to stick to the closest boundary of the other label level, to have the same time stamp. We made this choice, because the label levels are usually connected. For instance one boundary of a word is very likely to be the same boundary of a phoneme, and so on. This helps to have exactly the same time value for these boundaries.



Graphic, Text and Video Export

VisArtico allows to export graphic files and text file of the data.The graphic file is a vector-graphic file of the 2D View or the 3D view. It is possible to add some titles to the graphic. The text file is a selection of all or part of the data to export in text readable format.

Graphic Export

It is possible to export the 3D view and the 2D View, a in vector-graphic format, EPS, encapsulated postscript. The export module copy exactly what in the 3D or 2D view. So before exporting the graphic, you have to adjust the view to match exactly what you want to have in the graphic file (you can rotate, translate the content of a view ; display or hide a part of the view: for instance if the palate, front view, scale, etc. are not needed, you can remove them, by right-click on de/select the corresponding feature)
To export a graphic, you choose in the menu Export ► Graphic and then you choose the view. A new window will open and has exactly what will be exported. You cannot change the position of the elements of this window. If this is not what you want, you have to close the window and go back to the view and re-organize the content.
Now, you can just hit the Export button, and you choose the EPS file name. It is that simple!


Adding title and labels to the graphic

VisArtico allows adding titles and labels in the graphic window. Choose any position in the graphic window, and double click, a text area will appear and you can enter your text. You can add as many labels as you want, using the same technique, double click anywhere in the same window. You can change the position of any label, but dragging it. You can notice that some areas in the window are sticky: the label wants to stick there, because it is the center either horizontally or vertically.
It is possible to change the size of the text font, by using the mouse wheel.






Text Export

Sometimes it is handy to be able to export in text format some of the examined data. For instance, after doing some filtering, one might want to examine only 2 or 3 sensors, and let's say, X and Y axis. In this case, exporting only the needed data will be very useful.
VisArtico allows such text exportation in an intelligent way. In fact, The exportation is very flexible.
It is possible to export the whole articulatory data, or only the current selection.
To export, select in the menu Export ► Text and then choose between all or selection (in the latter, we export only the data in the selected segment). The Export window will open. By default, VisArtico suggest to export everything. However, it is possible to format the output and export only what is needed. For this reason, we explain the export format.


Text Export Format

In the field Format, you can enter the format of the output. The formatting applies on each line, and thus on each data entry (frame or time). The items are :

[sample] current sample (or frame) number.
• [time] current time stamp (corresponding to the current sample)
• [sensor:1-12:X,Y,Z,PHI,THETA,RMS] the articulatory data. if you add ' to any if these components, you will get the derivative information. for instance X' is the first derivative of X, X'' is the second derivative of X, etc.
  • sensor:a-b indicates that the exportation concerns the sensors a to b. It is possible to select non-consecutive sensors. For instance sensor:2,6,9.
  • X,Y,Z,PHI,THETA,RMS are the position of the sensor in 5D.
• [segmentation:1-2] if you already opened a segmentation file, the labels can be added to the output. for each time stamp the label text will be added only if it corresponds the this time stamp. It is possible to add one level or more. [segmentation:1] means add only the first label level.
• [projection:1,2,5:x,y] the midsagittal plane coordinates for the indicated sensors. It is the projection of the 3D sensor coordinates in the 2D midsagittal plane. It is possible to add the derivatives.

It is possible to add some other text to this formatting.
[sample][time][sensor:1-12:X,Y,Z,PHI,THETA,RMS,EXTRA][segmentation:1-2]

Some examples:
(1) Time: [time] Data: [sensor:1,4:X,Z]
Add the text "Time" and "Data" to each line. export only X and Z for sensor 1 and 4

An excerpt of the output:
Time: 0.0000 Data: 35.5379 1.9320 17.1804 11.7190
Time: 0.0050 Data: 35.5209 1.8326 17.1615 11.6910
Time: 0.0100 Data: 35.4022 1.5417 17.2025 11.6681
Time: 0.0150 Data: 35.4221 1.3275 17.2605 11.5507


(2) [sensor:2,5:X,X'] ** [sample]

An excerpt of the output:
-53.3663 0.5058 57.1491 -7.9094 ** 0
-53.3688 7.7805 57.1887 0.4211 ** 1
-53.4077 -1.0264 57.1866 -20.5505 ** 2
-53.3426 8.8120 57.2893 -13.0074 ** 3


Video Export

VisArtico allows the user to export the 3D and the 2D animation to a video file.

FFMPEG

In order to be able to generate video, VisArtico needs ffmpeg to be installed on the user machine. VisArtico is able to create MPEG, MP4 and AVI video files unless the corresponding encoders are not installed. For MPEG format, VisArtico uses the mpeg1video package, libx264 for MP4 format and mpeg4 for AVI.

If ffmpeg is already installed in your machine, VisArtico will make a first try to look for it's path and you will find the ffmpeg path stored in File > Preferences. As shown in the figure, you can also specify it by yourself to VisArtico by filling the textfield by the right ffmpeg path. VisArtico will use it to generate video and it will be stored for next use.



Video settings

Once your ffmpeg path is fixed, like in the graphic export, before exporting the animation, you should adjust the view to match exactly what you want to have in the video (Dimension, zooming, orientation, background, landmark...).

To export to video, you choose in the menu Export > Video and then you choose the view. A new window will open allowing you to choose the name of your video and the place where to store it as shown in the figure.



This view allows you also to choose to export either all the current animation or just a selection that you can limit via the spinners BEGIN and END or by the right click mouse selection.

Labels

If your experience contains segmentations files, it's possible also to add the corresponding labels to your video. For this, you just need to select the check box "add labels and titles". By default, VisArtico will place these labels in the bottom of the exported view. Labels will be written in black, with a light white background, in "SANSSERIF" font and having 18 as font size. Moreover, VisArtico offers you a tool to configure your labels accessible via the button "Configure...".

Once this button is clicked, a new window will appear showing you the view that is going to be exported, with all the labels placed by default. This window offers you the possibility to place those labels wherever you want them to appear in the video by dragging them to the place you want. For each label you can :

• Fix the font size by using the spinner or by the mouse wheel
• Choose the font from the combo box items
• Choose the foreground color by the button "A"
• Choose the background color by the button "BGD"
• Fix the background opacity via the slider in "Opacity" section



For this, you need just to select the label, it will appear into a white rectangle and then adjust its parameters. You can also apply the same parameters for 2 or more labels. For this, you should maintain the ctrl button down and select the labels you want to configure.

You can also remove a label by right clicking on it and choosing "remove" in the pop-up menu. You can return it back either by choosing "add label" in the pop-up menu or just via the short cut "ctrl+L".

Titles

Whether your experience contains labels or not, you can add titles to your video view. For this, once the configuration view is open you can add titles by choosing the item "add title" in the pop-up menu. You also use the short cut "ctrl+T". A new text field will appear in the view and then you can enter the title (text) that you want and commit it by pressing enter key.

Like for labels, you can also adjust your title parameters thanks to the configuration panel on the right of the view. But for titles, you can't change the background color and the slider controls the foreground opacity.

A selected title is surrounded by a dashed rectangle. You can also use the multiple selection by maintaining the ctrl button down. It's possible also to remove a title by right clicking on it and choosing the item "remove the title" in the pop-up menu. The button Reset puts all the labels in default configuration and removes all the titles.






Saving Video Configuration

Once you finish configuring your labels and titles, you need to click the button "Apply" so that this configuration will be saved and used for your video export.

Clicking again on the button "Configure..." will return you back to the label configuration view with the configuration that you have just saved. You can change it again if you want to change something.




Then, you need to choose an export format for your video within the available formats and finally click "Save".

A dialog window will appear showing you the video export progress. If ffmpeg faces a problem, the export will be interrupted, else you new dialog window will appear telling that the video export was successful.

And now, congratulations! you have your video. you just need to look for it in the folder where you saved it and use a video player to see the result !

The following video is an example generated by VisArtico with 3 levels of segmentation : phonemes, words and sentences.




Filtering

It is possible to filter some of the trajectory to remove noise for example. It is possible to use a low-pass FFT filter, where you provide a cutoff frequency. We provide also a fast smoothing filter that uses low-pass RC filter. We highly encourage the use of the first filter (Low-pass FFT filter). It is possible to remove the part of the trajectory where the RMS is higher than a given value. The points removed will be replaced using a linear interpolation.
You can select the sensors and the components to filter. If you already selected a segment in the temporal view, the filter panel will apply the filtering on this segment (unless, you uncheck the button on the top of the window). It is possible to apply the filtering on a list of files, that you already added to the playlist (see Play List).

N.B. VisArtico never modifies the original data in your disk. So if you did some filtering, if you do not export the articulatory data (File ► Export) in a pos file, for instance, the modification will be lost when you exit VisArtico.

Palate Detection

The visualization of the palate allows a better interpretation of the tongue movement. It helps, in particular, to better seeing the places of constriction. During the acquisition by an articulograph, usually one of the sessions is used to record the contour of the palate. This is done by drawing by hand the contour of the palate using a sensor. VisArtico can then exploit this session to extract the contour of the palate and used with other articulatory data from other sessions.
Under the menu Experiment ► Palate Shape, on the left side of the window, select the session that was used to record the contour of the palate. Then, under the name text area, check the sensor number, used to draw the palate. That's it. Now Visartico will use this information, to find the palate shape. Another window will open, and allows the user to adjust the contour length that will be used.
As the values of the contour are real measured values from several sweeps, the shape might not be smooth. After accepting the shape, you can smooth the palate contour using the filter that appears in the bottom of the window.







Sometimes, it may happen that the palate contour data are not available, for some reason. The software can then give an approximation of the contour of the palate using a simple but effective algorithm that we developed which predicts the contour from the positions of the sensors of the tongue. For each time frame, first we determine the contour of the tongue from the sensors associated to it. We obtain several contours of the tongue achieved throughout all the recorded sessions. Therefore, the algorithm recovers the maximum position in the midsagittal plane of each contour, which should correspond to a contact between the tongue and the palate. The more sessions, the greater the palate shape is realistic. This solution allows to approximate the actual shape of the palate.

To detect automatically the palate contour, select the files (the sweeps, experiences) to be used to compute the palate contour.You should select only valid files without any errors, otherwise the palate shape will be wrong. Then, press the button "compute". After few seconds, a new window appears and shows the shape of the plate. You can use the left and right cursor to determine the part of the contour to accept and display.
As the values of the contour are real measured values from several sweeps, the shape might not be smooth. After accepting the shape, you can smooth the palate contour using the filter that appears in the bottom of the window.

Important 1. The use of this feature should be done very carefully. The algorithm cannot verify if the values of the sensors are valid. So if there are errors in the coordinates of some the tongue sensors, the shape of the predicted palate will obviously be completely wrong. Thus, the method to follow to get the most realistic palate shape is to use only the sessions that has no errors in any of the tongue sensors. You can check this by going through all the sessions, and display the traces in the 3D view. If there are outliers in one of the tongue sensors, the file should not be used to detect the palate contour.

Important 2. The palate contour is computed on the midsagittal plane. So, if the midsagittal plane is not correct, the palate contour evidently will not be correct. If for some reason you changed the midsagittal plane orientation, you should recompute the palate contour.







Audio-Articulatory Animation

The animation control panel is very easy to apprehend. The buttons and allows playing the audio simultenousely while animating the 2D and 3D Views. When selecting a segment, the play button will play only the selection.
It is possible to change the volume of the audio by sliding the rainbow triangle , and to mute the audio by pressing ► . It is also possible to play faster or slower by clicking on speed percentage (the text "100%" in the figure) and modify the position of the slider.
Clicking on or allows go to the following or previous frame.




Play List

The playlist is used to open several EMA data files and apply some processing if needed. To fill the playlist, you can either drag and drop the EMA files from a folder the playlist window or press the button "Add" and select the files. To open the playlist window, use the file menu.
You can double click any file from the playlist to open it in VisArtico. You can go cycle through the files, using the animation control buttons .
It is possible to apply the filtering on all the files in the playlist. This is very handy to not open file by file and apply the same process. To export all files at once use the button "Export all" (the export in the File menu exports only the current file.)