Marius Comper
ROEN

A listening game · six Romance languages

What is left of a language without its consonants: The same sentence in Romanian and five sister languages, reduced to melody, rhythm or vowels

Each sentence was translated and read by native speakers of Romanian (the Romance language of Romania and Moldova), Italian, Spanish, Portuguese, Catalan and French. Here you hear it through a sieve that lets one thing through: the melody of the voice, the rhythm of the syllables, or the vowels. Guess the language, then hear the real voice. After 12 sentences you see which sieve helped you.

The real voice12 sentences · four sieves · six languages
Listen
You need sound; headphones help.

What does each sieve let through?

Every version is voiced by the same synthetic voice, built from the real recording. The speaker’s timbre, the room and the microphone are gone. Women’s and men’s voices are brought to the same pitch and, roughly, the same vocal-tract length, because in the corpus some languages are read only by men and others only by women. The sieves differ only in what they let through.

The idea comes from psycholinguistics labs. In 1999 Franck Ramus and Jacques Mehler resynthesised English and Japanese sentences, keeping only some cues. Syllable rhythm alone was enough for French listeners to tell the two languages apart. But English and Japanese are very different. Between Romance languages, which resemble each other, the task is probably much harder.

In the drawing, colour shows the vowel by where the mouth makes it. Vowels from the middle of the mouth, like Romanian ă and î, come out grey:

Example: the start of a sentence in Romanian.

1

Melody only

The rise and fall of the voice, hummed. No vowels, no consonants.

2

Rhythm only

Every vowel becomes “a”, every consonant “s”, on one flat note. Only the durations remain.

3

The vowels

The real vowels at their real lengths, on a flat note. The consonants are silent.

4

Vowels without rhythm

The same vowels, each one the same length, with equal gaps between them. Pauses stay.

How does the same sentence sound in all six languages?

Pick one of the 24 sentences and hear it through each sieve, language by language. Rows are languages, columns are sieves; the last column is the real voice.

melodyrhythmvowelsno rhythmthe real voiceRomanianItalianSpanishPortugueseCatalanFrench

Why, to a French ear, does Romanian sound more like Portuguese than Italian?

In 2000 Ioana Vasilescu, François Pellegrino and Jean-Marie Hombert, of the Dynamique du Langage laboratory in Lyon, asked four groups of 20 listeners (French, Romanian, Japanese and American) whether two six-second excerpts were in the same language. The languages were French, Italian, Spanish, Portuguese and Romanian.

The Japanese listeners answered close to chance. The French separated Italian and Spanish, which were more familiar to them, from Portuguese and Romanian, which they could not tell apart after a few seconds of familiarisation.

The authors link the grouping to the vowel systems. Italian and Spanish have front and back vowels. Romanian, French and Portuguese add a third series: the central vowels ă and î in Romanian, nasal vowels in French and Portuguese, front rounded vowels in French. To the Romanian listeners, Spanish and Portuguese seemed more familiar than French and Italian; the authors mention South American films shown on Romanian television.

In the game, the “Vowels” sieve passes part of this cue: the central and the front rounded vowels, but not nasality. Central vowels appear in the drawing as grey blocks.

Is Romanian a syllable-timed language?

Linguists long sorted languages into two rhythm families: “syllable-timed”, where syllables seem equal (Spanish, Italian, French), and “stress-timed”, where stresses come at roughly regular intervals and the syllables between them squeeze together (English, Dutch). Measurements never confirmed a clean split, and many indicators followed; three of the simplest are in the table. %V is the share of time filled by vowels. ΔC shows how much the length of consonant clusters varies: languages that mix long and short clusters have a high ΔC. VarcoV shows how much vowel durations differ from one another, relative to their average.

How much of the time is vowels, in each of the six languagesEach dot is one recording. The vertical line is the language median; the triangles mark each reader’s median, with voices told apart by timbre (readers with fewer than five recordings are not shown).30%40%50%60%70%share of time taken by vowels (%V)Italian51.0%Spanish48.6%Portuguese47.2%French44.9%Romanian43.4%Catalan41.9%
Each dot is one recording. The vertical line is the language median; the triangles mark each reader’s median, with voices told apart by timbre (readers with fewer than five recordings are not shown).
language%VΔC (ms)VarcoVrecordings
Italian51.05651107
Spanish48.65349119
Portuguese47.25760107
French44.9595382
Romanian43.45550110
Catalan41.96452112

On the same 40 sentences read in all six languages, 637 recordings in all, the measures were computed the same way, from the same automatically drawn vowel–consonant boundaries. In Romanian, vowels fill 43.4% of the time: less than Italian (51.0%), Spanish (48.6%) and Portuguese (47.2%), about as much as French (44.9%) and a little more than Catalan (41.9%). Compared sentence by sentence, the 95% interval of the gap to Italian, Spanish and Portuguese stays below zero.

But each language is read here by a few people: by timbre, the voices group into about three readers per language, and the readers differ from one another as much as the languages do. The three Romanian readers, all men, have 47.5%, 43.3% and 38.1%. The Italian women are around 51%, but the only Italian man, with six sentences, has 42.8%, like the Romanians. The two French men have 32.8% and 50.7%. With three readers per language, this corpus cannot say where Romanian rhythm sits; it can only say that people read very differently. Mairano and Romano also note that these indicators vary a great deal from speaker to speaker.

The Vasilescu study cited above puts Romanian beside Italian among the syllable-timed languages, following a 1998 classification. The data here neither confirm nor refute it. By ΔC (55 milliseconds in Romanian, 56 in Italian), consonant clusters vary almost equally in the two languages.

Absolute values depend on how the boundaries between sounds are drawn, so they cannot be compared directly with other studies’ figures. The order of languages measured the same way is safer than any single number. Analysing the fable “The North Wind and the Sun” read in 21 languages, Paolo Mairano and Antonio Romano found Romanian near the languages thought of as syllable-timed on the PVI measures, and noted that such indicators depend heavily on tempo, style and how speech is segmented.

How the sounds were made

The recordings come from FLEURS, a corpus published in 2022 by researchers at Google: sentences from English Wikipedia, translated into 101 other languages and read by native speakers. The Spanish is Latin American and the Portuguese is Brazilian. Of the sentences read in all six languages in the development and test sets, 141 contain no digits and no capitalised words after the first (usually proper names, which would give the language away). Of those, the 24 with the cleanest recordings in every language were kept. You hear the opening of each, roughly two to three and a half seconds. The rhythm measurements use all 40 sentences with the cleanest recordings, not only the game’s.

Each sentence’s text was transcribed into phonemes with eSpeak NG; a phoneme recogniser trained on many languages (wav2vec 2.0 XLSR-53) then aligned it to the audio. The model finds every sound, in order, but not how long it lasts. The vowel–consonant boundary was placed where the sound stops resembling a vowel: voiced, loud, and with more energy below about 1,000 hertz than above 3,500. Before any measurement, background noise was reduced the same way in every language, because the Romanian recordings are noisier.

In the vowels cut out this way, the pitch of the voice and the first three formants, the resonances that give a vowel its colour, were measured. The new sounds are synthesised from these measurements: a harmonic source passed through resonators that follow the formants. “Rhythm only” keeps the durations but puts an “a” and an “s” everywhere, on a flat note; “Melody only” keeps the shape of the pitch curve and the loudness, hummed, moved to the same register for every speaker and filtered below 600 hertz. “Vowels without rhythm” makes every vowel and every gap between them the same length; pauses keep their duration.

Limits: a very short vowel can be lost or merged with its neighbour. The synthesis does not reproduce the nasality of French and Portuguese vowels, only their formants. The FLEURS tables do not name the speaker; by timbre, each language seems to be read by about three people, so the same voice recurs across sentences. In the game’s sentences, the Romanian and Portuguese ones are read only by men and the Catalan and Italian ones only by women. Bringing voices to one register and one vocal-tract length is approximate: in Spanish, where the game’s sentences are read by both men and women, the median second formant stays about 8% higher for women. Answers in the game are not sent anywhere; only you see your score.

Sources

  1. Conneau, A. et al. (2022). FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech. arXiv:2205.12446. Recordings licensed CC BY 4.0. arxiv.org/abs/2205.12446
  2. The google/fleurs dataset, Hugging Face. huggingface.co/datasets/google/fleurs
  3. Vasilescu, I., Pellegrino, F. and Hombert, J.-M. (2000). Perceptual features for the identification of Romance languages. ICSLP 2000, Beijing, 543–546. www.isca-archive.org/icslp_2000/vasilescu00_icslp.pdf
  4. Ramus, F. and Mehler, J. (1999). Language identification with suprasegmental cues: a study based on speech resynthesis. Journal of the Acoustical Society of America 105(1), 512–521. doi.org/10.1121/1.424522
  5. Ramus, F., Nespor, M. and Mehler, J. (1999). Correlates of linguistic rhythm in the speech signal. Cognition 73(3), 265–292. doi.org/10.1016/S0010-0277(99)00058-X
  6. Mairano, P. and Romano, A. (2011). Rhythm metrics for 21 languages. ICPhS XVII, Hong Kong, 1318–1321. www.internationalphoneticassociation.org/icphs-proceedings/ICPhS2011/OnlineProceedings/RegularSession/Mairano/Mairano.pdf
  7. Xu, Q., Baevski, A. and Auli, M. (2021). Simple and effective zero-shot cross-lingual phoneme recognition. The wav2vec2-xlsr-53-espeak-cv-ft model. huggingface.co/facebook/wav2vec2-xlsr-53-espeak-cv-ft
  8. eSpeak NG, speech synthesiser and phonetic transcriber. github.com/espeak-ng/espeak-ng